Copy cookies from one WebClient to another?

Question

I wonder if it's possible to copy cookies from one web client to another.

Reason

I am using parallel web requests and it creates new instances of web client on every new thread.

Problem

Information is sensitive and requires authorization using post request and saves cookie. So basically those new web client instances can't access. I don't want to authorize every single web client that is being created so I wonder if it's possible to somehow just copy the cookies from one web client to another.

Example Code

public class Scrapper
{
    CookieAwareWebClient _web = new CookieAwareWebClient();

    public Scrapper(string username, string password)
    {
        this.Authorize(username, password); // This sends correct post request and then it sets the cookie on _web
    }

    public string DowloadSomeData(int pages)
    {
        string someInformation = string.Empty;

        Parallel.For(0, pages, i =>
        {
            // Cookie is set on "_web", need to copy it to "web"
            var web = new CookieAwareWebClient(); // No authorization cookie here
            html = web.DownloadString("http://example.com/"); // Can't access this page without cookie

            someInformation += this.GetSomeInformation(html)
        });

        return someInformation;
    }
}

// This is cookie aware web client that I use
class CookieAwareWebClient : WebClient
{
    private CookieContainer cookie = new CookieContainer();

    protected override WebRequest GetWebRequest(Uri address)
    {
        WebRequest request = base.GetWebRequest(address);
        if (request is HttpWebRequest)
        {
            (request as HttpWebRequest).CookieContainer = cookie;
        }
        return request;
    }
}

Answer 1

I believe you can share CookieContainer instances between WebClient objects. So once you've authenticated, re-use the same CookieContainer for each new client you create. Just take care that your subsequent requests are not modifying the CookieContainer, otherwise you may have a race condition because I doubt the class is thread-safe for parallel modifications.

First, modify CookieAwareWebClient with a custom constructor that can have a cookie container passed in. Also, provide a way to get the container reference via a property:

class CookieAwareWebClient : WebClient
{
    private CookieContainer cookie;

    public CookieContainer Cookie { get { return cookie; } }

    public CookieAwareWebClient() {
        cookie = new CookieContainer();
    }

    public CookieAwareWebClient(CookieContainer givenContainer) {
        cookie = givenContainer;
    }
}

Then your Scrapper class should pass in its own CookieContainer to each client after authentication:

public string DowloadSomeData(int pages)
{
    string someInformation = string.Empty;
    CookieContainer cookie = this._web.Cookie;

    Parallel.For(0, pages, i =>
    {
        // pass in the auth'ed cookie
        var web = new CookieAwareWebClient(cookie);
        html = web.DownloadString("http://example.com/");

        someInformation += this.GetSomeInformation(html)
    });

    return someInformation;
}