differentiate among multiple webclient results

Question

There is a List.

I want to download each url via webclient.DownloadStringAsync

the problem I encounter is: how do I know which e.Result corresponds to what url ?

public class ressource{
public string url { get; set; }
public string result  { get; set; }
}

List<ressource> urlist = new List<ressource>();
urlist.Add(new ressource(){url="blabla", result=string.empty});
....etc

var wc= new WebClient();
foreach(var item in urlist)
{
wc.DownloadStringCompleted += new DownloadStringCompletedEventHandler(wc_DownloadStringCompleted);
    wc.DownloadStringAsync(new Uri(item.url, UriKind.Absolute));
}

 void wc_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
 {
 urlist[?].result = e.Result; 
 }

I feel completely stuck. Thanks for your ideas.

Answer 1

the problem I encounter is: how do I know which e.Result corresponds to what url ?

There are various different options for this:

UserState

You can pass in a second argument to DownloadStringAsync, which is then available via DownloadStringCompletedEventArgs.UserState. For example:

// In your loop....
var wc = new WebClient();
wc.DownloadStringAsync(new Uri(item.url, UriKind.Absolute), item);

void wc_DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
{
    var item = (ressource) e.UserState;
    item.result = e.Result;
}

Multiple WebClients

You can create a new WebClient for each iteration of the loop, and attach a different event handler to it. A lambda expression is useful here:

// Note: this is broken in C# 3 and 4 due to the capture semantics of foreach.
// It should be fine in C# 5 though.
foreach(var item in urlist)
{
    var wc = new WebClient();
    wc.DownloadStringCompleted += (sender, args) => item.result = args.Result;
    wc.DownloadStringAsync(new Uri(item.url, UriKind.Absolute));
}

DownloadStringTaskAsync

You could DownloadStringTaskAsync instead, so that each call returns a Task<string>. You could keep a collection of these - one for each element in urlist - and know which is which that way.

Alternatively, you could just fetch all the results synchronously, but I suspect you don't want to do that.

Additional information

Unfortunately, WebClient doesn't support multiple concurrent connections, so with all the options above you should create a new WebClient per iteration anyway.

Answer 2

Another alternative, and the one I prefer, is to use Microsoft's Reactive Framework (Rx). It handles all the background threading for you, similar to the TPL, but often easier.

Here's how I would do it:

var query =
    from x in urlist.ToObservable()
    from result in Observable.Using(
        () => new WebClient(),
        wc => Observable.Start(() => wc.DownloadString(x.url)))
    select new
    {
        x.url,
        result
    };

Now to get the results back into the original urlist.

var lookup = urlist.ToDictionary(x => x.url);

query.Subscribe(x =>
{
    lookup[x.url].result = x.result;
});

Simple as that.