Non-blocking, non-threaded HTTP client implementation

This is a C# WinForms program targeting .Net 4.5. I am using it for a movie organizer application that will contact sites like OMDB, MyApiFilms, etc. for data about the movies. At the main application level a new movie list needs to have its movies matched to their info, which means searching these sites via their API, identifying the correct search result, downloading that additional details, downloading poster images from IMDB, and so on. The behavior that I'm trying to accomplish with the http downloader portion of the code is that I want to support multiple connections at once, I want each one to report back to the main application as soon as it is done without blocking to wait for the other requests to finish, and ideally I want it to report back via an event so that the downloader is simple to use from the main application.

Originally I implemented this with asynchronous delegates and BeginInvoke/EndInvoke and Invoke to get back on the UI thread to fire an event, but reading through SO and Stephen Cleary's blog I learned that this is bad because it blocks thread pool threads for every connection and is not scalable or efficient for that reason. This is my next attempt and I would be grateful if any problems I don't see could be pointed out to me. The googling I did mostly turned up suggestions to await all after sending a batch, which was not what I wanted because I don't want 9 good requests waiting on 1 that is going to eventually time out (I have really poor internet at home and long timeouts are necessary) and I also don't want those connection slots held up - meaning I am trying to be kind to the site by not hammering it with too many concurrent requests, but if 9 have returned I can send 9 more immediately while waiting on the 1 slow or failed response.

Here's the call at the main application and the event handler: EDIT: Added full event handler code per request.

```
private void SearchMovie(Movie movie, SearchTerms terms)
{
//Do some other a

Have you considered using HttpClient class from .Net 4.5+? That makes your code lot cleaner.

Also why not take Async pattern all the way? From your initiator (button click or main etc) you can call a async method, which will call all the downstream methods. That way it will be more scalable (as framework will take care of worker threads, scheduling etc)and will not block your main thread.

Currently you call the Request from main thread, but don't wait for the it to complete. You are adding the whole event mechanism to pass back the response. Instead you can make Request return a Task(ResponseArgs is nothing but ResponseEventArgs but named to match the pattern)

Then your code can be much easy to read/understand/maintain

public async void Search(SearchTerms terms, object caller)
{
    HttpGetter> getter = new HttpGetter>(
        new ResponseProcessorDelegate>(GetSearchResultsFromResponse)
        );
    String requestURL = GetSearchURL(terms);
    var response = await getter.Request(new RequestInfo(requestURL, terms, caller));  
    //process the response, what you are currently doing in event handler
}

Ideally your SearchTerms should return a task as well. Any exceptions from the Request should be handled in within the SearchTerms and it can return just a bool to indicate a success or failure.

Where you are calling SearchTerms (which in itself will be marked as async you will have

var tasks = Movies.Select(t=>Search(t));
await Task.WhenAll(tasks);

async Task Search(Movie m) 
{
    SearchTerms terms=GetSearchTerms(m); 
    await Search(terms,m);
}

This way you will line up all the work items (tasks) once, and let the system take care of executing them. Once all tasks are complete your code will proceed. Makes sense?

StackExchange Code Review Q#104562, answer score: 8