Consistent way to handle transient timeouts with WCF calls (timeouts, unreliable network, server load, etc)

I'm currently using the following code to manage calls to WCF services that are unreliable, and or suffer performance load issues due to contention.

Right now this code is limited to solving issue with types that return void. It exponentially backs off from a subsequent call from the server, preventing unhelpful "hammering" of the server.

Question

-
Is there any possible way I can abstract this into something more generic for varying return types?

-
Have I caught all the appropriate exceptions worthy of a retry?

-
What is the most appropriate way I can communicate to the calling process what is going on with this code, perhaps offering an opportunity to cancel early?

-
Any other enhancements you may think of...

Usage Example:

Service.Use(orderService=>
{
  orderService.PlaceOrder(request);
}

Source code

```
public delegate void UseServiceDelegate(T proxy);

public static class Service
{
public static ChannelFactory _channelFactory = new ChannelFactory("");

public static void Use(UseServiceDelegate codeBlock)
{
IClientChannel proxy = (IClientChannel)_channelFactory.CreateChannel();
bool success = false;

Exception mostRecentEx = null;
int millsecondsToSleep = 1000;

for(int i=0; i<5; i++) // Attempt a maximum of 5 times
{
try
{
codeBlock((T)proxy);
proxy.Close();
success = true;
break;
}

// The following is typically thrown on the client when a channel is terminated due to the server closing the connection.
catch (ChannelTerminatedException cte)
{
mostRecentEx = cte;
proxy.Abort();
// delay (backoff) and retry
Thread.Sleep(millsecondsToSleep * (i + 1));
}

// The following is thrown when a remote endpoint could not be found or reached. The endpoint may not be found or

One clean up is to consolidate the common code in the catch blocks, by putting it into a function. Other ideas:

• 5 tries is not many for WCF in general. I would increase it to 50 or so. If you do this, you need a ceiling to multiple applied to to the minimum sleep time.

• The milliseconds to sleep should be initialized to a semi-random value: a minimum value, plus a random fraction of an additional amount. Usually I make the additional amount equal to the minimum value. This will keep all your clients from hammering the server with the same rhythm, in the event all clients start at the same time.

With regards to questions:

• Make the method generic, with a type parameter defining the return value.

• I usually have only two catch blocks, one for FaultException to communicate exceptions explicitly from server to client, and one for Exception, to catch everything else. The Exception block is where you initiate a retry. Read about FaultContractAttribute.

• The only good way to allow cancellation is to run all of this in a background thread, possibly using BackgroundWorker.

StackExchange Code Review Q#41692, answer score: 3