WCF response time, throttling

Question

My WCF service configuration:

<system.net>
    <connectionManagement>
      <add address ="*" maxconnection="500"/>
    </connectionManagement>
</system.net>

<bindings>
  <basicHttpBinding>
     <binding name="customBasicHttpBinding" 
         maxBufferSize="2147483647" maxReceivedMessageSize="2147483647" 
         transferMode="StreamedResponse">
         <readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647" 
              maxArrayLength="2147483647" maxBytesPerRead="2147483647" 
              maxNameTableCharCount="2147483647"/>
         <security mode="None"/>
     </binding>
  </basicHttpBinding>
  <webHttpBinding>
    <binding name="customWebBinding" maxBufferSize="2147483647" 
        maxReceivedMessageSize="2147483647">
        <readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647" 
             maxArrayLength="2147483647" maxBytesPerRead="2147483647" 
             maxNameTableCharCount="2147483647"/>
        <security mode="None">
        </security>
    </binding>
  </webHttpBinding>
</bindings>

<serviceBehaviors>
   <behavior name="soapBehavior">
      <serviceMetadata httpGetEnabled="true" httpsGetEnabled="true"/>
      <dataContractSerializer maxItemsInObjectGraph="6553600"/>
      <serviceDebug includeExceptionDetailInFaults="false"/>
      <serviceThrottling maxConcurrentCalls="100"
            maxConcurrentInstances="100" maxConcurrentSessions="100" />
   </behavior>
</serviceBehaviors>

<services>
   <service behaviorConfiguration="soapBehavior" name="Service.Service">
      <endpoint name="soap" 
          address="" 
          binding="basicHttpBinding" bindingConfiguration="customBasicHttpBinding"  
          contract="ServiceModel.IService"/>
      <endpoint 
          address="mex" binding="mexHttpBinding" contract="IMetadataExchange"/>
    </service>
</services>

As you can see I set up throttling parameter to handle 100 concurrent instances.

For a test purpose I created dummy method on my interface that looks something like this

[OperationContract]
string Test(){
    return "test response time";
}

When I'm trying to call this method, it uses 100 parallel request ats once response time is very bad:

Now running 100 parallel requests...
ResponseTimes: 0,45205P, 10,047P, 0,43304P, 0,86609P, 1,33913P, 0,91409P, 1,34713P, 1,75718P, 1,37414P, 1,80718P, 1,80618P, 2,22622P, 2,64426P, 2,22822P, 2,62626P, 2,68127P, 3,0453P, 3,10731P, 3,47635P, 3,51035P, 3,91039P, 3,94039P, 3,9544P, 4,36844P, 4,34943P, 4,78748P, 4,37144P, 4,82248P, 4,79048P, 5,25052P, 4,81948P, 5,67657P, 5,25253P, 5,71657P, 5,67357P, 6,13761P, 5,70257P, 6,56566P, 6,12361P, 7,0117P, 6,53065P, 7,43674P, 6,9517P, 7,86679P, 7,36974P, 7,81778P, 8,29483P, 8,75988P, 8,71587P, 8,24182P, 8,70187P, 9,16392P, 9,12991P, 9,19492P, 9,57596P, 9,65797P, 10,08201P, 10,45205P, 10,52505P, 10,48905P, 10,9521P, 10,89709P, 11,37714P, 11,81118P, 11,32413P, 11,76418P, 11,83918P, 12,18222P, 12,31723P, 12,60526P, 12,75128P, 13,0423P, 13,17132P, 13,48935P, 13,64836P, 13,91039P, 14,07141P, 14,32843P, 14,48945P, 14,78548P, 14,91149P, 15,20652P, 15,33153P, 15,62856P, 15,75558P, 16,0516P, 16,19262P, 16,48265P, 16,61866P, 16,91169P, 17,05471P, 17,33773P, 17,48375P, 17,74677P, 17,92079P, 18,15782P, 18,34183P, 18,58086P, 18,77388P, 19,0069P,
0 request(s) failed.
Average response time:9,20126

Why are results so bad, I tried to change AppPool Worker Process count but no luck, can anyone tell what I'm missing, what is setting limits?

I'm using WCF 4.0, IIS7.5 on a Windows Server 2008R2 machine.

Thank you

Answer 1

It is difficult to provide much insight regarding communication performance issues without detailed information about the service, configuration and environment. At the least, you may way to provide the service binding, the ServiceBehaviorAttribute and information about the client configuration.

From years of conducting WCF performance testing and optimization, we have seen “similar” issues as you described … despite having 100 concurrent connections, the service does not seem to “respond” efficiently, even though the server resources do not seem busy. In our case, the “delay” was associated with a slow, “cold” startup and the time taken by the .NET thread pool to allocate threads.

The following article discusses our issue:
http://blogs.msdn.com/b/dmetzgar/archive/2011/05/04/wcf-scales-up-slowly-with-bursts-of-work.aspx

Good luck.

Answer 2

I just finished a big, industrial scale WCF project that employed throttling and found that throttling doesn't always yield the results you expect. We set up our WCF web servce on a production-grade virtual server, then we created a test harness that emulated 1000+ virtual clients on a multi-threaded program. Once we were ready, we ran, and re-ran, tests over and over using a bunch of different throttling settings from 1 - 1000, but were surprised by the results.

For instance, you'd think that running your web service with 200 max concurrent connections would be twice as fast as 100 max connections, but that's not what we found for settings for things like:

-max concurrent sessions

-max concurrent calls

-max concurrent instances

In reality, there wasn't much of a performance (callsProcessed/second) difference between MaxConcurrentSessions=10 and MaxConcurrentSessions=1000. The calls processed per second was about the same, only the memory usage was different. Same thing with other throttle settings.

The fastest setting we found for throttling? No setting at all; basically, let the System.ServiceModel library handle everything. That's the fastest that we found after days of testing.

As far as your performance is concerned, what I would do is try and figure out where the bottle necks are. For instance, if your WCF service is using SQL to retrieve data, try eliminating SQL and just return a static dataset and see if your time dramatically improves. If it does, then maybe you need to work on the database side of things. If it doesn't, maybe there's a problem processing your SOAP messages.