Slow SoapHttpClientProtocol constructor

Question

I'm doing some experiments with Microsoft Dynamics CRM. You interact with it through web services and I have added a Web Reference to my project. The web service interface is very rich, and the generated "Reference.cs" is some 90k loc.

I'm using the web reference in a console application. I often change something, recompile and run. Compilation is fast, but newing up the web service reference is very slow, taking some 15-20 seconds: CrmService service = new CrmService(); Profiling reveals that all time is spent in the SoapHttpClientProtocol constructor.

The culprit is apparently the fact that the XML serialization code (not included in the 90k loc mentioned above) is generated at run time, before being JIT'ed. This happens during the constructor call. The wait is rather frustrating when playing around and trying things out.

I've tried various combinations of sgen.exe, ngen and XGenPlus (which takes several hours and generates 500MB of additional code) but to no avail. I've considered implementing a Windows service that have few CrmService instances ready to dish out when needed but that seems excessive.

Any ideas?

Answer 1

The following is ripped from this thread on the VMWare forums:

Hi folks,

We've found that sgen.exe does work. It'just that there is a couple of additional steps beyond pre-generating the serializer dll's that we missed in this thread. Here is the detailed instruction

PROBLEM

When using the VIM 2.0 SDK from .NET requires long time to instantiate the VimService class. (The VimService class is the proxy class generated by running 'wsdl.exe vim.wsdl vimService.wsdl')

In other words, the following line of code:

_service = new VimService();

Could take about 50 seconds to execute.

CAUSE

Apparently, the .NET XmlSerializer uses the System.Xml.Serialization.* attributes annotating the proxy classes to generate serialization code in run time. When the proxy classes are many and large, as is the code in VimService.cs, the generation of the serialization code can take a long time.

SOLUTION

This is a known problem with how the Microsoft .NET serializer works.

Here are some references that MSDN provides about solving this problem:

http://msdn2.microsoft.com/en-us/library/bk3w6240.aspx http://msdn2.microsoft.com/en-us/library/system.xml.serialization.xmlserializerassemblyattribute.aspx

Unfortunately, none of the above references describe the complete solution to the problem. Instead they focus on how to pre-generate the XML serialization code.

The complete fix involves the following steps:

1. Create an assembly (a DLL) with the pre-generated XML serializer code

2. Remove all references to System.Xml.Serialization.* attributes from the proxy code (i.e. from the VimService.cs file)

3. Annotate the main proxy class with the XmlSerializerAssemblyAttribute to point it to where the XML serializer assembly is.

Skipping step 2 leads to only 20% improvement in the instantiation time for the VimService class. Skipping either step 1 or 3 leads to incorrect code. With all three steps 98% improvement is achieved.

Here are step-by-step instructions:

Before you begin, makes sure you are using .NET verison 2.0 tools. This solution will not work with version 1.1 of .NET because the sgen tool and the XmlSerializationAssemblyAttribute are only available in version 2.0 of .NET

1. Generate the VimService.cs file from the WSDL, using wsdl.exe:

   wsdl.exe vim.wsdl vimService.wsdl

   This will output the VimService.cs file in the current directory

2. Compile VimService.cs into a library

   csc /t:library /out:VimService.dll VimService.cs

3. Use the sgen tool to pre-generate and compile the XML serializers:

   sgen /p VimService.dll

   This will output the VimService.XmlSerializers.dll in the current directory

4. Go back to the VimService.cs file and remove all System.Xml.Serialization.* attributes. Because the code code is large, the best way to achieve that is by using some regular expression substitution tool. Be careful as you do this because not all attributes appear on a line by themselves. Some are in-lined as part of a method declaration.

   If you find this step difficult, here is a simplified way of doing it:

   Assuming you are writing C#, do a global replace on the following string:

   [System.Xml.Serialization.XmlIncludeAttribute

   and replace it with:

   // [System.Xml.Serialization.XmlIncludeAttribute

   This will get rid of the Xml.Serialization attributes that are the biggest culprits for the slowdown by commenting them out. If you are using some other .NET language, just modify the replaced string to be prefix-commented according to the syntax of that language. This simplified approach will get you most of the speedup that you can get. Removing the rest of the Xml.Serialization attributes only achieves an extra 0.2 sec speedup.

5. Add the following attribute to the VimService class in VimService.cs:

   [System.Xml.Serialization.XmlSerializerAssemblyAttribute(AssemblyName = "VimService.XmlSerializers")]

   You should end up with something like this:

   // ... Some code here ... [System.Xml.Serialization.XmlSerializerAssemblyAttribute(AssemblyName = "VimService.XmlSerializers")] public partial class VimService : System.Web.Services.Protocols.SoapHttpClientProtocol { // ... More code here

6. Regenerate VimSerice.dll library by

   csc /t:library /out:VimService.dll VimService.cs

7. Now, from your application, you can add a reference to VimSerice.dll library.

8. Run your application and verify that VimService object instanciation time is reduced.

ADDITIONAL NOTES

The sgen tool is a bit of a black box and its behavior varies depending on what you have in your Machine.config file. For example, by default it is supposed to ouptut optimized non-debug code, but that is not always the case. To get some visibility into the tool, use the /k flag in step 3, which will cause it to keep all its temporary generated files, including the source files and command line option files it generated.

Even after the above fix the time it takes to instantiate the VimService class for the first time is not instantaneous (1.5 sec). Based on empirical observation, it appears that the majority of the remaining time is due to processing the SoapDocumentMethodAttribute attributes. At this point it is unclear how this time can be reduced. The pre-generated XmlSerializer assembly does not account for the SOAP-related attributes, so these attributes need to remain in the code. The good news is that only the first instantiation of the VimService class for that app takes long. So if the extra 1.5 seconds are a problem, one could try to do a dummy instantiation of this class at the beginning of the application as a means to improve user experience of login time.

Answer 2

You might wish to look into the Sgen.exe tool that comes with .NET. There's also a handy little thing in Visual Studio's C# project properties "Build" page, at the very bottom, called "Build serialization assembly" that automatically runs Sgen for you.

Answer 3

I believe that this is not an SGEN issue. I have looked at the constructor code, and I see that it is doing a lot of reflection (based on the XmlIncludeAttribute on the class). It reflects on all of them, and can take a really long time.

Answer 4

There is a pre-generated XmlSerializer assembly that comes with CRM. Check to see whether you have SdkTypeProxy.XmlSerializers.dll and SdkProxy.XmlSerializers.dll in the GAC.

If you don't then that means that when you create the CrmService, .net will generate the XmlSerializer assembly which can take some time. Hope this helps

Answer 5

I came across this thread when trying to find out why my initial SoapHttpClientProtocol calls were taking so long.

I found that setting the Proxy to null/Empty stopped the Proxy AutoDetect from occurring - This was taking up to 7 seconds on the initial call:

this.Proxy = GlobalProxySelection.GetEmptyWebProxy();

Answer 6

I have used above detailed answer as guide, and went a few steps forward, making a script to automate process. Script is made out of two files :

generateproxy.bat :

REM if your path for wsdl, csc or sgen is missing, please add it here (it varies from machine to machine)
set PATH=%PATH%;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.6.1 Tools;C:\Program Files (x86)\MSBuild\14.0\Bin

wsdl http://localhost:57237/VIM_WS.asmx?wsdl REM create source code out of WSDL
PowerShell.exe -ExecutionPolicy Bypass -Command "& '%~dpn0.ps1'" REM proces source code (remove annotations, add other annotation, put class into namespace)
csc /t:library /out:references\VIM_Service.dll VIM_WS.cs REM compile source into dll
sgen /p references\VIM_Service.dll /force REM generate serializtion dll

generateproxy.ps1

(Get-Content VIM.cs) | 
    ForEach-Object { 
        $_ -replace "(?<attr>\[global::System.Xml.Serialization.[^\]]*\])", "/*${attr}*/" `
            -replace "public partial class VIM", "[System.Xml.Serialization.XmlSerializerAssemblyAttribute(AssemblyName = ""VIM_Service.XmlSerializers"")] `npublic partial class VIM" `
            -replace "using System;", "namespace Classes.WS_VIM {   `n`nusing System;"
    } |
Set-Content VIM.cs
Add-Content VIM.cs "`n}"

I have added those two files to client project, and in the pre-build event I have added lines

cd..\..
generateproxy

So, before every build, proxy classes are regenerated, and developer has (almost) no need to think about it. While building, WS must be up and running, and its URL must be in bat file. As a result of prebuild, two dll files will regenerate in client project's subfolder references. After first execution of scripts, you should add reference to new dll.