Azure Diagnostics doesnt transfer IIS logs to blob storage

Question

Ok, I have been banging my head on this one for a while, figure I would ask if anyone has encountered this before.

My configuration and setup is below. This is when deployed to the staging environment. I know that the logging is happening because I enabled RDP and I am able to go in and I can see the IIS logs created. However they do not transfer to blob storage, even though other diagnostics logs do without problem.

In the infrastructure logs i can see the following error, which I'm told is only about the proxy and not relevant:

WinHttpGetProxyForUrl(http://mystorage.blob.core.windows.net) failed ERROR_WINHTTP_AUTODETECTION_FAILED (12180)

The other warning I see in the Event Viewer is:

-<Event>
-<System>
 <Provider Name="Microsoft-Windows-IIS-W3SVC-WP"Guid="{670080D9-742A-4187-8D16-41143D1290BD}"EventSourceName="W3SVC-WP"/>
 <EventID Qualifiers="32768">2283</EventID>
 <Version>0</Version>
 <Level>3</Level>
 <Task>0</Task>
 <Opcode>0</Opcode>
 <Keywords>0x80000000000000</Keywords>
 <TimeCreated SystemTime="2011-02-18T22:46:34.000Z"/>
 <EventRecordID>266</EventRecordID>
 <Correlation/>
 <Execution ProcessID="0"ThreadID="0"/>
 <Channel>Application</Channel>
 <Computer>RD00155D3273B5</Computer>
 <Security/>
 </System>
-<EventData>
 <Data Name="FailureCount">3</Data>
 <Data Name="Minutes">5</Data>
 <Data Name="Directory">\\?\C:\Resources\directory\345345kjh325kj5432452345.MyWebRole.DiagnosticStore\FailedReqLogFiles\Web\W3SVC1273337584\</Data>
 <Binary>03000780</Binary>
 </EventData>
 </Event>

This is a fairly simple WCF application running as a WebRole. The config looks like this:

<system.diagnostics>
            <sources>
                <source name="System.ServiceModel" switchValue="Information,ActivityTracing" propagateActivity="false">
                    <listeners>
                        <add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" name="ModelListener">
                            <filter type="" />
                        </add>
                    </listeners>
                </source>
                <source name="System.ServiceModel.MessageLogging" switchValue="Information" propagateActivity="false">
                    <listeners>
                        <add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" name="MessageListener">
                            <filter type="" />
                        </add>
                    </listeners>
                </source>
            </sources>
            <sharedListeners>               
            </sharedListeners>
            <trace autoflush="true" indentsize="3">
                <listeners>
                    <add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" name="TraceListener">
                        <filter type="" />
                    </add>
                </listeners>
            </trace>            
        </system.diagnostics>
...
...
<system.webServer>
        <modules runAllManagedModulesForAllRequests="true" />
        <tracing>
            <traceFailedRequests>

                <add path="*">
                    <traceAreas>
                        <add provider="ASP" verbosity="Verbose" />
                        <add provider="ASPNET" areas="Infrastructure,Module,Page,AppServices" verbosity="Verbose" />
                        <add provider="ISAPI Extension" verbosity="Verbose" />
                        <add provider="WWW Server" areas="Authentication,Security,Filter,StaticFile,CGI,Compression,Cache,RequestNotifications,Module" verbosity="Verbose" />
                    </traceAreas>
                    <failureDefinitions verbosity="Warning" statusCodes="400-599" />
                </add>
            </traceFailedRequests>
        </tracing>
    </system.webServer>

I have also changed the listeners a few different ways, with the same outcome. So I dont think thats the problem, but im open to suggestions

The code in the webrole looks like this:

public override bool OnStart ()
        {

            // To enable the AzureLocalStorageTraceListner, uncomment relevent section in the web.config  
            DiagnosticMonitorConfiguration config = DiagnosticMonitor.GetDefaultInitialConfiguration();
            //Windows Performance Counters

            List<string> counters = new List<string>();
            counters.Add(@"\Processor(_Total)\% Processor Time");
            counters.Add(@"\Memory\Available Mbytes");
            counters.Add(@"\TCPv4\Connections Established");
            counters.Add(@"\ASP.NET Applications(__Total__)\Requests/Sec");
            counters.Add(@"\Network Interface(*)\Bytes Received/sec");
            counters.Add(@"\Network Interface(*)\Bytes Sent/sec");

            foreach (string counter in counters)
            {
                PerformanceCounterConfiguration counterConfig = new PerformanceCounterConfiguration();
                counterConfig.SampleRate = TimeSpan.FromMinutes(1);
                counterConfig.CounterSpecifier = counter;
                config.PerformanceCounters.DataSources.Add(counterConfig);
            }

            config.PerformanceCounters.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);

            //Windows Event Logs
            config.WindowsEventLog.DataSources.Add("System!*");
            config.WindowsEventLog.DataSources.Add("Application!*");
            config.WindowsEventLog.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
            config.WindowsEventLog.ScheduledTransferLogLevelFilter = LogLevel.Error;

            //Azure Trace Logs
            config.Logs.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
            config.Logs.ScheduledTransferLogLevelFilter = LogLevel.Error;

            //Crash Dumps
            CrashDumps.EnableCollection(true);

            //IIS Logs
            //This was added in a desperation move but it has not made a difference with or without
            DirectoryConfiguration directory = new DirectoryConfiguration();
            directory.Container = "wad-tracefiles";
            directory.DirectoryQuotaInMB = 10;
            directory.Path = RoleEnvironment.GetLocalResource("AppLocalStorage.svclog").RootPath; 

            config.Directories.DataSources.Add(directory);
            //end desperation move
            config.Directories.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);

            //infrastructure logs
            config.DiagnosticInfrastructureLogs.ScheduledTransferPeriod = TimeSpan.FromMinutes(1);
            config.DiagnosticInfrastructureLogs.ScheduledTransferLogLevelFilter = LogLevel.Error;

            DiagnosticMonitor.Start("DiagnosticsConnectionString", config);

            // For information on handling configuration changes
            // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.
            RoleEnvironment.Changing += new EventHandler<RoleEnvironmentChangingEventArgs>(RoleEnvironment_Changing);

            // This code is necessary to use CloudStorageAccount.FromConfigurationSetting
            CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSetter) =>
            {
                configSetter(RoleEnvironment.GetConfigurationSettingValue(configName));
                RoleEnvironment.Changed += (sender, arg) =>
                {
                    if (arg.Changes.OfType<RoleEnvironmentConfigurationSettingChange>()
                        .Any((change) => (change.ConfigurationSettingName == configName)))
                    {
                        if (!configSetter(RoleEnvironment.GetConfigurationSettingValue(configName)))
                        {
                            RoleEnvironment.RequestRecycle();
                        }
                    }
                };
            });


            return base.OnStart();

        }

The DiagnosticsConnectionString is set properly as some of the tables and containers are created and populated properly.

Any help is appreciated. Thanks in advance.

Answer 1

This is a known issue in Azure with SDK 1.3. Please go through this blog post for a workaround: http://robindotnet.wordpress.com/2011/02/16/azure-toolssdk-1-3-and-iis-logging/

Answer 2

This is fixed in Azure SDK 1.4.

From What's New in Windows Azure:

Resolved an IIS log file permission Issue which caused diagnostics to be unable to transfer IIS logs to Windows Azure storage.