AppDomain 2 (mssqlsystemresource.dbo[runtime].1) is marked for unload due to memory pressure message in SQL Server Log

dba.stackexchange https://dba.stackexchange.com/questions/116688

Question

From last two days I can see following in SQL Server errorlog.

AppDomain 2 (mssqlsystemresource.dbo[runtime].1) unloaded.
&
AppDomain 2 (mssqlsystemresource.dbo[runtime].1) is marked for unload due to memory pressure.

I have system environment like this :

OS- microosft windows server 2012 R2 Standard
processors - Intel(R) Xeon(R) CPU E5-2670 0 @2.60 GHz
Installed Memory (RAM) - 8GB
Hardware Information - VMWare, Inc. VMware Virtual Platform
SQL Server - Microsoft SQL Server Enterprise (64-bit)

I have checked out in local server for yesterday's event log . I have found thses events 

Server Name ID  Severity    Source            Log      Date and Time
ABC    1008 Error   Microsoft-Windows-Perflib Application 10/01/2015 2:21
ABC     2   Error   TeamCentral      Application      9/30/2015 4:20:41 PM
ABC    1309 Warning ASP.NET 4.0.30319.0 Application 9/30/2015 4:20:41 PM
ABC   10016 Error Microsoft-Windows-DistributedCOM System 9/30/2015 4:06:23 PM
ABC   10016 Error   Microsoft-Windows-DistributedCOM    System  9/30/2015 4:00:04 PM
ABC 10016   Error   Microsoft-Windows-DistributedCOM    System  9/30/2015 4:00:02 PM
ABC 2   Error   TeamCentral Application 9/30/2015 3:55:13 PM
ABC 1309    Warning ASP.NET 4.0.30319.0 Application 9/30/2015 3:55:13 PM
ABC 10016   Error   Microsoft-Windows-DistributedCOM    System  9/30/2015 3:40:06 PM
ABC 10016   Error   Microsoft-Windows-DistributedCOM    System  9/30/2015 3:40:03 PM
ABC 1309    Warning ASP.NET 4.0.30319.0 Application 9/30/2015 3:23:54 PM
ABC 2   Error   TeamCentral Application 9/30/2015 3:23:54 PM
ABC 2   Error   TeamCentral Application 9/30/2015 3:22:48 PM
ABC 1309    Warning ASP.NET 4.0.30319.0 Application 9/30/2015 3:22:48 PM
ABC 2   Error   TeamCentral Application 9/30/2015 3:21:43 PM
ABC 1309    Warning ASP.NET 4.0.30319.0 Application 9/30/2015 3:21:43 PM
ABC 2   Error   TeamCentral Application 9/30/2015 3:18:24 PM
ABC 1309    Warning ASP.NET 4.0.30319.0 Application 9/30/2015 3:18:24 PM
ABC 2   Error   TeamCentral Application 9/30/2015 3:16:49 PM
ABC 1309    Warning ASP.NET 4.0.30319.0 Application 9/30/2015 3:16:49 PM
ABC 1309    Warning ASP.NET 4.0.30319.0 Application 9/30/2015 3:15:43 PM
ABC 2   Error   TeamCentral Application 9/30/2015 3:15:43 PM
ABC 1010    Error   Microsoft-Windows-Perflib   Application 9/30/2015 10:42:00 AM

And in Event I am getting such type of message :

    Event code: 3005 
Event message: An unhandled exception has occurred. 
Event time: 9/30/2015 4:20:41 PM 
Event time (UTC): 9/30/2015 1:20:41 PM 
Event ID: 72c24cb89beb4d7f80bec250aaa686e3 
Event sequence: 757 
Event occurrence: 17 
Event detail code: 0 

Application information: 
    Application domain: /XX/XXXX/1/ROOT/TeamCentral-2-130850431401577570 
    Trust level: Full 
    Application Virtual Path: /TeamCentral 
    Application Path: C:\XXXX\wwwroot\TeamCentral\ 
    Machine name: ABC

Process information: 
    Process ID: 2180 
    Process name: w3wp.exe 
    Account name: NT AUTHORITY\NETWORK SERVICE 

Exception information: 
    Exception type: HttpException 
    Exception message: The remote host closed the connection. The error code is 0x800704CD.
   at System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result, Boolean throwOnDisconnect)
   at System.Web.Hosting.IIS7WorkerRequest.ExplicitFlush()
   at System.Web.HttpResponse.Flush(Boolean finalFlush, Boolean async)
   at System.Web.HttpWriter.WriteFromStream(Byte[] data, Int32 offset, Int32 size)
   at DevExpress.Web.ASPxClasses.Internal.HttpUtils.WriteFileToResponse(Page page, MemoryStream stream, String fileName, Boolean saveAsFile, String fileFormat, String contentType)
   at DevExpress.Web.ASPxClasses.Internal.HttpUtils.WriteFileToResponse(Page page, MemoryStream stream, String fileName, Boolean saveAsFile, String fileFormat)
   at DevExpress.Web.ASPxGridView.Export.ASPxGridViewExporter.WriteToResponse(String fileName, Boolean saveAsFile, String fileFormat, ExportToStream getStream, ExportOptionsBase options)
   at DevExpress.Web.ASPxGridView.Export.ASPxGridViewExporter.WriteXlsxToResponse(String fileName, Boolean saveAsFile)
   at DevExpress.Web.ASPxGridView.Export.ASPxGridViewExporter.WriteXlsxToResponse(String fileName)
   at InfoArch.Web.Grid.ExportHelper.Response(ExportFileType fileType, Grid structure)
   at InfoArch.Web.Grid.GridManager.SaveToXlsx(String filename)
   at System.Web.UI.WebControls.ImageButton.OnClick(ImageClickEventArgs e)
   at System.Web.UI.WebControls.ImageButton.RaisePostBackEvent(String eventArgument)
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)



Request information: 
    Request URL: http://ABC/TeamCentral/Reporting/GridView.aspx?layoutid=38&mf=report&where=&description=PZF6Q0PKJzWQDHkN08mydZNuJ3fZ(:#cd6TgCAL0hv4u4=&screenheight=618 
    Request path: /TeamCentral/Reporting/GridView.aspx 
    User host address:XXX.XXX.XX.XXX 
    User: rafeek 
    Is authenticated: True 
    Authentication Type:  
    Thread account name: NT AUTHORITY\NETWORK SERVICE 

Thread information: 
    Thread ID: 190 
    Thread account name: NT AUTHORITY\NETWORK SERVICE 
    Is impersonating: False 
    Stack trace:    at System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result, Boolean throwOnDisconnect)
   at System.Web.Hosting.IIS7WorkerRequest.ExplicitFlush()
   at System.Web.HttpResponse.Flush(Boolean finalFlush, Boolean async)
   at System.Web.HttpWriter.WriteFromStream(Byte[] data, Int32 offset, Int32 size)
   at DevExpress.Web.ASPxClasses.Internal.HttpUtils.WriteFileToResponse(Page page, MemoryStream stream, String fileName, Boolean saveAsFile, String fileFormat, String contentType)
   at DevExpress.Web.ASPxClasses.Internal.HttpUtils.WriteFileToResponse(Page page, MemoryStream stream, String fileName, Boolean saveAsFile, String fileFormat)
   at DevExpress.Web.ASPxGridView.Export.ASPxGridViewExporter.WriteToResponse(String fileName, Boolean saveAsFile, String fileFormat, ExportToStream getStream, ExportOptionsBase options)
   at DevExpress.Web.ASPxGridView.Export.ASPxGridViewExporter.WriteXlsxToResponse(String fileName, Boolean saveAsFile)
   at DevExpress.Web.ASPxGridView.Export.ASPxGridViewExporter.WriteXlsxToResponse(String fileName)
   at InfoArch.Web.Grid.ExportHelper.Response(ExportFileType fileType, Grid structure)
   at InfoArch.Web.Grid.GridManager.SaveToXlsx(String filename)
   at System.Web.UI.WebControls.ImageButton.OnClick(ImageClickEventArgs e)
   at System.Web.UI.WebControls.ImageButton.RaisePostBackEvent(String eventArgument)
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)


Custom event details:

I have gone through the Stackexchange before asking the question i have found out the ref App domain is marked for unload due to memory pressure but some point of view , unable to understand the event log message.

Output of select @@version;

Microsoft SQL Server 2012 (SP1) - 11.0.3000.0 (X64) Oct 19 2012 13:38:57 Copyright (c) Microsoft Corporation Enterprise Edition (64-bit) on Windows NT 6.2 (Build 9200: ) (Hypervisor)

and output of exec sp_configure;

Name                    minimum     maximum      config_value     run_value 
max server memory (MB)   128        2147483647    2147483647     2147483647

Any suggestion would be appreciated.

Was it helpful?

Solution

Without discounting anything stated in @Shanky's answer, it should be noted that the specific AppDomain being unloaded is in the [mssqlsystemresource] database. This is an internal database that stored built-in SQL Server functionality. Other built-in functionality resides in the [master] database. Even with the server-level configuration option of "CLR Enabled" turned off, you can still see one or both of these AppDomains being loaded if you use any of the following (and there could be other internal uses of CLR that are not in this list):

  • Datatypes:
    • HierarchyID
    • Geography
    • Geometry
  • Functions:
    • FORMAT
    • TRY_CONVERT
    • TRY_PARSE
  • Features:
    • Change Data Capture
    • Dynamic Management Framework
    • Replication
    • Policy-Based Management

Run the following:

SELECT * FROM sys.dm_clr_appdomains;
SELECT * FROM sys.dm_clr_loaded_assemblies;

If no rows returned, then run the following:

SELECT FORMAT(GETDATE(), 'dd');
SELECT * FROM sys.dm_clr_appdomains;
SELECT * FROM sys.dm_clr_loaded_assemblies;

And you should now see an entry for "master.sys[runtime].xx" (where the xx is some number), but still no loaded assemblies. Now run:

DECLARE @Test [HierarchyID];
SET @Test = '/1/';
SELECT * FROM sys.dm_clr_appdomains;
SELECT * FROM sys.dm_clr_loaded_assemblies;

And you should now see a loaded assembly as well. It should be assembly_id 1 and that is the same entry in all databases: Microsoft.SqlServer.Types. This assembly shows up in all databases if you run SELECT * FROM sys.assemblies;, but it only truly lives in [master].

OTHER TIPS

To start with you have SQL Server 2012 patched to SP1. I would urge you, ASAP, patch SQL Server with SQL Server 2012 SP2

There has been some prominent memory related fix in SQl Server 2012 Sp2. Please read one of the KB Article documenting OOM error fix. If you look closely the fix is also for

AppDomain 3181 () is marked for unload due to memory pressure. AppDomain 3181 () unloaded. Error: 9602, Severity: 16, State: 1.

I am not saying with surety that you might be hitting the bug mentioned as it requires more thorough analysis.

only two times . I have already written in first statement from last two days i am getting this error

You confirmed that within two days you have only seen this message twice which makes me believe that this is not much to worry about. Sometimes such messages are benign when CLR is loaded and unloaded. The problem arises when this message is frequent. This would then require more analysis.

App domain messages used to be real issue in 32 bit system due to VAS limitation. In 64 bit system App domain unloaded message is mainly due to either corrupt .net framework or when you have incorrectly set max server memory value. But later was true till SQL Server 2008 r2. From 2012 onwards since memory for CLR is allocated from max server memory you have set this should not be much of issue. So I suggest you also look at messages related to .Net framework. I dont have much experience how to go ahead with .net troubleshooting.

Do you use SQLCLR objects ?. See Various error messages logged when you use SQLCLR. There is Blog by Bob Dorr about how to proceed when you see lots of such messages in SQL Server errorlog.

With

ABC 1008 Error Microsoft-Windows-Perflib Application 10/01/2015 2:21 ABC 2 Error TeamCentral Application 9/30/2015 4:20:41 PM.

I hunch is this is somewhere related to .Net framework.

Licensed under: CC-BY-SA with attribution
Not affiliated with dba.stackexchange
scroll top