“error: 19 - Physical connection is not usable” with OWIN access in Azure database

Question

I have tried all the other postings on the dreaded "error 19" and found that the few with answers do not apply or do not help, hence this new post. This is a very serious potential problem for all Azure+EF users.

First occurrence:

I am using the latest version of everything in a VS2013 EF6.1 Razor project (packages listed at the end). The database is hosted on SQL Azure.

After running my webapp a few time (in a dev environment) I get this error: A transport-level error has occurred when receiving results from the server. (provider: Session Provider, error: 19 - Physical connection is not usable)

The line it dies on is always this:

I gather the error relates to connection pooling (and running out of connections), but I cannot spot a leak anywhere.

As I access OWIN membership and other database features throughout the app I have a DatabaseContoller from which all other controllers inherit. This creates all the relevant components and disposes of them.

DatabaseController.cs

[Authorize]
public class DatabaseController : Controller
{
    #region properties
    /// <summary>
    /// User manager - attached to application DB context
    /// </summary>
    protected UserManager<ApplicationUser> UserManager { get; set; }

    /// <summary>
    /// Role manager - attached to application DB context
    /// </summary>
    protected RoleManager<IdentityRole> RoleManager { get; set; }

    /// <summary>
    /// Application DB context
    /// </summary>
    protected ApplicationDbContext ApplicationDbContext { get; set; }

    /// <summary>
    /// Database context used by most controllers
    /// </summary>
    protected ApplicationEntities Context { get; set; }
    #endregion properties

    #region Constructors
    public DatabaseController()
    {
        this.Context = new ApplicationEntities ();
        this.ApplicationDbContext = new ApplicationDbContext();
        this.UserManager = new UserManager<ApplicationUser>(new UserStore<ApplicationUser>(this.ApplicationDbContext));
        this.RoleManager = new RoleManager<IdentityRole>(new RoleStore<IdentityRole>(this.ApplicationDbContext));
        this.UserManager.UserValidator = new UserValidator<ApplicationUser>(UserManager) { AllowOnlyAlphanumericUserNames = false };
    }
    #endregion Constructors

    protected override void Dispose(bool disposing)
    {
        if (disposing)
        {
            if (UserManager != null)
            {
                this.UserManager.Dispose();
                this.UserManager = null;
            }
            if (this.RoleManager != null)
            {
                this.RoleManager.Dispose();
                this.RoleManager = null;
            }
            if (this.ApplicationDbContext != null)
            {
                this.ApplicationDbContext.Dispose();
                this.ApplicationDbContext = null;
            }
            if (this.Context != null)
            {
                this.Context.Dispose();
                this.Context = null;
            }
        }
        base.Dispose(disposing);
    }
}

Packages installed

  <package id="Antlr" version="3.5.0.2" targetFramework="net45" />
  <package id="bootstrap" version="3.1.1" targetFramework="net45" />
  <package id="EntityFramework" version="6.1.0" targetFramework="net45" />
  <package id="jQuery" version="1.11.0" targetFramework="net45" />
  <package id="jQuery.Validation" version="1.11.1" targetFramework="net45" />
  <package id="json2" version="1.0.2" targetFramework="net45" />
  <package id="Microsoft.AspNet.Identity.Core" version="2.0.0" targetFramework="net45" />
  <package id="Microsoft.AspNet.Identity.EntityFramework" version="2.0.0" targetFramework="net45" />
  <package id="Microsoft.AspNet.Identity.Owin" version="2.0.0" targetFramework="net45" />
  <package id="Microsoft.AspNet.Mvc" version="5.1.1" targetFramework="net45" />
  <package id="Microsoft.AspNet.Razor" version="3.1.2" targetFramework="net45" />
  <package id="Microsoft.AspNet.Web.Optimization" version="1.1.3" targetFramework="net45" />
  <package id="Microsoft.AspNet.WebApi" version="5.1.2" targetFramework="net45" />
  <package id="Microsoft.AspNet.WebApi.Client" version="5.1.2" targetFramework="net45" />
  <package id="Microsoft.AspNet.WebApi.Core" version="5.1.2" targetFramework="net45" />
  <package id="Microsoft.AspNet.WebApi.WebHost" version="5.1.2" targetFramework="net45" />
  <package id="Microsoft.AspNet.WebPages" version="3.1.2" targetFramework="net45" />
  <package id="Microsoft.jQuery.Unobtrusive.Validation" version="3.1.2" targetFramework="net45" />
  <package id="Microsoft.Owin" version="2.1.0" targetFramework="net45" />
  <package id="Microsoft.Owin.Host.SystemWeb" version="2.1.0" targetFramework="net45" />
  <package id="Microsoft.Owin.Security" version="2.1.0" targetFramework="net45" />
  <package id="Microsoft.Owin.Security.Cookies" version="2.1.0" targetFramework="net45" />
  <package id="Microsoft.Owin.Security.Facebook" version="2.1.0" targetFramework="net45" />
  <package id="Microsoft.Owin.Security.Google" version="2.1.0" targetFramework="net45" />
  <package id="Microsoft.Owin.Security.MicrosoftAccount" version="2.1.0" targetFramework="net45" />
  <package id="Microsoft.Owin.Security.OAuth" version="2.1.0" targetFramework="net45" />
  <package id="Microsoft.Owin.Security.Twitter" version="2.1.0" targetFramework="net45" />
  <package id="Microsoft.Web.Infrastructure" version="1.0.0.0" targetFramework="net45" />
  <package id="Modernizr" version="2.7.2" targetFramework="net45" />
  <package id="Newtonsoft.Json" version="6.0.2" targetFramework="net45" />
  <package id="Owin" version="1.0" targetFramework="net45" />
  <package id="Owin.Security.Providers" version="1.3.1" targetFramework="net45" />
  <package id="Respond" version="1.4.2" targetFramework="net45" />
  <package id="WebGrease" version="1.6.0" targetFramework="net45" />

Assuming it is a connection leak, how can I track down the source of the leak?

If you need any more information, just ask.

Update: 22 May 2014 Second Bounty offered

I still have the same problem, with some slight project changes made since last posting, so will post latest details below shortly.

I have added Connection Lifetime=3;Max Pool Size=3; to my connection strings, based on this post.

Update: 23 May 2014 Error still occurs

The next day, after debugging a few dozen times, this error returned.

Update: 11 June 2014

After 2 bounties and countless Google investigations (no real answer to this), I have to assume it is a flaw in Entity Framework 6, that I am somehow causing to appear.

More Information:

I just had the same error in a WinForm project, connect to Azure. In this instance I was accidentally not clearing an entity list after each 20 new items were added.

Every time the code ran it added 20 more records and updated the DateModified field on all of them. By the time it hit 1700 records being updated it suddenly gave the dreaded "error 19 - Physical connection is not usable". After that I needed to restart my debug IIS for it to work at all.

Obviously the code had run a massive number of updates, and maybe something about this will help someone think of something.

Answer 1

Error 19 is not a comms error! (or not just a comms error)

Just ensure you have all required .Include(x=>x.ForeignTable) calls in your LINQ to SQL query!

Updated Aug 2015 (possible solution for, at least, some scenarios):

We just had a 100% repro case on this problem, that we were able to resolve with trial and error testing, so it may well be a solution or at least provide clues on what to look for.

Scenario:

• The error only occurred under release builds running under IIS. It did not occur under debug or under IIS Express.
• We also turned on SQL profiling to see when/where the server was actually hit.
• The query in question was fetching matching records, then creating view-models in a foreach iteration of the results (ie. lazy evaluation). The view-model was dependant on a value in a related table of the queried entity.

Testing:

First attempt: remove any complex filters on the query

• Result: still failed with error 19

Second attempt: add ToList() to the query to force the query to run to completion immediately.

• Result: successful!!! (obviously something going on here)

Third attempt: remove ToList() and add .Include(x=>x.ForeignTable) to the query to force the inclusion of the related data.

• Result: success!

My new theory is:

If you accidentally leave out an Include of a foreign table, EF will randomly fail to fetch the related data when lazily-evaluating. This can then lead to the infamous error 19.

As there are foreign-key relationships in Identify Framework, you might assume there is also a missing .Include(), or equivalent, on a query somewhere within OWIN. This might be causing the random problem when using OWIN or other queries.

Notes:

• A key point to take away is that the error 19 is not a comms error. The queries do hit SQL server. It is a problem client-side with failing to fetch related data.

Pause for applause (we are very happy to have found this) :)

Updated 28 August 2015:

Just had the dreaded Error 19 again today, connecting to a local SQL database (usually this was a problem with Azure for me). Based on the results above I simply added an .Include(x=>x.ForeignTable) statement where appropriate and the problem vanished! This does seem to be a problem of EF not always being able to lazy-load related-table information.

Answer 2

Did you try the SqlAzureExecutionStrategy? Sounds like the connection is cut off, but with this Strategy EF should be automatically retry to reconnect.

http://msdn.microsoft.com/en-us/data/dn456835.aspx