Mono random CS0006 compilation errors w/ fastcgi-mono-server4

Question

I'm trying to deploy an ASP MVC project developed on Mono/OSX on my Linux server using mono 2.10.8.1 w/ fastcgi-mono-server4

The webapp always starts fine, but then I start getting random CS0006 compilation errors for various URIs, and once they break, they remain broken until I restart the server application.

An example error:

Server Error in '/' Application

Compilation Error

Description: Error compiling a resource required to service this request. Review your source file and modify it to fix this error.

Compiler Error Message: CS0006: Metadata file `/tmp/root-temp-aspnet-0/ed68754/App_global.asax_40e709ea.dll' could not be found

~/Views/Order/Download.aspx

There is a related thread from January, but both the question and the answer seem to be mod_mono specific and rather hand-wavy. Anyone have any advice on what to try to debug/solve/work around this issue? It's getting very frustrating. In particular, is there any "unsupported" workaround where I can copy something from my Windows Server machines to use an MS implementation instead of the buggy mono one?

(I've filed a bug report too.)

Answer 1

Since the errors appear to be issue with the Mono JIT attempting to compile temporary files that don't exist, I spent a few days trying different methods of working around this issue (vs solving it). One solution that worked was using aspnet_compiler on Windows to create a binary version that could be copied and run as-is on Linux/Mono (as the latest versions of Mono now support precompiled ASP.NET applications).

However, I was looking for a native Linux solution, and I don't want to have to compile and sync binaries (vs syncing a GIT repo of code) to the server, so I was looking for another solution when I came across Mono ahead of time compilation, which is pretty much the equivalent of ngen.exe on Windows.

While it doesn't precompile everything, it seems to have done the trick. For me, this deployment script does the job without any runtime build failures:

xbuild SystemDiscs.sln
mono --aot -O=all SystemDiscs/bin/SystemDiscs*.dll
killall -9 mono
nohup fastcgi-mono-server4 /socket=tcp:127.0.0.1:8000 /applications=/:/var/asp/S
ystemDiscs/SystemDiscs/ > /var/log/systemdiscs.log &

Where SystemDiscs*.dll is the output of the solution compiled with xbuild in the first step. I don't think this precompiles the ASP pages (and --aot=full isn't supported on x86, as far as I can tell), but somehow it did the job. I was waiting to see whether or not that was just a fluke, but it's been going fine with maybe a dozen commits/deploys since I asked this ten days ago, so I reckon its safe to say it works.

Answer 2

I had a similiar problem:

Compilation Error

Description: Error compiling a resource required to service this request. Review your source file and modify it to fix this error.

Compiler Error Message: CS0006: Metadata file `/tmp/<DOMAIN/>/<username/>-temp-aspnet-0/5ed74d00/App_global.asax_34cccb99.dll' could not be found
/Default.aspx

But it didn't have anything to do with precompiling but with a backslash in my username. I use likewise-open to login to a windows domain, so my username is <DOMAIN/>\<username/> while my home-directory is without a backslash: /home/likewise-open/<DOMAIN/>/<username/>. This difference or just the backslash itself caused mono not finding the compiled global.asax in the temp directory. If you look closely to the detailed output you see that the backslash \ in the out parameter is changed to a forward slash /:

dmcs /target:library /lib:"/home/likewise-open/<DOMAIN/>/<username/>/Documents/test9999/test9999/bin" /debug+ /optimize- /warn:4 /out:"/tmp/<DOMAIN/>\<username/>-temp-aspnet-0/5ed74d00/App_Web_17ca7bdd.dll" /r:"/usr/lib/mono/4.0/mscorlib.dll" /r:"/usr/lib/mono/gac/Microsoft.CSharp/4.0.0.0__b03f5f7f11d50a3a/Microsoft.CSharp.dll" /r:"/usr/lib/mono/gac/System/4.0.0.0__b77a5c561934e089/System.dll" /r:"/usr/lib/mono/gac/System.Configuration/4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll" /r:"/usr/lib/mono/gac/System.Web/4.0.0.0__b03f5f7f11d50a3a/System.Web.dll" /r:"/usr/lib/mono/gac/System.Data/4.0.0.0__b77a5c561934e089/System.Data.dll" /r:"/usr/lib/mono/gac/System.Web.Services/4.0.0.0__b03f5f7f11d50a3a/System.Web.Services.dll" /r:"/usr/lib/mono/gac/System.Xml/4.0.0.0__b77a5c561934e089/System.Xml.dll" /r:"/usr/lib/mono/gac/System.Drawing/4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll" /r:"/usr/lib/mono/gac/System.EnterpriseServices/4.0.0.0__b03f5f7f11d50a3a/System.EnterpriseServices.dll" /r:"/usr/lib/mono/gac/System.IdentityModel/4.0.0.0__b77a5c561934e089/System.IdentityModel.dll" /r:"/usr/lib/mono/gac/System.Runtime.Serialization/4.0.0.0__b77a5c561934e089/System.Runtime.Serialization.dll" /r:"/usr/lib/mono/gac/System.Xaml/4.0.0.0__b77a5c561934e089/System.Xaml.dll" /r:"/usr/lib/mono/gac/System.ServiceModel/4.0.0.0__b77a5c561934e089/System.ServiceModel.dll" /r:"/usr/lib/mono/gac/System.ServiceModel.Web/4.0.0.0__31bf3856ad364e35/System.ServiceModel.Web.dll" /r:"/usr/lib/mono/gac/System.Core/4.0.0.0__b77a5c561934e089/System.Core.dll" /r:"/usr/lib/mono/gac/System.Web.Extensions/4.0.0.0__31bf3856ad364e35/System.Web.Extensions.dll" /r:"/usr/lib/mono/gac/System.Data.DataSetExtensions/4.0.0.0__b77a5c561934e089/System.Data.DataSetExtensions.dll" /r:"/usr/lib/mono/gac/System.Xml.Linq/4.0.0.0__b77a5c561934e089/System.Xml.Linq.dll" /r:"/usr/lib/mono/gac/System.ComponentModel.DataAnnotations/4.0.0.0__31bf3856ad364e35/System.ComponentModel.DataAnnotations.dll" /r:"/usr/lib/mono/gac/System.Web.DynamicData/4.0.0.0__31bf3856ad364e35/System.Web.DynamicData.dll" /r:"/usr/lib/mono/gac/System.Data.Linq/4.0.0.0__b77a5c561934e089/System.Data.Linq.dll" /r:"/usr/lib/mono/gac/System.Web.ApplicationServices/4.0.0.0__31bf3856ad364e35/System.Web.ApplicationServices.dll" /r:"/home/likewise-open/<DOMAIN/>/<username/>/Documents/test9999/test9999/bin/test9999.dll" /r:"/tmp/<domain/>/<username/>-temp-aspnet-0/5ed74d00/App_global.asax_34cccb99.dll"  /nowarn:0169 /d:DEBUG  -- "/tmp/<DOMAIN/>\<username/>-temp-aspnet-0/5ed74d00/App_Web_17ca7bdd_0.cs" 

So:

/out:"/tmp/<DOMAIN/>\<username/>-temp-aspnet-0/5ed74d00/App_Web_17ca7bdd.dll 

becomes

/tmp/<DOMAIN/>/<username/>-temp-aspnet-0/5ed74d00/App_global.asax_34cccb99.dll

If I copy the directory 5ed74d00 to /tmp/<DOMAIN/>/<username/>-temp-aspnet-0/ the application works as expected.

It took me almost 2 days to see this, so hopefully I can help others with this answer.