WCF service failed in authentication on Server 2012 but OK in Win7?

Question

I had created a WCF service with role base authentication and authorization. And the implementation of each operation is like

[PrincipalPermission(SecurityAction.Demand, Role = RoleConstants.Customer)]
[PrincipalPermission(SecurityAction.Demand, Role = RoleConstants.CustomerStaff)]
public void DoSomething()
{
}

The membership provider and RoleManager are the ones for MySql, not in GAC. The SSL certificates are generated by IIS 7 for https.

And I have some integration tests like

using (var client = new MyProxy("DefaultBinding_ILicensingService"))
{
    client.ClientCredentials.UserName.UserName = "AbcShop";
    client.ClientCredentials.UserName.Password = "tttttttt";

    client.DoSomething();
}

One instance is running on my dev machine of Win7, and the other is on a test server of Server 2012. Both instances of the WCF service had been working well.

However, after 1 months, I found that the instance on Server 2012 is not working anymore, and failed on authentication, with the following message.

Update:

System.ServiceModel.Security.MessageSecurityException : An unsecured or incorrectly secured fault was received from the other party. See the inner FaultException for the fault code and detail.
---- System.ServiceModel.FaultException : An error occurred when verifying security for the message.
Stack Trace:

Server stack trace: 
   at System.ServiceModel.Channels.SecurityChannelFactory`1.SecurityRequestChannel.ProcessReply(Message reply, SecurityProtocolCorrelationState correlationState, TimeSpan timeout)
   at System.ServiceModel.Channels.SecurityChannelFactory`1.SecurityRequestChannel.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
   at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)

Update:

I have observed the following and ruled out some possible causes.

1. The auth databases on both machines are identical, and I could see the WCF service is making SQL queries to retrieve member information after the client is making a request.
2. I had compared the IIS binding and settings, as well as web.config on both machine, which are basically identical except some addresses.
3. The Event Viewer show no warning about the WCF service.
4. The WCF service implementation is decorated by this attributed: public class ErrorHandlerBehaviorAttribute : Attribute, IServiceBehavior, IErrorHandler, and all uncaught exceptions will go to this attribute which will log the errors into a log file, however, the log file recorded nothing for this matter.

And I had compared the loaded assemblies in both instances. In Windows 7, the system assemblies are basically from Windows\Microsoft.NET\assembly\GACXXX\XXX, however in Server 2012, some assemblies are from Windows\assembly\NativeImages_v4.0.30319_64,here are some assemblies located in NativeImages:

System   
System.Activities   
System.Core   
System.Data.DataSetExtensions   
System.Drawing   
System.Enumerics
System.ServiceModel.Activities   
System.WorkflowServices   
System.XAML   
System.ServiceModel.Web

In contrast, in Win 7, only System is from NativeImages. I am not sure if the location of assemblies or native images could alter the behaviors.

The services and clients are developed using VS 2012 on .NET Framework 4.5.

What else could possibly make the instance on Server 2012 fail?

Answer 1

Problems resolved.

The root of causes: missing ASP .NET MVC assemblies in Server 2012.

The background is, I want the WCF service to share the same auth database with MVC applications to be built in the future. So the member provider and auth manager are the ones from MVC, and the database is MySql, so the providers are from an open source 3rd party component which is coupled with MVC.

The reasons why my uncaught exception handler and Event Viewer could not catch warnings about missing assemblies are: 1. The dev PC was used by a prior developer who loved GAC. You know GAC could cause a lot problems in a dev PC. Tons of assembly from MVC 1,2,and 3 were in GAC. 2. The 3rd component interacting with MVC has some poor programming practice somewhere swallowing some exceptions about missing assemblies.

After I compared loaded assemblies one by one in both machine, I found the WCF service instance on Server 2012 had no WebMatrix.WebData.dll. After I copied the assembly to the bin directory of the virtual directory, I got a chain of error messages about more missing assemblies after I copied missing assemblies reported one by one. These assemblies basically belong to MVC. The error messages appeared on the client side in the response from the service in HTML, as well as Event Viewer in the server side.

So I now ensure the deployment scripts to include respective assemblies.

Apparently after the service retrieved member info from the auth database, the service will delegate the info to other assemblies of MVC for further processing, and somehow the 3rd component swallowed the exception about missing assemblies. And the other parts of the authentication got a wrong signal, and then gave misleading warning message about "Verifying Security...".

The lesson is: 1. Do not GAC in the dev machine. If you inherited a dev machine from someone, better to invest sometime to un-gac some which should had been private assemblies through XCopy. 2. Some 3rd party components might have some dirty practice against defensive programming. In my case, the component swallowed the warnings about missing assemblies. 3. It is cheaper to compare loaded assemblies before investigating other possible causes. I use Process Explorer to list loaded assemblies in w3wp.exe.