The private key does not support the exchange KeySpec during ADFS2.0 response

Question

Have such a problem

We have claim awared asp.net site and adfs server configured for this site

So, that we have - we launch our web application - moved for authentication to adfs server and move back to web site - as result we have error server page with url of our site

**System.NotSupportedException: The private key does not support the exchange KeySpec.
Source Error: 
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.  
Stack Trace: 
[NotSupportedException: The private key does not support the exchange KeySpec.]
   System.IdentityModel.Tokens.X509AsymmetricSecurityKey.DecryptKey(String algorithm, Byte[] keyData) +237615
   System.IdentityModel.Selectors.SimpleTokenResolver.TryResolveSecurityKeyCore(SecurityKeyIdentifierClause keyIdentifierClause, SecurityKey& key) +378
   Microsoft.IdentityModel.Tokens.EncryptedSecurityTokenHandler.ReadToken(XmlReader reader) +253
   Microsoft.IdentityModel.Tokens.SecurityTokenHandlerCollection.ReadToken(XmlReader reader) +105
   Microsoft.IdentityModel.Web.TokenReceiver.ReadToken(String tokenXml, XmlDictionaryReaderQuotas readerQuotas) +282
   Microsoft.IdentityModel.Web.WSFederationAuthenticationModule.SignInWithResponseMessage(HttpRequest request) +312
   Microsoft.IdentityModel.Web.WSFederationAuthenticationModule.OnAuthenticateRequest(Object sender, EventArgs args) +1099462
   System.Web.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +80
   System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +171**

Windows logs shows that in this time we receive such warning - as I suggested completely corelated with error from server error page

Log Name:      Application
Source:        ASP.NET 2.0.50727.0
Date:          3/23/2011 2:05:50 PM
Event ID:      1309
Task Category: Web Event
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      machine-name
Description:
Event code: 3005 
Event message: An unhandled exception has occurred. 
Event time: 3/23/2011 2:05:50 PM 
Event time (UTC): 3/23/2011 2:05:50 PM 
Event ID: 356af0be20744d15bba97111f6bbd475 
Event sequence: 7 
Event occurrence: 6 
Event detail code: 0 

Application information: 
    Application domain: /appl-domain/ClaimsAwareWebAppWithManagedSTS-1-129453622903756000 
    Trust level: Full 
    Application Virtual Path: /ClaimsAwareWebAppWithManagedSTS 
    Application Path: C:\tools\AdvantageSso\Using Managed STS\ClaimsAwareWebAppWithManagedSTS\ 
    Machine name: machine-name 

Process information: 
    Process ID: 3840 
    Process name: w3wp.exe 
    Account name: NT AUTHORITY\NETWORK SERVICE 

Exception information: 
    Exception type: NotSupportedException 
    Exception message: The private key does not support the exchange KeySpec. 

Request information: 
    Request URL: https://app-url/ClaimsAwareWebAppWithManagedSTS/default.aspx 
    Request path: /ClaimsAwareWebAppWithManagedSTS/default.aspx 
    User host address: host-name
    User:  
    Is authenticated: False 
    Authentication Type:  
    Thread account name: NT AUTHORITY\NETWORK SERVICE 

Thread information: 
    Thread ID: 10 
    Thread account name: NT AUTHORITY\NETWORK SERVICE 
    Is impersonating: False 
    Stack trace:    at System.IdentityModel.Tokens.X509AsymmetricSecurityKey.DecryptKey(String algorithm, Byte[] keyData)
   at System.IdentityModel.Selectors.SecurityTokenResolver.SimpleTokenResolver.TryResolveSecurityKeyCore(SecurityKeyIdentifierClause keyIdentifierClause, SecurityKey& key)
   at Microsoft.IdentityModel.Tokens.EncryptedSecurityTokenHandler.ReadToken(XmlReader reader)
   at Microsoft.IdentityModel.Tokens.SecurityTokenHandlerCollection.ReadToken(XmlReader reader)
   at Microsoft.IdentityModel.Web.TokenReceiver.ReadToken(String tokenXml, XmlDictionaryReaderQuotas readerQuotas)
   at Microsoft.IdentityModel.Web.WSFederationAuthenticationModule.SignInWithResponseMessage(HttpRequest request)
   at Microsoft.IdentityModel.Web.WSFederationAuthenticationModule.OnAuthenticateRequest(Object sender, EventArgs args)
   at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Part with empty user and IsAuthenticated = false is very confused... So does it mean that we don't pass authentication ob ADFS or it shows current results of requested site

So, unfortunally I'm not large expert in this area and fighting with it ner the week

Could anybody help me to fix the problem?

Thanks a lot

Answer 1

What seems to happen here is that your Relying Party Trust in AD FS is configured to encrypt its tokens using (the public part of) some certificate. It does that, and your web application tries to decrypt them using the private key of that same certificate.

However, the certificate needs to support 'key exchange', and from your error message it looks like yours doesn't. I have no idea where you got your certificate from, but if you're creating it programmatically through Windows COM using IX509Enrollment, then you need to set the KeySpec for the IX509PrivateKey to XCN_AT_KEYEXCHANGE.

A different thing you need to get right, is to make sure the certificate supports 'document signing'. For that make sure the IX509ExtensionEnhancedKeyUsage contains XCN_OID_KP_DOCUMENT_SIGNING or 1.3.6.1.4.1.311.10.3.12.

Also, the COM class names behind the above COM interfaces are documented on the IX509EnrollmentWebClassFactory::CreateObject page.

Answer 2

You should also check if the (service) user addressing the certificate has enough rights on the local machine to access the certificates. That was my problem.