How to use digital certificate to check the author of a program?

Question

We develop a Win32 program (=host) which allows 3rd party to write plug-ins. As some plug-ins contains valuable piece of code (for example, high quality video scalar), the 3rd parties want to limit their plug-in to work only with our host program.

Our idea is to use Microsoft Authenticode technology to sign the host. Then, the 3rd parties are asked to implement the following algorithms to check the host. (The 3rd parties are expected to do sufficient code obfuscation for the algorithm).

1. Use WinVerifyTrust() API to verify the certificate of the host is valid (= Not revoked, not tampered, etc).

2. Verify the certificate that the subject is our company.

The question is about step (2). The 3rd parties cannot simply check thumb print or serial number because the digital certificate of the host will be renewed after the certificate expiration date.

My idea is to check parts of subject's distinguished name, specifically "country (C)" and "common name (CN)", assuming that there is no company name confliction in the United States. We shouldn't check other attributes such as state and city because our company might move - in fact, we have moved from one city to another just a year ago.

Question: Is it good way to accomplish the goal?

Answer 1

While the scheme is workable, it's possible to relatively easily circumvent protection by just patching plugins so that they ignore the signature or skip signature verification altogether.

What is even more important, - if you plan to have multiple plugins/vendors, you would have hard time ensuring that all vendors obfuscate validation code right.

Then, I'd say that it can be against plugin vendor's interest to limit their plugin to your application only - if they want bigger market, they might want to have the same plugin run on wider scope of hosts.