Here's what I managed to get working so far.
1. Building a CSR, signing it with some other engine
I mostly followed demos/x509/mqreq.c
, with some twists.
(NB: error checking, fancy modulus length/label/subject DN generation/handling has been left out for brevity and focus on actual flow).
unsigned char* mod = NULL;
unsigned char* exp = NULL;
size_t mod_l = 0;
size_t exp_l = 0;
P11_handle h_key = P11_gen_rsa(&mod, &mod_l, &exp, &exp_l);
RSA* rsa = RSA_new();
rsa->n = BN_bin2bn(rsa_mod, rsa_mod_l, NULL);
rsa->e = BN_bin2bn(rsa_exp, rsa_exp_l, NULL);
EVP_PKEY* pkey = EVP_PKEY_new();
EVP_PKEY_assign_RSA(pkey, rsa);
X509_REQ* csr = X509_REQ_new();
X509_REQ_set_pubkey(csr, pkey);
/* Boring X509_NAME/X509_EXTENSION stuff */
X509_REQ_INFO* csr_req = csr->req_info;
unsigned char* pop_in = NULL;
size_t pop_in_l = ASN1_item_i2d((void*)csr_req, &pop_in,
ASN1_ITEM_rptr(X509_REQ_INFO));
unsigned char* sig = NULL;
size_t sig_l = 0;
P11_make_pop(SIGN_RSA_PKCS, DIGEST_SHA256,
pop_in, pop_in_l, &sig, &sig_l,
h_key);
/* Add signature to CSR (heavily inspired from ASN1_item_sign_ctx())
* (please don't ask about the flags) */
if (csr->signature->data != NULL) OPENSSL_free(csr->signature->data);
csr->signature->data = sig;
csr->signature->length = sig_l;
csr->signature->flags&= ~(ASN1_STRING_FLAG_BITS_LEFT|0x07);
csr->signature->flags|= ASN1_STRING_FLAG_BITS_LEFT;
/* Add signature algorithm information to CSR */
int sig_algo_nid = 0;
OBJ_find_sigid_by_algs(&sig_algo_nid,
EVP_MD_nid(EVP_sha256()), EVP_PKEY_RSA);
X509_ALGOR_set0(csr->sig_alg, OBJ_nid2obj(sig_algo_nid),
V_ASN1_NULL, NULL));
After that, the X509_REQ structure is good for PEM export. openssl req -verify
seems to validate the process, so as far as I'm concerned this works.
2. Building nested CMS structures (as in SignedData(EnvelopedData(Data)))
Finally got it, using 1.0.2 (any previous version would have needed patching or ASN.1-level parsing). Many thanks to Dr. Stephen Henson and Tom Francis for helping me with this via the mailing list.
/* Make EnvelopedData structure */
BIO* in = BIO_new_file(in_path, "rb");
int flags = CMS_BINARY | CMS_USE_KEYID | CMS_PARTIAL | CMS_KEY_PARAM;
CMS_ContentInfo* edata = CMS_encrypt(NULL, NULL, cipher, flags);
CMS_RecipientInfo* r_info = CMS_add1_recipient_cert(edata, r_cert, flags);
EVP_PKEY_CTX* wrap_ctx = CMS_RecipientInfo_get0_pkey_ctx(r_info);
EVP_PKEY_CTX_set_rsa_padding(wrap_ctx, RSA_PKCS1_OAEP_PADDING);
EVP_PKEY_CTX_set_rsa_oaep_md(wrap_ctx, EVP_sha256());
EVP_PKEY_CTX_set_rsa_mgf1_md(wrap_ctx, EVP_sha256());
EVP_PKEY_CTX_set0_rsa_oaep_label(wrap_ctx, oaep_label, oaep_label_l);
/* NB: oaep_label must be heap-allocated, and will be freed by OSSL */
CMS_final(edata, in, NULL, flags);
BIO* tmp = BIO_new(BIO_s_mem());
i2d_CMS_bio(tmp, edata);
/* Make SignedData structure */
flags|= CMS_NOSMIMECAP | CMS_NOCERTS;
flags&= ~(CMS_KEY_PARAM);
CMS_ContentInfo* sdata = CMS_sign(NULL, NULL, NULL, NULL, flags);
ASN1_OBJECT* ectype_edata = OBJ_nid2obj(NID_pkcs7_enveloped);
CMS_set1_eContentType(sdata, ectype_edata);
CMS_SignerInfo* s_info =
CMS_add1_signer(sdata, s_cert, s_key, NULL, flags);
CMS_SignerInfo_sign(s_info);
CMS_final(sdata, tmp, NULL, flags);
BIO* out = BIO_new_file(out_path, "wb");
i2d_CMS_bio(out, sdata);
BIO_flush(out);
3. Parsing the structure and getting the fields I need.
I basically wrote my own CMS parser. ASN.1 is actually simple to parse when you know the spec. I've tried compiling the ASN.1 modules in the RFC using some "ASN.1 to C structs" compilers but had no luck (they kept choking on the syntax).