PKCS #7 SignedData issue in bouncycastle

Question

I there!

I have a problem with an openssl generated smime file. I need to parse (with the help of BouncyCastle) an smime SignedData package, generated with openssl. I used this command to create an example:

openssl smime -sign -inkey Signer/signer_key.pem -in Signer/message.txt -text -out SMIME/signed.data -signer Signer/signer_cert.pem -certfile cacert.pem -outform smime

The problem is that I need to fill a pkcs7 ASN.1 structure in the process, and then apply the algorithm described in the standard, which can be found here: pkcs7 signedData

The ASN.1 structure is the following:

SignedData ::= SEQUENCE { version Version, 
                          digestAlgorithms DigestAlgorithmIdentifiers, 
                          contentInfo ContentInfo, 
                          certificates [0] IMPLICIT ExtendedCertificatesAndCertificates OPTIONAL,
                          crls [1] IMPLICIT CertificateRevocationLists OPTIONAL,
                          signerInfos SignerInfos }

SignerInfo ::= SEQUENCE { version Version, 
                          issuerAndSerialNumber IssuerAndSerialNumber, 
                          digestAlgorithm DigestAlgorithmIdentifier, 
                          authenticatedAttributes [0] IMPLICIT Attributes OPTIONAL,
                          digestEncryptionAlgorithm DigestEncryptionAlgorithmIdentifier,
                          encryptedDigest EncryptedDigest, 
                          unauthenticatedAttributes [1] IMPLICIT Attributes OPTIONAL }

The problem occurs in the verification of the signature. It is said in the standard that

The result of the message digest calculation process depends on whether the signedAttrs field is present. When the field is absent, the result is just the message digest of the content as described above. When the field is present, however, the result is the message digest of the complete DER encoding of the SignedAttrs value contained in the signedAttrs field.

In the smime package I have the plaintext, and I'm calculating its digest and comparing to the result of reading the encryptedDigest from the smime file, and then decrypting it. Assuming it has no attributes. But it looks obvious to me that somehow, the signature contained in the smime package is calculated based not only on the plaintext, but also on some attributes. What I want to know is a way of making this work, by appending the cleantext the necessary attributes in order that the digest of both equals the encryptedDigest field read from the SMIME package.

Below is the main I'm using. I removed the (huge amount of) debugging parts in order to simplify, and placed some english comments, replacing the portuguese ones. I think it's quite simple to understand it, but I can post my entire code if someone gets interested in this.

    package signedData;

import PKCS7.SignedData;
import java.io.ByteArrayOutputStream;
import java.io.FileInputStream;
import java.io.ObjectOutputStream;
import java.security.MessageDigest;
import java.security.Security;
import java.security.cert.CertStore;
import java.security.cert.X509Certificate;
import java.util.Arrays;
import java.util.Collection;
import java.util.Iterator;
import java.util.Properties;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.mail.Session;
import javax.mail.internet.MimeMessage;
import javax.mail.internet.MimeMultipart;
import org.bouncycastle.cms.SignerInformation;
import org.bouncycastle.cms.SignerInformationStore;
import org.bouncycastle.mail.smime.SMIMESigned;
import org.bouncycastle.util.Store;
import sun.misc.BASE64Encoder;


public class Main {

    public static void main(String[] args) {

        String ksFile = "Recipient/ks_recipient", ks_type = "JCEKS", key_alias = "recipient_pkcs12", cert_alias = "cacert", algorithm, provider = "BC";
        byte[] encrypted, encrypted2,  decrypted, digest;
        Cifra cipher;
        Digest dgst;
        X509Certificate cert;
        Properties props;
        Session session;
        MimeMessage msg;
        SMIMESigned signed;
        SignerInformationStore signers;
        SignerInformation s;
        CertStore certs;
        Collection c;
        Iterator it, certIt;
        ByteArrayOutputStream baos;
        ObjectOutputStream oos;


        if( args.length == 1) {

            /**
             * args[0] - SMIME's Path file
             */


            try {

                props = System.getProperties();
                session = Session.getDefaultInstance(props);

                msg = new MimeMessage(session, new FileInputStream(args[0]));
                signed = new SMIMESigned((MimeMultipart) msg.getContent());

                /* Putting information from various parties ( contained in the SMIME package) in a Collection, in order to process the information from each individually */
                signers = signed.getSignerInfos();
                c = signers.getSigners();

                /* Read certificates from SMIME package */
                certs =   (CertStore) signed.getCertificatesAndCRLs("Collection", provider);


                /* Iterate through the signers */
                it = c.iterator();

                while(it.hasNext()) {

                    /* Isolate each signer information */
                    s = (SignerInformation) it.next();

                    Collection certCollection = certs.getCertificates(s.getSID());

                    /* Iterate through this chain's certificates */
                    certIt = certCollection.iterator();
                    cert = (X509Certificate) certIt.next();


                    /* Verify signer certificate with CA */
                    if(Certificate_Handler.verifyCertificate( RW_KeyStore.getCertificate(ksFile, ks_type, cert_alias), cert, Security.getProvider(provider))) {

                        /* Read encryption algorithm identifier from SMIME package */
                        algorithm = Gadgets.getBC_Algorithm(s.getEncryptionAlgOID());

                        /* Read encrypted signature bytes */
                        encrypted = s.toASN1Structure().getEncryptedDigest().getOctets();

                        /* Create an instance of Digest, which is responsible for computing message digests */
                        dgst = new Digest(Gadgets.getBC_DigestAlgorithm(s.getDigestAlgOID()), provider);

                        /* Specify the encryption algorithm */
                        cipher = new Cifra(algorithm, provider);

                        // decrypt digest
                        decrypted = cipher.decifrar(encrypted2, cert.getPublicKey());


                        /* Compute plaintext digest */
                        digest = dgst.computeMessageDigest(((String) signed.getContent().getContent()).getBytes());

                        /* Verify signature */
                        if( MessageDigest.isEqual(decrypted, digest) )
                            System.out.println("check");
                        else
                            System.out.println("fail");
                    }
                    else
                        throw new Exception("Invalid Certificate");
                }
            }
            catch (Exception ex) {

                Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
            }
        }
        else
            System.err.println("Invalid parameter number: " + args.length);
    }
}

And now an example of an SMIME file, which is the one I'm actually using:

    MIME-Version: 1.0
Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg="sha1"; boundary="----BF337FE7381F86CE88DBE2ED7A24396E"

This is an S/MIME signed message

------BF337FE7381F86CE88DBE2ED7A24396E
Content-Type: text/plain

Executive Mansion,
Washington, August 22, 1862.

Hon. Horace Greeley:
Dear Sir.

I have just read yours of the 19th. addressed to myself through the New-York Tribune. If there be in it any statements, or assumptions of fact, which I may know to be erroneous, I do not, now and here, controvert them. If there be in it any inferences which I may believe to be falsely drawn, I do not now and here, argue against them. If there be perceptable in it an impatient and dictatorial tone, I waive it in deference to an old friend, whose heart I have always supposed to be right.

As to the policy I "seem to be pursuing" as you say, I have not meant to leave any one in doubt.

I would save the Union. I would save it the shortest way under the Constitution. The sooner the national authority can be restored; the nearer the Union will be "the Union as it was." If there be those who would not save the Union, unless they could at the same time save slavery, I do not agree with them. If there be those who would not save the Union unless they could at the same time destroy slavery, I do not agree with them. My paramount object in this struggle is to save the Union, and is not either to save or to destroy slavery. If I could save the Union without freeing any slave I would do it, and if I could save it by freeing all the slaves I would do it; and if I could save it by freeing some and leaving others alone I would also do that. What I do about slavery, and the colored race, I do because I believe it helps to save the Union; and what I forbear, I forbear because I do not believe it would help to save the Union. I shall do less whenever I shall believe what I am doing hurts the cause, and I shall do more whenever I shall believe doing more will help the cause. I shall try to correct errors when shown to be errors; and I shall adopt new views so fast as they shall appear to be true views.

I have here stated my purpose according to my view of official duty; and I intend no modification of my oft-expressed personal wish that all men every where could be free.

Yours,
A. Lincoln.
------BF337FE7381F86CE88DBE2ED7A24396E
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"

MIIQDQYJKoZIhvcNAQcCoIIP/jCCD/oCAQExCzAJBgUrDgMCGgUAMAsGCSqGSIb3
DQEHAaCCDHAwggX0MIID3KADAgECAgIBHzANBgkqhkiG9w0BAQUFADCBgjELMAkG
A1UEBhMCUFQxDjAMBgNVBAgTBUJyYWdhMQ4wDAYDVQQHEwVCcmFnYTELMAkGA1UE
ChMCVU0xCzAJBgNVBAsTAkRJMRAwDgYDVQQDEwdMRVNJIENBMScwJQYJKoZIhvcN
AQkBFhhwZzIwMTg1QGFsdW5vcy51bWluaG8ucHQwHhcNMTIwMjAxMDIwNzE1WhcN
MTMwMTMxMDIwNzE1WjB6MQswCQYDVQQGEwJQVDEOMAwGA1UECBMFQnJhZ2ExCzAJ
BgNVBAoTAlVNMQswCQYDVQQLEwJESTEfMB0GA1UEAxMWSm9hbyBQZWRybyBKb3Jn
ZSBDZXNhcjEgMB4GCSqGSIb3DQEJARYRanBqYzEwN0BnbWFpbC5jb20wggIiMA0G
CSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDsii4ImDJyK8PcBYCoz3/pMTVBYxkM
djhcRe7Le7bBU10CD7avxrPziroQu605DJUNDHLFNHgcJNt1k/fIW7LVqDuu+zJa
r0BUXcae4ojGJBmtojp8I7Nv4LSJDk89VSCDL54jEOtc8utayY7UOuS/ZI+99Ptp
OTZYkR84HDgDpPO1DpF7L2NVeW1Mq5kVn5BsFPCy24nlfAAZrWiK23Jm4s46/dfD
+dlHev5b1gGhKBPrs9U0OzvFYf3BLWglnzAatTLG4uqgB6lI76NQJsC4pSNNg+al
YGQXC9+Vr97B8j7ebeLZENERKyk0eHgtqx9IkIbVhjHHh3InSSXWr91BvL4yv9YV
dw336kC5AF9a//V1LLcgKcGal9HVRM6ruAHywvOE4E9uh10e2QxgGbowY/UxLFkj
/MbPfd4gx+fOQFsgXRF2ibYR4tPOa7GHoMNYUKqTnR6iGrxl+tsk1GWa34bF38Nx
4ia4jSpMR95pRt/2Bqw3cwG9ctogdk5xoOfA3vJspUbzyp+KmHi+tTSBRjPi8W1W
2sC1JQxr3AXYBHQwccJlVXyY2o99mBcixC5ZqD+9pSg/e/pE1j3t0HgXleEP/PTE
OYHm5jHNdhxFqQeW2xv0jHwPtimqacQMkC4ko6WfQhkcOBBXRjd50S6kOI2KupTe
RAbQNVGcawEJGwIDAQABo3sweTAJBgNVHRMEAjAAMCwGCWCGSAGG+EIBDQQfFh1P
cGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQURlZLZmo0DWJT
y+DtuRkjYbGaUdcwHwYDVR0jBBgwFoAUjAfWC3H0Y/0R7AOSVZ0qxUi82nAwDQYJ
KoZIhvcNAQEFBQADggIBAENNtqOelajUS3LyyKzm2ZIHSnZcOgThSFDVChOoRnT/
J4U8oyg7ug1Hhwo7ayOp0iS5PzuhNFr1oZtj9TO76l7O9OWI6qIT/M/Ggzvn0iZk
VeIv6ql52Z5n+x+TW2iu9VIgswU3yMWBe006+ciFyvpzA6nma6cfeqAEv1Ip/z/q
ci3Q7a33NJV4TmYqip4FWrhlZYTpq/AOVlxysgP8lKCVHpZ4UOSg2Tz2F5+7NBp4
yR6HSGHkHgwvA5Rf4Rl6Wxc9cVICfxUh7G2j/EDvnFbj30JRYyEDwmVOlS2PQs4r
816MHmtdXcojytq4QjgckaV9h7lD73BsIqlTZZPsdIpCcGl4VMCnJhvLH92FsFYA
Df2trzFWPH5QxWdYrypC8RMv58qxFcTaN+CcMB6mFBh6kMkLIHv69dlrTM84vqgb
7cO1HQnenODVAhnP25kChAhgYMQDlffijllSWQPuP9PtUw4rOvHP62MIlV17rQLh
9E3OTg9v4uXtISSsjPehcY2tCIhA01xTGYorr/IL0tCl3bnArMW7RnhJSYSPPjOb
mDM3dUua97Eeji5L79ntX0xDWp8FchTGxQCnfzmixrSbDDy7T2zW7fcg4HEYcfl/
9ObSOIs8Do/+t5Nrgdan7LA391dtOZRlScjgE+vKr1l+Zr86pnnblO4mmW47L1HP
MIIGdDCCBFygAwIBAgIJAJPdIDpA9PesMA0GCSqGSIb3DQEBBQUAMIGCMQswCQYD
VQQGEwJQVDEOMAwGA1UECBMFQnJhZ2ExDjAMBgNVBAcTBUJyYWdhMQswCQYDVQQK
EwJVTTELMAkGA1UECxMCREkxEDAOBgNVBAMTB0xFU0kgQ0ExJzAlBgkqhkiG9w0B
CQEWGHBnMjAxODVAYWx1bm9zLnVtaW5oby5wdDAeFw0xMjAyMDEwMjA1MzJaFw0x
MjAzMDIwMjA1MzJaMIGCMQswCQYDVQQGEwJQVDEOMAwGA1UECBMFQnJhZ2ExDjAM
BgNVBAcTBUJyYWdhMQswCQYDVQQKEwJVTTELMAkGA1UECxMCREkxEDAOBgNVBAMT
B0xFU0kgQ0ExJzAlBgkqhkiG9w0BCQEWGHBnMjAxODVAYWx1bm9zLnVtaW5oby5w
dDCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBANZhB/KUpzHEPB1cG+mv
FcmVUSwUAzafyC+4PCrmaWKwUHz6Y4QbhcqC+cYOQtrEmW2k2BK8gBG8aeCpmQq/
7JRABJBHECOjpqEIDg1k4SUG09cJ3EZ2j3kUdqlvXwwUQJrA9WGIb9hln+noDcbw
zFxaOjn/FdgF23Hn5KOK9lU3ARdFXGhk/RYOrNkaFveH8aV4W9R6P6FbT3VvOW3v
pC93qaeSzpCp04XbwGrMXkQ8QwTRzufAEpETd4aEwDu+kurMLump5SVDJbRw4BaG
6NrRKOl/8C4tdawLSUsEhNvglMb+c1oduB4CxyhHU6yyVoRFyGNX4PWMW8od7wx3
TvmQUm0vR7qIIu0Vq1OPsotOJz9/cZ7Ci0Z3BySQDHjDOhUGZo/bRWoeQf+BNpBU
vlj5tfcJg/YXbwsGB+ONC1hguIWuIi+l6tEG2Dj2kHHuxKaCKhTzMFUgzlN+Xowi
ksBD6sbtSsGYYpYhy85Q4RlvsHvf7wnpdaKhvcGFYwz8jkkgJmISATUXz4kjJvgC
mZr3yeQpARXH+BiKZ3MaGzw9RdM8gsJUSkq0Ljy2Qr4cQ4s4mkLNeHPZdDjmhRsK
XtyMsWVRg/FAQYBxXEJD1j587hP4jddU1vsmR8kHPyjzL6uxMv5OyNDCYnyD663u
1qeG75egYH7Vm1n/nc37xXhxAgMBAAGjgeowgecwHQYDVR0OBBYEFIwH1gtx9GP9
EewDklWdKsVIvNpwMIG3BgNVHSMEga8wgayAFIwH1gtx9GP9EewDklWdKsVIvNpw
oYGIpIGFMIGCMQswCQYDVQQGEwJQVDEOMAwGA1UECBMFQnJhZ2ExDjAMBgNVBAcT
BUJyYWdhMQswCQYDVQQKEwJVTTELMAkGA1UECxMCREkxEDAOBgNVBAMTB0xFU0kg
Q0ExJzAlBgkqhkiG9w0BCQEWGHBnMjAxODVAYWx1bm9zLnVtaW5oby5wdIIJAJPd
IDpA9PesMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADggIBAI4iEmfkYmYN
LHtTcyzztSTKaczV1LZEgX6FiXI1s9zAL4OZbU3cvWiYzyFWUbKUvHZTkL6MiQCQ
kjppCRZd39cBsoDHI/nbFh2WHMfXrZdkPm+5ifOokUV4MeR7lp4d0Lj4Jy1LdL82
s5p1duu/9Bw/dE/1lKnu/jsSx6pL/IWQ8mRIsVYhJPjcTypKvOONc2GJNqG/+LYk
TjZIAZjJyw+sJs1g+6byEL+KXlQtjJ6AIMteryfOqP3+VgSj8IFYtDEkVPbM24n8
y1B/IXv1wMD9P9pQM3htkZC5cV5obqAL0yrn+9tcU1TdzDEKZtI3PKE7ulto0Ipj
GSokpxe9WyMlvQnoZMX1tUKGIcfXtBWctJWcLmKRG3+IJPfc7kZffjEaAEqUR1RP
tM48Aj8XTwDyleULWySh6p9XqOzx5gzQqeMsVxDTTe5FtiDYVyhXTFZKZOlFqW+w
vUfuEzGUVexqeNIga+4nkNS1FF0n/5ZPSC46ix8BXfJkUcogCOdPlURqiBXn5mIP
Gb8Sp/t48YPY5tot8Tp3vKWnhFr7s4JEwsYXq1czoKBBDGllnqOZ2MbasZrJwipw
yozWW9MM71nVvRRSd6UJIxSQSFESR6cH7Byl179kF7PUIa88Zy765Uxzft4S6b2j
0FcMxjIvEAZQF8NoATiWM/qAhBFcCJREMYIDZTCCA2ECAQEwgYkwgYIxCzAJBgNV
BAYTAlBUMQ4wDAYDVQQIEwVCcmFnYTEOMAwGA1UEBxMFQnJhZ2ExCzAJBgNVBAoT
AlVNMQswCQYDVQQLEwJESTEQMA4GA1UEAxMHTEVTSSBDQTEnMCUGCSqGSIb3DQEJ
ARYYcGcyMDE4NUBhbHVub3MudW1pbmhvLnB0AgIBHzAJBgUrDgMCGgUAoIGxMBgG
CSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTEyMDIwNzE3
Mzg1N1owIwYJKoZIhvcNAQkEMRYEFBUmj01YomLqORfCsJHc0YS+6izcMFIGCSqG
SIb3DQEJDzFFMEMwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3
DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMA0GCSqGSIb3DQEBAQUABIIC
AEUyazaDG4N4bBDCx0R4haGSjtr6eQtAcyjtkwRf4W5HZ1SR4z4ap5Jk/bCXZUjI
Z/ip6ydX2oeuHQmnarjbLi5d5GlJ2ve5vTdbrH5iCBAPguE0rK/+rtSORT1hb8YV
GSG0gx87Tat5drQ7AjazYpnUpQkjbbb+BaLsi/bdE//V5Gb2epwtl95vLA5Z4OMU
YzrUo8N4BtlTwSlYXvTOyXcU3XSN207BY8jUxY41GUQOl4FAuybkIvHiQ/jZFejF
7NQvnhZOWvZAxD9m3+DDHZZAocMXVdX8w3RT84agDwudcl37YazCK+xTMGyA33Bn
HdvoXm/+q0Gg2c699bTZDnvk7wVXcTGwIs7js9Mt94jQbHoVsBAbzHEgRZY300/e
Tnoz+PxmberFbFBD8QHHLcMclsf09iBZDjkU+Y+vKjAQyqDGKDVwdWQ03jb0rQcz
gkoNMmle1HrH0tWfFY/FdwNCoOu5gd12vzawzF2Tde9J1JPRpJU8Swkp9friYur2
uqJwUp5pWgvd/92MnUpsuDaX29dHUcCcUUfa17ykuUw+htwzBpP2NsvkDWeNFA7N
2B3Tx1MTBNZutV2N13ncpq0DISyMoz4Gp4wptcvjHdYOC3Uaxz75COSGJpKb/XPL
4+O0uY53u4xfh/6OToAGGdU+wvTXxRrJ+i0fwBpPK5sE

------BF337FE7381F86CE88DBE2ED7A24396E--

Answer 1

I solved it out. The (pkcs7) standard creates an auxiliar structure (DigestInfo) after calculating the digest.

DigestInfo ::= SEQUENCE { digestAlgorithm DigestAlgorithmIdentifier, 
                          digest Digest }

After deciphering the EncryptedDigest that I read from the SMIME package, the result is a DigestInfo.

Then I compute the digest of the signedAttributes (for each signer):

SignerInformation.getSignedAttributes()

And finally place the result into a DigestInfo. Only then I check the validity of the signature. Et voila, it's working!

Hope this helps anyone who has the same problem as I did, as this was really a time consuming problem...