Key Signature Validates in Python, Won't in Java?

Question

I generated a keypair in python using pycrypto

key=RSA.generate(bit_size,os.urandom)

exportedPrivateKey = key.exportKey('PEM', None, pkcs=1).decode("utf-8")
exportedPublicKey = key.publickey().exportKey('PEM', None, pkcs=1).decode("utf-8")

I wrote a tiny utility that takes the hash of a message and signs the hash...

hash = MD5.new(json_info.encode("utf-8")).digest()
privateKey = RSA.importKey(USER_TOKEN_PRIVATE_KEY)
signature = privateKey.sign(hash,'')

I then wrote something that used the public key to verify that it validated okay..the signature in my tokens work fine..

hash = MD5.new(packet.encode("utf-8")).digest()
publicKey = RSA.importKey(tokenPublicKey)

if publicKey.verify(hash, signature):
    return json.loads(packet)
else:
    return None

Now, because I needed to use this in Java as well as python, I was porting a similar library to java, but I started running into issues. Namely, my validation would always fail...

I start by creating the PublicKey object from the PEM I exported...

byte[] encoded = Base64.decodeBase64(USER_TOKEN_PUBLIC_KEY);

//decode the encoded RSA public key
X509EncodedKeySpec keySpec = new X509EncodedKeySpec(encoded);
KeyFactory kf = KeyFactory.getInstance("RSA");
PublicKey pubKey = kf.generatePublic(keySpec);

I'm able to get the signature, and it's the exact same signature, and the value hashes to the exact same value (well, similar; java represents bytes as signed integers, whereas python represents them as unsigned, but they're the same binary representation). But it seems to always fail to validate my signature..here's what I'm using to do it:

byte[] hash = hasher.digest(packet.getBytes("UTF-8"));

InputStream hashStream = new ByteArrayInputStream(hash);

final Signature sign = Signature.getInstance("MD5withRSA");
sign.initVerify(pubKey);

byte[] buffer = new byte[256];
int length;
while ((length = hashStream.read (buffer)) != -1)
    sign.update (buffer, 0, length);

hashStream.close();

System.out.println(sign.verify(signature.getBytes("UTF-8")));

Unfortunately, this only prints false.

The only difference I can really see is that when I pass it to verify in Java, it asks for an array of longs, whereas in python it wants a byte sequence. My best guess was to take the string representation of that long and convert it into a bunch of bytes, but that failed. All of my other attempts failed as well (look at the byte representation of the underlying big integer, look at the byte representation of the array, etc). I feel like I'm missing something REALLY simple, but for the life of me I can't figure out what it is...

For an example of what the signature looks like, in python, I'm given:

[688304594898632574115230115201042030356261470845487427579402264460794863484312‌​120410963342371307037749493750151877472804877900061168981924606440672704577286260‌​395240971170923041153667805814235978868869872792318501209376911650132169706471509‌​89646220735762034864029622135210042186666476516651349805320771941650]

Answer 1

You are handling the signature as a Java String, using the UTF-8 encoding of that string as the signature value. As the signature can be any encoding, including bytes that do not encode into a printable string, that cannot be correct.

[EDIT]

OK, so the integer looks like a 1024 bit signature represented as a number between brackets. So this code should help:

import java.math.BigInteger;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class SignatureFromPython {
    private static final Pattern PAT = Pattern.compile("\\[(\\d+)\\]");

    private static byte[] i2osp(final BigInteger i, final int bitSize) {
        if (i == null || i.signum() == -1) {
            throw new IllegalArgumentException(
                    "input parameter should not be null or negative");
        }

        if (bitSize < Byte.SIZE) {
            throw new IllegalArgumentException(
                    "bitSize parameter should not be negative and a multiple of 8");
        }

        final int byteSize = (bitSize - 1) / Byte.SIZE + 1;
        final byte[] signedBigEndian = i.toByteArray();
        final int signedBigEndianLength = signedBigEndian.length;
        if (signedBigEndianLength == byteSize) {
            return signedBigEndian;
        }

        final byte[] leftPadded = new byte[byteSize];

        if (signedBigEndianLength == byteSize + 1) {
            System.arraycopy(signedBigEndian, 1, leftPadded, 0, byteSize);
        } else if (signedBigEndianLength < byteSize) {
            System.arraycopy(signedBigEndian, 0, leftPadded, byteSize
                    - signedBigEndianLength, signedBigEndianLength);
        } else {
            throw new IllegalArgumentException(
                    "Integer i is too large to fit into " + bitSize + " bits");
        }
        return leftPadded;
    }

    public static String toHex(final byte[] data) {
        final StringBuilder hex = new StringBuilder(data.length * 2);
        for (int i = 0; i < data.length; i++) {
            hex.append(String.format("%02X", data[i]));
        }
        return hex.toString();
    }

    public static void main(String[] args) {
        String sigString = "[68830459489863257411523011520104203035626147084548742757940226446079486348431212041096334237130703774949375015187747280487790006116898192460644067270457728626039524097117092304115366780581423597886886987279231850120937691165013216970647150989646220735762034864029622135210042186666476516651349805320771941650]";
        Matcher sigMatcher = PAT.matcher(sigString);
        if (!sigMatcher.matches()) {
            throw new IllegalArgumentException("Whatever");
        }
        BigInteger sigBI = new BigInteger(sigMatcher.group(1));
        // requires bouncy castle libraries
        System.out.println(toHex(i2osp(sigBI, 1024)));
    }
}

[EDIT2]

privateKey.sign(hash,'') uses "raw" RSA signatures. It is required to use PKCS115_SigScheme instead.

To be more secure, try and use PSS style signatures and a higher key size. Furthermore, the use of MD5 is broken for signature applications. Use either SHA-256 or SHA-512 instead.