Encrypt String using PBEWithMD5AndDES to compare to stored hash with Java

Question

I am working with an application that stores challenge responses in some type of md5 hash. There is an api method to check that all the responses are correct, but for some reason there is not a method to check one response which is a requirement that I have. I'm not trying to decrypt. I'm just trying to encrypt the same way the application is doing it in order to compare. I do have a JCE library. I'm afraid I'm pretty new to encryption so please help a noob out with a detailed answer. Here's what I've gathered from a properties file:

algorithm=PBEWithMD5AndDES
password=pooface
digest=MD5

A sample hash looks like this:

MD5:MXgxY21tdXR4bjB0:oRu+jlpCO/eSdwMb0iTVbw==

They all return MD5:<16chars>:<24chars>. Any guidance is helpful. Thanks in advance.

Answer 1

Not really an "Answer", but a few pointers:

• In the absence of documentation, you'd probably be best off reverse-engineering the API implementation (if you have it, and it's legal) to figure out what it's doing. (From a point of principle, you ought to be in a position to demand documentation of the algorithm that the API uses to hash (or otherwise) the passwords, otherwise it's security-by-obscurity). In the absence of that:

• The oRu+jlpCO/eSdwMb0iTVbw== is Base64-encoded binary. If you Base64-decode it you get 128 bits, which is the size of an MD5 hash: 0xa11bbe8e5a423bf79277031bd224d56f.

• The MXgxY21tdXR4bjB0 is probably a salt (i.e. something added to the received password to protect against dictionary attacks: MD5("password") is known and easily pre-computed, whereas MD5("MXgxY21tdXR4bjB0password") isn't). This, together with the previous point, suggest that you compute the MD5 hash of the salt plus something else to produce the 128 bits that are produced by Base64-decoding the oRu... data.

• PBE stands for password-based encryption, i.e. generate a symmetric key (using MD5 in this case) from a password (perhaps pooface in this case...) then use that key to encrypt something (perhaps encrypt the user's password) with DES. See "AlgorithmParameters Algorithms" in http://docs.oracle.com/javase/7/docs/technotes/guides/security/StandardNames.html, and PKCS#5: http://www.emc.com/emc-plus/rsa-labs/standards-initiatives/pkcs-5-password-based-cryptography-standard.htm.

• DES is a block cipher with a block size of 64 bits, so the output (i.e. ciphertext) will be a multiple of 64 bits long. It's possible that this is stored directly in your sample hash, but unlikely, not least because that would mean that the "MD5" in your sample hash line would be meaningless.

• I suspect (but it's only a guess) that:

  1. Use PBEWithMD5AndDes to derive a DES key based on the password "pooface".
  2. Encrypt the user's password with that DES key.
  3. Hash the salt and ciphertext with MD5, giving 128 bits of hash output.
  4. Base64-encode the hash to give you the oRu... value.

• Other things to consider:

  • The salt might be used to derive the encryption key.

  • The salt might be used within the encryption.

  • You say "challenge response": this probably means that the authenticating application invents a challenge (probably a random number), then gets the client to compute something based on this challenge and knowledge of some secret. The challenge might be the salt (but there are a few reasons why that might not be the case).

  • In the PBE you have two inputs: the password (used to derive the encryption key), and the thing being encrypted. I've assumed that response is encrypted using a key derived (using PBE/MD5) from "pooface". It's possible that this is the wrong way round: derive a key (using PBE/MD5) using the challenge response, and use that to encrypt "pooface". That seems less likely to me though.

Presumably you can create your own user with a known password (or challenge response, or whatever they call it), i.e. you can provide known input and see what output you end up with. That's useful from a reversing perspective. When figuring out what crypto is going on, I'd also use a few utilities:

• Write a Java prog that takes a password and outputs a DES encryption key (i.e. implements PBEWithMD5AndDes).
• Familiarise yourself with the openssl command, which will do MD5 hashing, DES encryption and Base64 (en|de)coding on demand.

BTW, using MD5 and DES is a bit poor these days...