Encrypting data in Cocoa, decoding in PHP

Question

The situation I'm trying to solve: in my Cocoa app, I need to encrypt a string with a symmetric cipher, POST it to PHP, and have that script decode the data. The process needs to work in reverse for returning an answer (PHP encodes, Cocoa decodes).

I'm missing something because even though I can get both the key and initialization vector (iv) to be the same in both PHP and Cocoa, the decoding never works when one app sends its encoded data to the other. Both work just fine encoding/decoding their own data (verified to make sure there wasn't some PEBKAC issue at hand). I have a suspicion that there's a padding issue someplace, I just don't see it.

My cocoa app encodes using SSCrypto (which is just a handy-dandy wrapper around OpenSSL functions). The cipher is Blowfish, mode is CBC. (forgive the memory leaks, code has been stripped to the bare essentials)

NSData *secretText = [@"secretTextToEncode" dataUsingEncoding:NSUTF8StringEncoding];
NSData *symmetricKey = [@"ThisIsMyKey" dataUsingEncoding:NSUTF8StringEncoding];

unsigned char *input = (unsigned char *)[secretText bytes];
unsigned char *outbuf;
int outlen, templen, inlen;
inlen = [secretText length];

unsigned char evp_key[EVP_MAX_KEY_LENGTH] = {"\0"};
int cipherMaxIVLength = EVP_MAX_IV_LENGTH;
EVP_CIPHER_CTX cCtx;
const EVP_CIPHER *cipher = EVP_bf_cbc();

cipherMaxIVLength = EVP_CIPHER_iv_length( cipher );
unsigned char iv[cipherMaxIVLength];

EVP_BytesToKey(cipher, EVP_md5(), NULL, [symmetricKey bytes], [symmetricKey length], 1, evp_key, iv);

NSData *initVector = [NSData dataWithBytes:iv length:cipherMaxIVLength];

EVP_CIPHER_CTX_init(&cCtx);

if (!EVP_EncryptInit_ex(&cCtx, cipher, NULL, evp_key, iv)) {
    EVP_CIPHER_CTX_cleanup(&cCtx);
    return nil;
}
int ctx_CipherKeyLength = EVP_CIPHER_CTX_key_length( &cCtx );
EVP_CIPHER_CTX_set_key_length(&cCtx, ctx_CipherKeyLength);

outbuf = (unsigned char *)calloc(inlen + EVP_CIPHER_CTX_block_size(&cCtx), sizeof(unsigned char));

if (!EVP_EncryptUpdate(&cCtx, outbuf, &outlen, input, inlen)){
    EVP_CIPHER_CTX_cleanup(&cCtx);
    return nil;
}
if (!EVP_EncryptFinal(&cCtx, outbuf + outlen, &templen)){
    EVP_CIPHER_CTX_cleanup(&cCtx);
    return nil;
}
outlen += templen;
EVP_CIPHER_CTX_cleanup(&cCtx);

NSData *cipherText = [NSData dataWithBytes:outbuf length:outlen];

NSString *base64String = [cipherText encodeBase64WithNewlines:NO];
NSString *iv = [initVector encodeBase64WithNewlines:NO];

base64String and iv are then POSTed to PHP that attempts to decode it:

<?php

import_request_variables( "p", "p_" );

if( $p_data != "" && $p_iv != "" )
{
    $encodedData = base64_decode( $p_data, true );
    $iv = base64_decode( $p_iv, true );

    $td = mcrypt_module_open( MCRYPT_BLOWFISH, '', MCRYPT_MODE_CBC, '' );
    $keySize = mcrypt_enc_get_key_size( $td );
    $key = substr( md5( "ThisIsMyKey" ), 0, $keySize );

    $decodedData = mcrypt_decrypt(MCRYPT_BLOWFISH, $key, $encodedData, MCRYPT_MODE_CBC, $iv );
    mcrypt_module_close( $td );

    echo "decoded: " . $decodedData;
}
?>

decodedData is always gibberish.

I've tried reversing the process, sending the encoded output from PHP to Cocoa but EVP_DecryptFinal() fails, which is what leads me to believe there's a NULL padding issue somewhere. I've read and re-read the PHP and OpenSSL docs but it's all blurring together now and I'm out of ideas to try.

Answer 1

I think your problem is that the method of deriving the raw encryption key from the key string is different on the two sides. The php md5() function returns a hexadecimal string, i.e 'a476c3...' which you are chopping down to the key size, while EVP_BytesToKey() is a fairly complicated hash routine that return a raw byte string. It might, with the parameters supplied simplify down to a raw MD5 hash, but I can't really tell. Either way, it's going to be different from the php hash.

If you change the php to md5( "ThisIsMyKey", TRUE ), that will give you a raw md5 hash. On the cocoa side of things, SSCrypto's +getMD5ForData: method should generate the same one for the same string (text encoding issues aside).

Edit 1: If the php string and Cocoa data print out identically, they're still different at the byte level. The php string is hex-encoded (i.e consists only of characters 0-9 and a-f) while the cocoa data is the raw bytes (although NSData helpfully prints out a hex-encoded string of its contents when NSLogged). You still need to add the second TRUE parameter to php's md5() function to get the raw byte string.

Edit 2: OpenSSL 1.1.0c changed the digest algorithm used in some internal components. Formerly, MD5 was used, and 1.1.0 switched to SHA256. Be careful the change is not affecting you in both EVP_BytesToKey and commands like openssl enc.

Answer 2

I figured out my problem. The short answer: the key being used was of different lengths under Cocoa and PHP. The long answer...

My original inquiry was using Blowfish/CBC which is a variable key length cipher from 16 bytes to 56. Going off of Boaz's idea that the key was somehow to blame, I switched to TripleDES for the cipher as that uses a fixed key length of 24 bytes. It was then I noticed a problem: the key returned by Cocoa/EVP_BytesToKey() was 24 bytes in length, but the value returned by md5() hashing my key was only 16.

The solution to the problem was to have PHP create a key the same way EVP_BytesToKey does until the output length was at least (cipherKeyLength + cipherIVLength). The following PHP does just that (ignoring any salt or count iterations)

$cipher = MCRYPT_TRIPLEDES;
$cipherMode = MCRYPT_MODE_CBC;

$keySize   = mcrypt_get_key_size( $cipher, $cipherMode );
$ivSize    = mcrypt_get_iv_size( $cipher, $cipherMode );

$rawKey = "ThisIsMyKey";
$genKeyData = '';
do
{
    $genKeyData = $genKeyData.md5( $genKeyData.$rawKey, true );
} while( strlen( $genKeyData ) < ($keySize + $ivSize) );

$generatedKey = substr( $genKeyData, 0, $keySize );
$generatedIV  = substr( $genKeyData, $keySize, $ivSize );

$output = mcrypt_decrypt( $cipher, $generatedKey, $encodedData, $cipherMode, $generatedIV );

echo "output (hex)" . bin2hex($output);

Note that there'll most likely be PKCS#5 padding on the end of that output. Check out the comments here http://us3.php.net/manual/en/ref.mcrypt.php for pkcs5_pad and pkcs5_unpad for adding and removing said padding.

Basically, take the raw md5 value of the key and if that isn't long enough, append the key to the md5 result and md5 that string again. Wash, rinse, repeat. The man page for EVP_BytesToKey() explains what it's actually doing and shows where one would put salt values, if needed. This method of regenerating the key also correctly regenerates the initialization vector (iv) so it's not necessary to pass it along.

But what about Blowfish?

EVP_BytesToKey() returns the smallest key possible for a cipher as it doesn't accept a context by which to base a key size from. So the default size is all you get, which for Blowfish is 16 bytes. mcrypt_get_key_size(), on the other hand, returns the largest possible key size. So the following lines in my original code:

$keySize = mcrypt_enc_get_key_size( $td );
$key = substr( md5( "ThisIsMyKey" ), 0, $keySize );

would always return a 32 character key because $keySize is set to 56. Changing the code above to:

$cipher = MCRYPT_BLOWFISH;
$cipherMode = MCRYPT_MODE_CBC;

$keySize   = 16;

allows blowfish to decode properly but pretty much ruins the benefit of a variable length key. To sum up, EVP_BytesToKey() is broken when it comes to variable key length ciphers. You need to create a key/iv differently when using a variable key cipher. I didn't go into it much because 3DES will work for what I need.