Python Crypto RSA textbook encryption versus openssl - what is it doing?

Question

I have an existing set of Python code that uses the primitive (textbook) RSA encryption mechanism to encrypt small pieces of data. Specifically, the python code loads a public key into variable publickey and then encrypts this data using the following code:

ciphertext = publickey.encrypt(plaintext,None)

Yes I have read the disclaimer that "this function performs the plain, primitive RSA encryption (textbook). In real applications, you always need to use proper cryptographic padding,..." Unfortunately, I cannot change this code at the present time, so I am stuck with using this "textbook" encrypt command.

Note that even though this command does not use any padding scheme, it still seems to be able to encrypt any (appropriately small) amount of plaintext. In other words, the plaintext can be any length up to whatever limit is imposed by RSA.

Now however, I want to use the OPENSSL C library in an IOS app to do an identical RSA encryption. I cannot figure out how to do the exact same thing that the python function is doing. OPENSSL has a method RSA_public_encrypt where you pass in the size of the plaintext, the plaintext itself, an RSA object pointer which contains public key information, and the RSA padding mechanism. To replicate what python is doing in the crypto library, I thought I could use RSA_NO_PADDING as the padding mechanism. The problem is that OPENSSL states in their documentation that if you use RSA_NO_PADDING, then the length of the plaintext to encrypt must be EXACTLY equal to a certain value: RSA_Size(rsa) where rsa is a pointer to the RSA object passed in (the object that contains the public key).

In other words, the python crypto library seems to be able to encrypt variable length plaintext with no padding, but OPENSSL requires plaintext to be fixed length. So what exactly is the python crypto library doing to handle the variable size of the plaintext, and is there any way I can replicate this in OPENSSL?

Answer 1

If you look at the documentation for RSA_NO_PADDING, they explain:

This mode should only be used to implement cryptographically sound padding modes in the application code. Encrypting user data directly with RSA is insecure.

In other words, RSA_NO_PADDING is for cases where you're going to handle padding on your own, so OpenSSL expects an input of the proper size. It doesn't mean "textbook RSA".

Is there a compelling reason you can't change the python code? Textbook RSA is insecure and you're taking a great risk by using it.