Unsure of parameter to store BIGNUM in char - BN_bn2bin()

Question

I have the following code:

/*
* Special note about BN_bn2bin():
*      OpenSSL's BN_bn2bin() will truncate numbers at the front, but ISAKMP
* wants leading zero's instead.
*/
void _BN_encode(BIGNUM *bn, unsigned char *p, int len)
{
        int     pad;

        assert(bn);
        if ((pad = len - BN_num_bytes(bn))) {
                if (pad < 0) {
                        assert(1);
                        return;
                }
                memset(p, '\0', 1);
                BN_bn2bin(bn, p + pad);
        } else {
                BN_bn2bin(bn, p);
        }
}

And I gather that *bn is the BIGNUM I want to convert, *p is where I want to store it, but what should the argument len be?

I was also looking at this question which seems related. Any advice would be appreciated!

Edit

Would something along the lines of this be correct?

_BN_encode(bignum, bin_bignum, sizeof(char));

Answer 1

len is the length of the buffer pointed to by p. The function pads out the converted bignum to len bytes (with zeroes, which doesn't change the mathematical value of the number represented).

That function appears to have a bug, by the way - I strongly suspect that the memset() call should be memset(p, 0, pad);.