Stack around the variable 's' was corrupted

Question

I got this error in my rc4 algorithm, it works well, but i got this error every time when the message is too big, like 1000kB, here is the code:

char* rc4(const int* key, int key_size, char* buff, int buff_size){
int i, j, k;
int s[255], rk[255];    //rk = random_key
char* encrypted = alloc_char_buffer(buff_size);

for (i = 0; i < 255; i++){
    s[i] = i;
    rk[i] = key[i%key_size];
}


j = 0;

for (i = 0; i < 255; i++){
    j = (j + s[j] + rk[i]) % 256;
    SWITCH(s + i, s + j);
}

i = 0;
j = 0;

for (k = 0; k < buff_size; k++){
    i = (i + 1) % 256;
    j = (j + s[i]) % 256;
    SWITCH(s + i, s + j);

    //try{

    //}
    //catch ()
    encrypted[k] = (char)(s[(s[i] + s[j]) % 256] ^ (int)buff[k]);
}

encrypted[buff_size] = 0;

return encrypted;
}

at the end o the last loop i got this error, i think this is some type of buffer overflow error, the only variable able to do that is the 'encrypted' but at the end of the loop, the value of the variable 'k' have the exactly same value of 'buff_size' that is used to alloc memory for 'encrypted', if someone can help i'll thank you

the 'encrypted' is "non-null terminated", so if the string have 10 bytes i will allocate only 10 bytes, not 11 for the '\0'

if you need, here is the code for alloc_char_buffer(unsigned int)

char* alloc_char_buffer(unsigned int size){

char* buff = NULL;

buff = (char*)calloc(size+1, sizeof(char));

if (!buff)
    _error("program fail to alloc memory.");

return buff;
}

SWITCH:

//inversão de valores
void SWITCH(int *a, int *b){
*(a) = *(a) ^ *(b); //a random number
*(b) = *(a) ^ *(b); //get a
*(a) = *(a) ^ *(b); //get b
}

Answer 1

char* encrypted = alloc_char_buffer(buff_size);
/* ... */
encrypted[buff_size] = 10;

Here is the problem. You allocate buff_size elements. Thus, the last valid index is buff_size-1, not buff_size.

Another issue:

j = (j + s[j] + rk[i]) % 256;

Thus the range of j is [0, 255], but the legal index of s is only [0, 254]. You should either declare s as a 256-element array or review the algorithm implementation.

Answer 2

Your following line is creating the problem as you are trying to access beyond your allocated memory.

encrypted[buff_size] = 10;

Additionally, you should avoid use calloc instead of writing your own function alloc_char_buffer. It would allocate memory and initialize with 0.

calloc(buff_size, sizeof(char));