Why does invalid memset() after malloc() leads to free(): invalid next size (fast) [duplicate]

Question

This question already has an answer here:

free char*: invalid next size (fast) [duplicate] (1 answer)

Closed 9 years ago.

Code snippet, at the bottom, which I was trying lead to the following error

free(): invalid next size (fast)

Above error, was caused by declaration of integer variable mock after accidently calling memset() on pointer check with size more than what was originally allocated using malloc(). And when free() is called on pointer check it leads to a runtime error.

It would be helpful if somebody could explain how the internal memory is being manipulated or why this error actually occurred.

Thanks in advance.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main()
{
    char *check = (char *)malloc(sizeof(char) * 10);

    memset(check, 0, 100);

    memcpy(check, "memsetcpy", 10);

    int mock = 9;

    free(check);

    return 1;
}

EDIT My intention of asking the question was to know/understand what happens to the book-keeping information in the heap memory when the program runs. And may be to have some information on data structures that are modified would be great. OR I wanted to know from the design perspective.

Thanks everyone for chipping-in.

Answer 1

A typical dynamic memory (heap) implementation stores a lot of household information right there, in the very same memory, interleaved with user data. Each block of memory you allocate typically includes a special header region and, possibly, a special footer region. These regions surround your user region from both ends. You are only allowed to manipulate memory inside your user region. But the moment you write anything outside the boundaries of that user region, you most certainly damage those internal header and footer regions. This completely destroys the integrity of dynamic memory management subsystem. The rest follows.

In your case you go outside your user region by 90 bytes. This is a lot. You probably destroyed a lot of internal household data that is critical for proper operation of dynamic memory.

The specific details of this (what gets damaged and how it affects free later) are very, very, very implementation-dependent. Different implementations can be wildly different in that regard. There's no way to provide a more specific description without at least knowing what specific implementation you are using.

Answer 2

When the program writes outside defined memory space, anything could happen.

In this case, the program has damaged the heap.

Answer 3

This is the signature of memset:

void * memset ( void * ptr, int value, size_t num );

With malloc, you allocate 10 bytes of memory, and assign a pointer to that memory to check. (Not a solution, but you should not cast the return value of malloc)

Then with memset, you write 100 bytes of zeroes to the block of memory pointed to by check.

Your program writes correctly to the first ten bytes, then zeroes out whatever is in the 90 following the allocated space. Since you're writing outside of allocated memory, anything could happen, and in this case, I'd guess you're writing over memory that contains bookkeeping info for the malloc/free memory allocation system. Of course, since you're invoking undefined behavior, it would also be acceptable for it to cause your computer to grow legs and walk away.

Answer 4

In modern operating systems, allocated memory is managed in very large chunks from kernel space and then allocated in smaller chunks by user space code.

The user space code keeps track of the allocated and free memory typically through the use of a doubly linked list along with sentinel data before and after each allocated chunk. The sentinel data includes things like a magic number, chunk size, etc.

The sentinel data helps the memory manager detect when a user program writes past (or before) it's allocated size. The use of a doubly linked list helps the memory manager combine the freed smaller chunks into larger freed segments to help avoid fragmentation.

When you write beyond your allocated size, you not only destroy the sentinel data but also probably the linked-list data. Once the linked list gets destroyed, most memory managers fall apart thus the comments about things going off the reservation.

Hope this helps.