Can the operating system “break up” a memory allocation (Linux)?

Question

Let's say a process (P1) is asking for 100 MB of memory, and the RAM looks like this:

[[50 MB free] [USED] [60 MB free] [USED]]

Since there are technically enough memory that are available (110MB free), what would happen? According to some sources I saw online, the OS will just refuse to allocate the memory, but then again isn't Linux only supposed to throw a memory error when there aren't enough memory?

Thanks

Answer 1

If the process asks for 100MB, there has to be a contiguous 100MB free in the address space of the process (virtual memory), but those addresses can be mapped to non-contiguous pages of physical memory. The allocated addresses might not even be backed by physical pages until they are written to – Linux happily overcommits memory.

On 64 bit systems it's extremely rare that there wouldn't be enough space in the address space since there can be up to 256TB worth of virtual addresses on x86-64 hardware. This space is also not allocated in a linear manner. Due to using the ASLR security feature, parts of the virtual address space are allocated at random locations, with large parts of unmapped addresses in between.

Memory that has been dynamically allocated by the operating system to the process (e.g. with sbrk() or mmap()) has to be managed by the process itself. The application might use a garbage collector or C standard library functions like malloc() for this task. It is rather common that this allocated memory becomes fragmented over time, i.e. there are allocated addresses at which the application cannot store useful data at the moment. Whereas a garbage collection can likely move objects around in memory to reduce fragmentation, a malloc() implementation will have to request more memory from the operating system if no “hole” is large enough to satisfy an allocation request.

Answer 2

Asking about RAM is wrong, what counts is address space. If your OS is using virtual memory (which is most likely the case), then the mapping of address space to RAM is arbitrary and can change at any time, so address space counts.

The address space assigned to a process is (almost) free and can be huge; how much of it is used is what actually costs. So in your case the OS should have no problem increasing your address space by 100 MB and assigning that address space to your app. However, the OS may have a limit on how much used address space you have, and if your 100 MB exceeds that, then the request can be refused even if there is a free block of 100 MB in the processes address space.

(And you may run in trouble if you allocated 100 MB, allocated 100 MB and shrink the first block to 1KB, allocate 100 MB and shrink the second block to 1KB etc. where your "huge" address space might not be huge enough, depending on processor and OS).