Resident Set Size (RSS) limit has no effect

Question

The following problem occurs on a machine running Ubuntu 10.04 with the 2.6.32-22-generic kernel: Setting a limit for the Resident Set Size (RSS) of a process does not seem to have any effect. I currently set the limit in Python with the following code:

import resource
# (100, 100) is the (soft, hard) limit. ~100kb.
resource.setrlimit(resource.RLIMIT_RSS, (100, 100))
memory_sink = ['a']*10000000   # this should fail

The list, memory_sink, succeeds every time. When I check RSS usage with top, I can easily get the process to use 1gb of RAM, which means that the limit is not working. Do RSS limits not work with this kernel or distro? If it helps, resource.RLIMIT_NPROC (user process limit) does work.

Answer 1

Form the getrlimit manpage:

RLIMIT_RSS
Specifies the limit (in pages) of  the  process's  resident  set
(the  number of virtual pages resident in RAM).  This limit only
has effect in Linux 2.4.x, x < 30, and there only affects  calls
to madvise(2) specifying MADV_WILLNEED.

It seems this is just not supported on Linux kernel 2.6.

Answer 2

You can accomplish this using cgroups. The long version is on my blog, but the short version (tested on Ubuntu 11.04) is:

• Install the cgroup-bin package.

• Edit /etc/cgconfig.config and create a group with limited memory. For instance, I added:

  group limited {
    memory {
      memory.limit_in_bytes = 50M;
    }
  }
  

• Run

  $ sudo restart cgconfig
  $ sudo chown -R jlebar /sys/fs/cgroup/memory/limited
  $ cgexec -g memory:limited your/program
  

I observed my process with an RSS of 93M when I asked it to use only 50M, but that wasn't a problem for me, since my goal was just to get the program to page.

cgclassify lets you attach restrictions to a running process too. Note for RSS this only applies to memory allocated after the restriction comes into effect.

Answer 3

A related limit - virtual memory or address space (RLIMIT_AS) - does work. This allows limiting the python process and subprocesses memory without external tools.

>>> size = 50*1024*1024 # In bytes
>>> resource.setrlimit(resource.RLIMIT_AS, (size, resource.RLIM_INFINITY))
>>> a = 'a' * size
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError

From the man page:

RLIMIT_AS. The maximum size of the process's virtual memory (address space) in bytes.

Here is a good explanation of the difference between the Resident Set and the VM size - What is RSS and VSZ in Linux memory management.

Answer 4

I created a script to limit memory usage using cgroups and cgroup manager, useable for ad-hoc commands and not needing root privileges. See https://unix.stackexchange.com/questions/134414/how-to-limit-the-total-resources-memory-of-a-process-and-its-children/174894#174894