I can't run more than 100 processes

Question

I have a massive number of shell commands being executed with root/admin priveleges through Authorization Services' "AuthorizationExecuteWithPrivileges" call. The issue is that after a while (10-15 seconds, maybe 100 shell commands) the program stops responding with this error in the debugger:

couldn't fork: errno 35

And then while the app is running, I cannot launch any more applications. I researched this issue and apparently it means that there are no more threads available for the system to use. However, I checked using Activity Monitor and my app is only using 4-5 threads.

To fix this problem, I think what I need to do is separate the shell commands into a separate thread (away from the main thread). I have never used threading before, and I'm unsure where to start (no comprehensive examples I could find)

Thanks

Answer 1

As Louis Gerbarg already pointed out, your question has nothing to do with threads. I've edited your title and tags accordingly.

I have a massive number of shell commands being executed with root/admin priveleges through Authorization Services' "AuthorizationExecuteWithPrivileges" call.

Don't do that. That function only exists so you can restore the root:admin ownership and the setuid mode bit to the tool that you want to run as root.

The idea is that you should factor out the code that should run as root into a completely separate program from the part that does not need to run as root, so that the part that needs root can have it (through the setuid bit) and the part that doesn't need root can go without it (through not having setuid).

A code example is in the Authorization Services Programming Guide.

The issue is that after a while (10-15 seconds, maybe 100 shell commands) the program stops responding with this error in the debugger:

couldn't fork: errno 35

Yeah. You can only run a couple hundred processes at a time. This is an OS-enforced limit.

It's a soft limit, which means you can raise it—but only up to the hard limit, which you cannot raise. See the output of limit and limit -h (in zsh; I don't know about other shells).

You need to wait for processes to finish before running more processes.

And then while the app is running, I cannot launch any more applications.

Because you are already running as many processes as you're allowed to. That x-hundred-process limit is per-user, not per-process.

I researched this issue and apparently it means that there are no more threads available for the system to use.

No, it does not.

The errno error codes are used for many things. EAGAIN (35, “resource temporarily unavailable”) may mean no more threads when set by a system call that starts a thread, but it does not mean that when set by another system call or function.

The error message you quoted explicitly says that it was set by fork, which is the system call to start a new process, not a new thread. In that context, EAGAIN means “you are already running as many processes as you can”. See the fork manpage.

However, I checked using Activity Monitor and my app is only using 4-5 threads.

See?

To fix this problem, I think what I need to do is separate the shell commands into a separate thread (away from the main thread).

Starting one process per thread will only help you run out of processes much faster.

I have never used threading before …

It sounds like you still haven't, since the function you're referring to starts a process, not a thread.

Answer 2

This is not about threads (at least not threads in your application). This is about system resources. Each of those forked processes is consuming at least 1 kernel thread (maybe more), some vnodes, and a number of other things. Eventually the system will not allow you to spawn more processes.

The first limits you hit are administrative limits. The system can support more, but it may causes degraded performance and other issues. You can usually raise these through various mecahanisms, like sysctls. In general doing that is a bad idea unless you have a particular (special) work load that you know will benefit from specific tweaks.

Chances are raising those limits will not fix your issues. While adjusting those limits may make you run a little longer, in order to actually fix it you need to figure out why the resources are not being returned to the system. Based on what you described above I would guess that your forked processes are never exiting.