monitoring unix resources from inside a process

Question

I've had a bunch of EAGAIN's from trying to fork() or spawning threads, which lead me to believe that I'm leaking resources somewhere.

Is it possible, in POSIX, to get the following from inside the process itself:

• number of active pthreads
• number of active child processes
• number of active pipes
• number of active sockets (or maybe this and pipes would be counted as file descriptors?)

Or do these have to be counted manually? There's already counters for them, but I think one of them are leaking.

Answer 1

My guess it that you are getting EAGAIN because you are hitting resource limits. There is a system limit to how many threads/process that can be run. You can view it with

cat /proc/sys/kernel/threads-max

and change it with

echo 100000 > /proc/sys/kernel/threads-max

respectively. More likely you are hitting user limits on processes. You should read the man pages and do a search on getrlimit and ulimit. There is plenty of information on those right here on SO.

Assuming you are spawning all these processes from one parent it should be easy to keep track of them. You increment a counter in the parent before you fork(). If children are spawning children then it becomes more complicated and you'll need use shared memory or some other IPC mechanism. In any case you can determine the status of your children using the wait() and waitpid() system calls and decrement your process counters based on that. The status data returned by wait will tell you the terminating disposition of each child. Again, there are quite a few questions on SO that go into considerable detail on this if you run into trouble.

In answer to your primary question, there are ways to query the system (usually via /proc) for some of the resources you are concerned with. This almost always the wrong way to go if you are concerned about your own resources. It is well within your capability to keep track of them yourself - and far more efficient.

Good luck. Hope this points you in the right directions.