Counting open files per process

Question 1

Have a look at the /proc/ file system:

ls /proc/$pid/fd/ | wc -l

To do this for all processes, use this:

cd /proc
for pid in [0-9]*
do
    echo "PID = $pid with $(ls /proc/$pid/fd/ | wc -l) file descriptors"
done

As a one-liner (filter by appending | grep -v "0 FDs"):

for pid in /proc/[0-9]*; do printf "PID %6d has %4d FDs\n" $(basename $pid) $(ls $pid/fd | wc -l); done

As a one-liner including the command name, sorted by file descriptor count in descending order (limit the results by appending | head -10):

for pid in /proc/[0-9]*; do p=$(basename $pid); printf "%4d FDs for PID %6d; command=%s\n" $(ls $pid/fd | wc -l) $p "$(ps -p $p -o comm=)"; done | sort -nr

Credit to @Boban for this addendum:

You can pipe the output of the script above into the following script to see the ten processes (and their names) which have the most file descriptors open:

  ...
done | sort -rn -k5 | head | while read -r _ _ pid _ fdcount _
do
  command=$(ps -o cmd -p "$pid" -hc)
  printf "pid = %5d with %4d fds: %s\n" "$pid" "$fdcount" "$command"
done

Here's another approach to list the top-ten processes with the most open fds, probably less readable, so I don't put it in front:

find /proc -maxdepth 1 -type d -name '[0-9]*' \
     -exec bash -c "ls {}/fd/ | wc -l | tr '\n' ' '" \; \
     -printf "fds (PID = %P), command: " \
     -exec bash -c "tr '\0' ' ' < {}/cmdline" \; \
     -exec echo \; | sort -rn | head

Question 2

Try this:

ps aux | sed 1d | awk '{print "fd_count=$(lsof -p " $2 " | wc -l) && echo " $2 " $fd_count"}' | xargs -I {} bash -c {}

Question 3

I used this to find top filehandler-consuming processes for a given user (username) where dont have lsof or root access:

for pid in `ps -o pid -u username` ; do echo "$(ls /proc/$pid/fd/ 2>/dev/null | wc -l ) for PID: $pid" ; done  | sort -n | tail

Question 4

This works for me:

ps -opid= -ax | xargs -L 1 -I{} -- sudo bash -c 'echo -n "{} ";lsof -p {} 2>/dev/null | wc -l' | sort -n -k2

It prints numopenfiles per pid sorted by numopenfiles.

It will ask for sudo password once.

Note that the sum of the above numbers might be bigger than the total number of open files from all processes.
As I read here: forked processes can share file handles

Question 5

How can I take the open files count for each process in Linux?

procpath query -f stat,fd

if you're running it from root (e.g. prefixing the command with sudo -E env PATH=$PATH), otherwise it'll only return file descriptor counts per process whose /proc/{pid}/fd you may list. This will give you a big JSON document/tree whose nodes look something like:

{
  "fd": {
    "anon": 3,
    "blk": 0,
    "chr": 1,
    "dir": 0,
    "fifo": 0,
    "lnk": 0,
    "reg": 0,
    "sock": 3
  },
  "stat": {
    "pid": 25649,
    "ppid": 25626,
    ...
  },
  ...
}

The content of fd dictionary is counts per file descriptor type. The most interesting ones are probably these (see procfile.Fd description or man fstat for more details):

reg – count of open (regular) files
sock – count of open sockets

I'm the author of Procpath, which is a tool that provides a nicer interface to procfs for process analysis. You can record a process tree's procfs stats (in a SQLite database) and plot any of them later. For instance this is how my Firefox's process tree (root PID 2468) looks like with regards to open file descriptor count (sum of all types):

procpath --logging-level ERROR record -f stat,fd -i 1 -d ff_fd.sqlite \ 
  '$..children[?(@.stat.pid == 2468)]'
# Ctrl+C
procpath plot -q fd -d ff_fd.sqlite -f ff_df.svg

If I'm interested in only a particular type of open file descriptors (say, sockets) I can plot it like this:

procpath plot --custom-value-expr fd_sock -d ff_fd.sqlite -f ff_df.svg