How can I coordinate port usage across processes?

Question

I need to have several processes coordinated with each other on a machine such that each process chooses a different port for creating a socket to communicate with a different machine. Each process must choose a port number from a range of available ports such that at any point in time only one process is using a given port number.

Is there an existing mechanism to do this coordination or do I need to build my own?

Presently, I am using a file on disk to register the port number and the PID that is using that port. If an entry is found where there is no PID running anymore, the entry is removed as the process is assumed to have exited non-gracefully and thus the entry needs to be reaped.

However, I seem to have some kind of race condition as in rare instances, I end up with two processes using the same port and the second process does not see the file contents showing the port reserved and ends up using it a second time.

Rather than fix this, I'd rather adopt an existing mechanism for this collision avoidance of ports, so I'm asking if such a utility already exists.

Answer 1

When communicating with TCP or UDP, the socket is distinguished by a 4-tuple:

• local address
• local port
• remote address
• remote port

If you leave the local assignment alone, and you are communicating with a connected protocol, the connect call will find an available local port for you. If you are restricted to a particular range of ports, then you'll need to do local address and port binding. However, in that case, since each process is only handling a single connection, I would just assign each process a unique port when it is started, rather than having it try to find an available one.

That is, assuming there is a single process responsible for starting all the other processes, that single process would decide which port should be used by the process it is about the start. This is simpler because the decision is made by a single entity, so it can use a simple algorithm, like:

for (unsigned short p = minPort; p < maxPort; ++p) {
    child_pid = startWorker(p);
    child_pid_map[child_pid] = p;
}

If the child dies, the starter process can consult the child_pid_map to reuse that port immediately.

If using an external agent to assign ports is not possible, the simplest technique is to just attempt a bind call to a random port in the range. If it fails with EADDRINUSE, then incrementally try the next port number in the sequence (wrapping if necessary) until success is achieved, or you have tried all the ports.