How can two services discover each other without static addresses?

Question

Supposed I have two services that need to share and / or exchange data. Both instances are separate from each other, and they shall not know anything about where the other part is located.

Now in order for them to be able to share and / or exchange data, they need to connect to each other.

How do they find each other without the need to configure the IP addresses explicitly? In other words: How could they detect each other automatically?

Basically, I have two ideas:

• Pull: You need to have a central service where they register. Then you can ask that service for the address of a service, and that service then returns those data. While this works, it has the drawback that it only shifts the problem to the next level: What if I have multiple instances of that service, and I don't want them to know each other in advance?

• Push: Each service broadcasts its own address, so that other services get it to know. Each service repeats this from time to time. Drawback: This does hardly work in the internet.

Any idea of how I could solve this in an intelligent way?

PS: If you want to say so, I'm looking for a way to handle dynamic IPs without the need for a central DNS server.

Answer 1

The usual way is to have some fault-tolerant server where services register and can then look for other services - Curator framework implements that over zookeeper.

If you want autodiscovery then you should probably implement some sort of gossip protocol so that the servers would know which other servers are out there in a reliable way. You should keep in mind that getting gossip protocols right is tricky (e.g. some of past Amazon cloud failures where due to problems in their implementation)

Answer 2

"broadcast packets are not forwarded everywhere on a network, but only to devices within a broadcast domain."

If your devices are on different broadcast domains then broadcasting is not going to work.

You are probably going to have to implement your own central service, unless you can use one of the free dynamic dns servers, for example: Free