In order for clients to establish a P2P ICE session, they have to rendezvous through a reliable signaling server. The signaling server exists for clients to exchange address candidates in order for ICE negotiation to actually begin. This is typically a SIP server. Although ICE allows for almost anything to be used for exchanging addresses.
When the INVITE is sent out, it gets sent to a server that knows how to forward that message to the other client. When the remote client gets the INVITE, it responds back with a series of SIP messages - one of which is the "180 RINGING" or "200 OK" message that contains the address candidates from the client that received the INVITE.