It depends on what transport you're using.
In the case of a brokered message queue, like Azure Service Bus, as long as that service is available, the fact the machine that will eventually retrieve the messages is offline is irrelevant, as that machine is simply asking the external queuing service for messages. The same goes for a transport like SQL Server.
In the case of a transport like MSMQ, which is a store a forward style queue, the messages will remain in a local outgoing queue until the remote machine becomes available.