NServicebus for Client/Server Communication?

Question

I am starting out on a new project and have fallen in love with the concept of a Service Bus to connect your services on the backend. Since I was mostly doing Web development so far I was simply connecting the Web tier to the services over the ESB.

Now I am starting out on a Desktop Application that requires Server/Client connectivity, and an ESB seems like a good idea since it forces an Async Model by default and also allows some flexibility in distributing load. Also Pub/Sub makes a lot of sense in many cases..

You read a lot on the internet on ESB, Ayende also did his Alexandria App that used an ESB in a Request/Reply scenario. Then others say doing Request/Response over an ESB is a bad thing..

What are the big problems I am likely to see with a ESB backing my Server/Client communication?