I managed to fix this, and am posting it just in case for any future viewers.
I had upgraded from AppFabric 1.0 (1) to AppFabric 1.1 (3). Apparently, the cache-hosts are locked to accept only from version 1, and you need to manually upgrade them to accept connections from version 3 (AppFabric 1.1) clients.
This can be done by following the guide at:
http://msdn.microsoft.com/en-us//library/hh343304(v=azure.10).aspx