The behaviour of ImageLoader in Volley seems to be tuned for "pages" of thumbnails and not so much a much longer or contiguous stream. To that end, responses from the ImageLoader are actually delayed / batched up before being delivered. So images that are ready are not being delivered as soon as they possible can be to achieve a sort of page load aesthetic, aluded to briefly in the Google I/O talk.
http://www.youtube.com/watch?v=yhv8l9F44qo
You can remove this behaviour with the following
mImageLoader = new ImageLoader(mQueue, imageCache);
mImageLoader.setBatchedResponseDelay(0);
I'm not sure why this causes any problems at the end of the day though, because it's only meant to delay by 100ms, and I don't see it extending that delay based on new content arriving, but will continue to investigate and report back on anything else I find. But this does fix the responsiveness on older devices and newer devices are of course still buttery.