How to determine if Varnish/Pressflow is caching content in a useful way?

Question

I recently converted one of my D6 sites to Varnish/Pressflow and saw a very modest improvement in average load speed (from ~550ms before to ~450ms now). But, I also noticed that the initial load time for the first load in a round of tests were consistently much higher --- around 1000ms. They quickly fall down to ~450ms during subsequent reloads during the same test.

This makes me think that I've configured something incorrectly, and Varnish isn't serving from the cache until after the (anonymous) user has already visited the page in question. Perhaps the session IDs are causing each session to be cached independently from eachother (though I understand that shouldn't happen for anonymous users under Pressflow).

The X-Varnish headers appear in both the initial page load, and subsequent loads, so I know the content is at least getting passed through Varnish.

Is there a way to get details about part of the Varnish cache is serving a particular page (e.g. when the cached page was created)? I've tried looking at the logs generated by varnishlog, but apart from various apearances of "X-Drupal-Cache: HIT" I'm not sure what to look for.