Varnish / Squid content caching with cookies

Question

I'm an absolute newbie to Varnish & caching in general, so this question might be trivial to some !

I'm building a web app (in rails), and I was looking at some options to cache content on production servers. I've so far looked at Varnish, and am slightly concerned about the non-caching of with-cookie requests. I understand why such a behaviour is intended, but there is one use-case that could really benefit if there was some way to cache content over cookies.

In my particular case, a lot of urls have data that is same irrespective of the logged-in user. For example, tripadvisor would possibly be returning every logged-in user the same data (at least majority of the data) for a hotel listing. So essentially, although cookies would accompany the request / response, about 90% of the page would be common for each user.

In such scenarios, is it possible to use Varnish to cache such urls / pages (maybe the common part only). And if yes, then how.

Thanks a ton !

Answer 1

If I understood your use case correctly. You would like to cache a subset of pages even in presence of cookies because those pages will be the same for all users.

The best way to do this would be to identify the pages you want to cache all the time, let's say /foo.html and /bar.html. For those pages you can remove the Cookie request header in VCL so Varnish can cache the page.

You can use the following VCL code as an example:

sub vcl_fetch {
    [...]
    if(req.url ~ "/foo.html" || req.url ~ "/bar.html") {
        unset req.http.Cookie
    }

    [...]
    if (req.http.Authorization || req.http.Cookie) {
        /* Not cacheable by default */
        return (pass);
    }
}

If you have much more pages you can use a regular expression in the if condition to match you page names.

If you want to only cache parts of a page then you can use ESI since it is supported (the support is limited but sufficient) by Varnish but this requires you to rewrite part of your application and IMHO using ESI is really a PITA in the long run (harder maintenance).

Answer 2

The default Varnish vcl will 'pass' requests with cookies but using the default vcl is only good for testing that your Varnish server is up and running. Almost never does the default vcl work as desired without some modification.

You do not necessarily need to remove the cookie. It is easier to change how you handle requests with cookies from within vcl_recv(). vcl_hash() does not hash over a cookie by default.

In addition vcl_fetch() no longer exists in Varnish 4.