Varnish: Clear Cache after POST data

Question

I have a Plone website that sits behind Varnish. All is working OK except for one thing.

This is a dynamic site and hence there will be new content from time to time. The scenario is as follows:

I have a page that displays a list of items. This page has been cached. So I add another item via some form, and return to the same page, but the new item is not shown. This is because the page displayed is from the cache and is still within its TTL.

How can I ensure that upon submitting of a new item, that page is purged from the cache, and a new page from the backend server with the new item will be displayed?

My simple VCL is as shown:

backend default { 
    .host = "127.0.0.1"; 
    .port = "8080"; 
} 

sub vcl_recv { 
    if (req.request != "GET" && req.request != "HEAD") {
        # We only deal with GET and HEAD by default
        return (pass);
    }

    # remove unnecessary cookies 
    if (req.http.cookie ~ "wc.cookiecredentials|Path|Domain") { 
        # found wc.cookiecredentials in request, passing to backend server 
        return (lookup); 
    } else { 
        unset req.http.cookie; 
    } 
} 

sub vcl_fetch { 
    #unset beresp.http.Set-Cookie; 
    set beresp.ttl = 12h; 
    return(deliver); 
} 

# Routine used to determine the cache key if storing/retrieving a cached page. 
sub vcl_hash { 
    # Do NOT use this unless you want to store per-user caches. 
    if (req.http.Cookie) { 
        set req.hash += req.http.Cookie; 
    } 
} 

sub vcl_deliver { 
    # send some handy statistics back, useful for checking cache 
    if (obj.hits > 0) { 
        set resp.http.X-Cache-Action = "HIT"; 
        set resp.http.X-Cache-Hits = obj.hits; 
    } else { 
        set resp.http.X-Cache-Action = "MISS"; 
    } 
} 

Or to put it plainly, how can I purge or clear the entire cache for the domain whenever I receive a POST request?

Answer 1

To accomplish this you need to customize your Varnish VCL to handle PURGE requests and your Plone CMS so that it issues a purge request to Varnish when content is changed.

Plone Developer Documentation has pretty good and thorough documentation on using Varnish with Plone. You can adapt it to your specific needs.

The example in the documentation explains how to create a custom ACL and regexp purge handling in VCL, and then how to use that in Plone to purge the entire cache. I have copied the VCL and Plone view from the examples here, just in case they get removed from the Plone site some time in the future:

acl purge {
        "localhost";
        # XXX: Add your local computer public IP here if you
        # want to test the code against the production server
        # from the development instance
}

...

sub vcl_recv {

        ...

        # Allow PURGE requests clearing everything
        if (req.request == "PURGE") {
                if (!client.ip ~ purge) {
                        error 405 "Not allowed.";
                }
                # Purge for the current host using reg-ex from X-Purge-Regex header
                purge("req.http.host == " req.http.host " && req.url ~ " req.http.X-Purge-Regex);
                error 200 "Purged.";
        }
}

Then for Plone you create a custom view for issuing PURGE requests to Varnish:

import requests

from Products.CMFCore.interfaces import ISiteRoot
from five import grok

from requests.models import Request

class Purge(grok.CodeView):
    """
    Purge upstream cache from all entries.

    This is ideal to hook up for admins e.g. through portal_actions menu.

    You can access it as admin::

        http://site.com/@@purge

    """

    grok.context(ISiteRoot)

    # Onlyl site admins can use this
    grok.require("cmf.ManagePortal")

    def render(self):
        """
        Call the parent cache using Requets Python library and issue PURGE command for all URLs.

        Pipe through the response as is.
        """

        # This is the root URL which will be purged
        # - you might want to have different value here if
        # your site has different URLs for manage and themed versions
        site_url = self.context.portal_url() + "/"

        headers = {
                   # Match all pages
                   "X-Purge-Regex" : ".*"
        }

        resp = requests.request("PURGE", site_url + "*", headers=headers)

        self.request.response["Content-type"] = "text/plain"
        text = []

        text.append("HTTP " + str(resp.status_code))

        # Dump response headers as is to the Plone user,
        # so he/she can diagnose the problem
        for key, value in resp.headers.items():
            text.append(str(key) + ": " + str(value))

        # Add payload message from the server (if any)

        if hasattr(resp, "body"):
                text.append(str(resp.body))

As mentioned, this simply purges the entire cache on demand. I am no expert in Plone, so I'm unable to give you a detailed answer on how to adapt this to purging just specific content. Basically you need to determine which pages you need to purge in specific cases, and then adapt the example above to automatically issue a PURGE request to Varnish when POST requests are handled in Plone.

Handling purging in VCL alone (i.e. detecting POST calls and purging content based on those) is quite complex. I believe it'll be much more efficient handling the logic and purging in Plone.

Update:

If you wish to purge the entire cache on every POST, this can be accomplished as follows.

sub vcl_recv {
    if ( req.request == "POST") {
        ban("req.http.host == " + req.http.Host);
        return(pass);
    }
}

Now every POST request causes ALL pages that have been cached under the same hostname to be purged from the cache. I recommend looking into the earlier solution in a long run though. It is much more efficient purging only the pages that actually need to be purged when a POST occurs.