How to write varnish script to do something very particular

Question

Varnish scripting seems rather robust for the vcl but I can't yet figure out how to make it do what I need. I run various sites from the same code base and I want a unified varnish cache for most of the directories so

x.mysite.org/theme/something.gif and y.mysite.org/theme/something.gif should not store two copies of the same gif in varnish cache

However

x.mysite.org/file.php/1 and y.mysite.org/file.php/1 should have separate caches based on the url.

Also mysite.org is a whole other site that has its own cache.

My current direction is as follows

sub vcl_fetch {
  if (req.url ~ ".*\.org/file\.php") {
    # do normal site specific caching
  } elseif (req.url ~ "^+?\.mysite.org") {
    # cache all found material in a base directory so everyone knows where to look
    set req.url = regsub(req.url, "(.*\.org)(.*)", "base.mysite.org\2");
  } else {
    # do normal site specific caching for base site
  }
}

sub vcl_recv {
  # do I need to do something here to look in base.mysite.org
}

I can make base.mysite.org a real apache served site if necessary so the requests can fall through if no cache.

Am I on the write path, any help.

Answer 1

You should normalize req.http.host instead of req.url, so

sub vcl_fetch {
  # if it starts with /theme or /static, or contains .gif,.png etc, 
  #   then consider the host to the normalized/common host
  if (req.url ~ "^/(theme|static)" || req.url ~ "\.(gif|png)" ) {
    set req.http.host = "base.mysite.org";
    return (lookup);
  }
  # else, do non shared stuff here
}

Answer 2

By default Varnish will use the hostname + the URL to get a hash of a cache object. It means that even if x.mysite.org/theme/something.gif and y.mysite.org/theme/something.gif point to the exact same content Varnish will see them as two different cache objects. The only way for you to make them point to the same cache object is to normalize the hostname as Ivy explained in his post.

'Hope that helps.