Log/measure vary keys

General
1

Is it possible to log the “vary key” associated with a response? I ask because we would like to measure the cardinality of our vary response header.

For context we are using VCL.

2

Hi Oliver :wave: it’s been a while!

So the short answer is “No” but there’s a longer one that’s kind of a Yes.

The vary key or secondary cache key is constructed using data from the request that triggered the response that has the vary header on it. So if you want to you can log those pieces of data. Two things make this suck, unfortunately:

  • No loops in VCL so you can’t enumerate all the entries in the Vary header
  • No dynamic lookups on maps so you can’t address the request headers unless you know the name of the header at compile time.

That limits the flexibility of the solution, but if you have a good understanding of what the range of the possible is, you could do this:

sub process_vary_key {
   if (req.http.vary ~ "^([\w-]+)([;, ]*(.*))?$") {
      set req.http.vary = re.group.3;
      if (re.group.1 == "accept-encoding") {
         set beresp.http.vary-key = set beresp.http.vary-key + " " + bereq.http.accept-encoding;
      } elseif (re.group.1 == "accept-language") {
         set beresp.http.vary-key = set beresp.http.vary-key + " " + bereq.http.accept-language;
      } elseif (re.group.1 == "custom-header") {
         set beresp.http.vary-key = set beresp.http.vary-key + " " + bereq.http.custom-header;
      }
   }
}

sub vcl_fetch {
   if (besresp.http.vary) {
      set req.http.vary = beresp.http.vary;
      set beresp.http.vary-key = "";
      call process_vary_key;
      call process_vary_key;
      call process_vary_key;
      call process_vary_key;
      call process_vary_key;
   }
}

A few points about how this works:

  • Copying the vary header into req.http.vary allows it to be trimmed by one token each time you call process_vary_key without affecting the actual vary header that the cache will use (which is beresp.http.vary)
  • The req object in vcl_fetch is a transient copy of the original, so any changes you make to it here will not persist into vcl_deliver so it’s a good scratchspace.
  • Putting the eventual value into beresp ensures it will be included in the object when it’s saved into the cache.

You’ll also want to consider removing that vary-key response header in vcl_deliver if you don’t want it served to end users (or even better have it depend on the Fastly-Debug request header so you can enable it for debugging).

How’s that?

2 Likes
3

Thanks Andrew! Really appreciate the detailed response. We’ll give that a go.