Cache-Control max-age and Age headers


#1

Hi,

I have my origin server responding to requests with these cache headers:

'Surrogate-Control' : 'max-age=86400, stale-while-revalidate=1800, stale-if-error=86400'
'Cache-Control' : 'max-age=30'

I want Fastly to cache the content for a day, while the browser will only cache the content for 30 seconds. This works for the first thirty seconds, but afterwards the browser will request the content every time (receiving a 304, since the content has actually not changed). I’d like for the browser to receive that 304 and then load the content from the cache for the next 30 seconds. Then receive another 304 and load from cache and so on (until the content actually has changed or the content goes stale in Fastly).

First response looks like this (some headers omitted for brevity):

HTTP/1.1 200 OK
Cache-Control: max-age=30
Content-Type: text/javascript; charset=utf-8
Etag: W/"140fa6-gG4hMYN4QxLRultLws1I3EqDVVE"
Via: 1.1 vegur
Content-Length: 1314726
Accept-Ranges: bytes
Date: Mon, 17 Jul 2017 19:15:02 GMT
Via: 1.1 varnish
Age: 0

The second request made less than 30 seconds later results in the content being loaded from cache.

The third request made more than 30 seconds later results in a 304:

HTTP/1.1 304 Not Modified
Date: Mon, 17 Jul 2017 19:20:23 GMT
Via: 1.1 varnish
Cache-Control: max-age=30
ETag: W/"140fa6-gG4hMYN4QxLRultLws1I3EqDVVE"
Age: 321

The fourth request made right afterwards still results in a 304 instead of content being read from browser cache:

HTTP/1.1 304 Not Modified
Date: Mon, 17 Jul 2017 19:20:33 GMT
Via: 1.1 varnish
Cache-Control: max-age=30
ETag: W/"140fa6-gG4hMYN4QxLRultLws1I3EqDVVE"
Age: 331

I’m assuming this is because the Age header is more than 30. Do I need to configure Fastly to send a different Age header so the Cache-Control: max-age=30 header is respected for the browser’s cache?


#2

Hi @rayd,

I’d like to summarize your question below to confirm that I’m following along.

I'd like for the browser to receive that 304 and then load the content from the cache for the next 30 seconds. Then receive another 304 and load from cache and so on (until the content actually has changed or the content goes stale in Fastly).

Once the first response is received (200 OK), and stored in local cache, you would prefer that for 30 seconds (until the TTL expires), the response is to be served from browser cache rather than from Fastly cache. Once the TTL expires, the browser will issue a new request to the server (which would be Fastly in this case), checking to see if the content is unchanged, and if so, serve back a 304. At that point, the browser would then serve the object from local cache for up to the 30s once more. This cycle would continue until the content has changed at the server.

If I got that right, it’s my understanding this not possible due to the Etag being tied to the original response. Because Etag is a cache validator, the browser is required to check if the object has been modified or not on every request, which nullifies its ability to serve the resource from disk. This appears to be one of the unfortunate side-affects of using Etags.

Let me know if this helps clear things up at all.

Taylor


#3

Hi @rayd

I’ve done some deeper digging into this, and it turns out that this possible at the HTTP level, but there does appear to be a caveat associated with it.

Here’s a JS Bin that shows this working in action. While watching these responses, you’ll notice every 30s we receive a new revalidation request at the server, and all requests in-between are getting served from local cache.

The caveat I’m running into though is that when I test this by manually requesting the page, it turns out revalidation is always required. This may be attributed to different revalidation rules for top level navigation requests, as compared to “sub” HTTP requests, but I can’t say for sure.

If the requests that you’re looking to serve with this logic aren’t pages directly being requested by end users, then this should work for your case. I’ve been able to configure this by doing exactly as you had suggested initially-- in the event of a 304 response, pass an Age header of 0 back to the browser. You can do this on Fastly by adding the following into your vcl_deliver subroutine.

if (resp.status == 304) {
  set resp.http.Age = "0";
}

Taylor


#4

Hi Taylor,

Thanks for all the investigation! This definitely sounds promising. I think I’ve noticed the same thing you pointed out – that the browser chooses to load things from private (browser) cache or not depending on how the resource is requested (i.e. if the resource is requested directly vs being a resource loaded from a page). I know I came across a document discussing Chrome’s behavior with regard to this, but I can’t seem to find it now.

Anyways, I think I’ll give this a try. I’m curious if changing the age in this way will mess with Fastly’s ability to evict the item from the cache once it becomes stale, but this may just be something I need to play around with. Thanks again for all the info!

-Ray