Suggestions for Better Caching


#1

A frequent question our customers encounter is how to optimize their cache and how they can configure their settings to increase their hit ratio. Below are some suggestions and tips for managing your content through Fastly.

Check Headers

A do-not-cache directive is often enforced through the use of HTTP headers. Some headers that will cause your requests to NOT to be cached on Fastly include:

1.Cache-Control: :private
2.Set-Cookie - pass without caching

However, setting Cache-Control:

                  :max-age = 0 
                  :s-maxage = 0

is NOT like a ‘pass’ in the traditional sense. Here’s why: when a object has a max-age =0, the object is cached on Fastly (which might seem counterintuitive) but it must be IMMEDIATELY validated with the origin. So technically, assets with these headers set to zero are cacheable, but they cannot be served without revalidation. If you want your requests to always pass the cache and get served from your backend, do not set the headers max-age = 0 or s-maxage=0 (these also do not preclude request collapsing) and instead set it to Cache-Control: private.

Expires: 0 - as the lowest-priority cache header, the value set here will generally be honored if it is the only one present. Be aware that even with an Expires in the past, content can still be used to satisfy multiple outstanding requests at the same time.

NOTE: Headers follow a hierarchy of precedence, from highest to lowest: Surrogate-Control, Cache-Control, Expires

In other words, only the s-maxage, max-age and private Cache-Control headers will influence Fastly’s caching. All other Cache-Control headers will not, but will be passed through to the browser.

So by default, Fastly will respect these Cache-Control headers if present. However, there is another, more proxy-specific header that can be used: Surrogate-Control. Surrogate-Control headers work in much the same way as Cache-Control headers, but are used to give directions specifically to reverse proxy caches like Fastly. If you use Surrogate-Control headers in conjunction with Cache-Control headers from origin, you can dictate a [different caching behavior] (https://docs.fastly.com/guides/caching/is-it-possible-to-tell-fastly-to-cache-for-a-different-amount-of-time-than-for-end-users) for Fastly than you do for the client.

*Caveat: one exception to Surrogate-Control always overriding the Cache-Control cache policy is in the case where Cache-Control: private & Surrogate-Control: max-age=(whatever) are present in conjunction. The VCL directive dictates that we will reach Cache-Control: private and return: PASS before getting to the logic about Surrogate-Control.

Note: While it’s difficult to get S3 to set Surrogate-Control, you can set X-AMZ-Meta-Surrogate-Control instead of Surrogate-Control on origin and Fastly will honor that.

Time to Live (TTL)

If your origin isn’t sending Expires or Cache-Control headers, the next thing to check would be to make sure your default TTL isn’t set to zero. This can be handled in the UI under Settings–>Default Settings. Increasing your expiry times on static/unchanging assets will increase the likelihood that the objects will be served from the cache.

Request Settings/ Cache Settings

Check to see if any of your request (under Settings–> Request Settings) or cache rules (Settings–> Cache Settings) have an action of Pass and how that affects segments of your traffic.

Enable Shielding

A shield PoP serves as the “source of truth” for the rest of the network PoPs, so that when any other datacenter has a cache-miss, it gets deflected to your shield. All edge PoPs will attempt to retrieve the object from the shield PoP first before trying your backend, thus optimizing requests to your origin.

Purge Discriminately

Purging all content that hasn’t changed from the caches too often will result in an increased load on your backend. Purging by URL might be a better option but in the situation where it’s difficult to keep track of all the URLs where an object might appear, consider using Surrogate keys to selectively purge all assets containing a specific tag header you set.

Setting up Serving Stale

Setting up stale-if-error and stale-while-revalidate will allow Fastly to serve stale content in the event that your backend is unreachable/down or your website is experiencing delays from re-fetching objects from the origin that have expired in the cache.

beresp.stale_if_error controls the maximum allowed age for delivering stale for an object
bresp.stale-while-revalidate will serve a stale response while checking the origin for a new version of the object. This way, the user does not have to wait for us to fetch the object from origin and gets a reply immediately.

The following updated document on configuring stale through custom VCL provides a more thorough explanation as well as custom code.

Vary Header

The Vary header is the industry standard way of telling any HTTP cache which parts of the request header, other than the path and the Host header, to take into account when trying to find the right object. You send it back in your origin server’s responses, indicating that for each unique value of the header, the cache should hold a different representation. However, varying on something that is notoriously ambiguous without normalizing the request headers first can have undesired effects when serving requests back to the client, as well as decrease your cache hit-ratio. For further information on the vary header, please reference our document here.

Miscellaneous

  • We do not cache POST, PUT, or DELETE requests by default. GET requests are cached.
  • Fastly will honor Cache-Control: no cache, no store headers if you create a Cache Setting (Settings–> Cache Settings) with an action of PASS. Add a condition to the new rule to execute if that header is present (ie, beresp.http.Cache-Control ~ “(no cache | no store)” ).

Pointers for testing

Running the following command in your terminal will show relevant headers and pass instances you have set:

curl -svo /dev/null <your URL here>

Note: We strongly recommend using curl -svo /dev/null over curl -i as the latter is a HEAD request rather than a GET (and thus can misrepresent whether assets are caching or not).

Further links



WordPress Installation
Cache images and CSS files in a MVC.Net project