Compute service caching

How does caching work for a compute service?

When a request arrives at the service is the cache checked before the edge code is run? And if there’s a hit then the edge code doesn’t run?

What about fetch requests to an origin server from edge code - is there an automatic cache check here before the request is made of the origin server?

Or is it necessary to manage a cache explicitly in your edge code if you want to avoid making an origin request? If this is so then alternatively one could chain the edge service with a vcl intermediate service to manage the caching, rather than dealing with it in your code?

1 Like

Hey @PaulRudin , good questions…

When a request arrives at the service is the cache checked before the edge code is run? And if there’s a hit then the edge code doesn’t run?

Not really. For a compute service, when a request arrives, the compute code will run without checking a cache.

What about fetch requests to an origin server from edge code - is there an automatic cache check here before the request is made of the origin server?

Yes, there’s some automatic caching at this layer. It’s a pass-through cache that will cache origin responses unless the origin request is set as a Pass, and only for if HTTP status of the origin fetch is one of 200, 203, 300, 301, 302, 404, or 410. For cached responses, there’s a default TTL of 2 minutes, unless it’s set via response headers from the origin. There’s a fuller explanation at Cache freshness and TTLs | Fastly Developer Hub.

Or is it necessary to manage a cache explicitly in your edge code if you want to avoid making an origin request? If this is so then alternatively one could chain the edge service with a vcl intermediate service to manage the caching, rather than dealing with it in your code?

This is an option too. Generally I think about it depending on what you’re trying to achieve. For example if you’re assembling cacheable pages at the edge (personalization, Graphql, etc.), then the request flow might go like
user -> vcl service -> compute service -> origin(s)

Conversely, if you’re using compute@edge as a router, then you might want a service chain like
user -> compute -> vcl service(s)

For that first use case, building cacheable objects in compute, we’re now exposing cache controls directly in C@E, so there’s less reason to put Compute behind a VCL service. If you want to check out those interfaces, here is the pertinent section of the Rust SDK.

3 Likes

At the moment we’re using vcl → compute service → origin, because only some of the incoming requests need to go to the compute service so, the vcl can make that choice.

The thing that I don’t quite understand at the moment: The edge code makes 2 different origin requests. As far as I can tell, one of these requests is being cached, but the other is not. Is there any reason why this might be the case?

After a bit of a break I’m back at looking at this again.

There’s still a thing I don’t understand - I make two origin requests from my compute code, one of these is served from cache when within the TTL, but the other always makes a backend request. I need to ensure that both origin requests use the cache.

What am I missing?

I’ve raised a support ticket about this, but in the meantime I’ve done a bit of experimentation, and it seems that the origin request from my code are not cached if the response “cache-control” header says not to cache. Since we send a “surrogate-control” header I thought the cache-control header would be irrelevant here (and that seems to be the case for a VCL service).

I don’t want to have to change the responses from the origin if it can be avoided. I suppose I can use CacheOverride setting when making requests from my code.

I’ve tried using CacheOverride in the code to get past this issue, but even then I still see an origin request arriving. CacheOverride() | @fastly/js-compute