(Apologies if this is the wrong category, I wasn’t entirely certain.)
I have created a VCL Service that is using the Shield feature to protect our origin server. In certain cases, depending on the headers present in the request, we want to prevent the request to our origin from the shield so we return (error); from vcl_miss when req.backend.is_origin == true if these specific hears are not there.
Within vcl_error we set the obj.status to 404, and return a synthetic {"{"error": "Not found" } "}; response. This works well for our needs currently.
BUT, I’ve been unable to tell how or IF I there is a way to set a separate TTL for these synthetic error responses. obj.ttl is available in vcl_error, but according to the docs ( obj.ttl | Fastly Documentation ),
This variable is writable only in vcl_hit and vcl_error. Doing so in vcl_error does nothing because error objects are not cacheable. Setting obj.ttl in vcl_hit will only be effective if the value is zero (and will trigger the object to be purged); otherwise, it will retain its existing TTL.
(I don’t understand what “error objects are not cacheable” means here as from what I can tell, these errors are cached at the edge using the TTL I set there)
I think this means that these synthetic responses returned from vcl_error will have a TTL set to whatever I set the as the beresp.ttl in vcl_fetch for the initial edge fetch (keeping in mind I’m using shield, so the first request before I might trigger a error in the second vcl_miss will likely go through an edge node’s vcl_fetch).
Sorry if I’ve written this in a confusing or inexact way, I may be making some assumptions or overloading some terms in ways that are incorrect.
The bottom line is my goal here is to have something like this:
edge TTL for hit: 10 minutes
shield TTL for hit: 2 hours
TTL for synthetic error: 1 minute
Is this possible?
1 Like
Hey @shoelessone – Synthetic responses aren’t managed by a TTL because they don’t live in cache. By nature, they’re permanently in your config code, so there’s no way for the object to have a TTL expiration. They’re consistently executing when your logic instructs the service to do so.
I don’t want to get prematurely pedantic about the definition of TTL and “in cache” though.
- Is the intent that the Client receives a
cache-control: max-age=60 header from Fastly for the 404?
- Or, is there conditional state/logic in your service that would cause the synthetic response to execute only occasionally, hence the 60s “TTL” goal?
Thanks for your thoughts @aspires , and genuinely sorry about the lack of precession in some of my word choices, I know it doesn’t make it easier to help when it’s not clear what I’m talking about!
The goal is this one: Or, is there conditional state/logic in your service that would cause the synthetic response to execute only occasionally, hence the 60s “TTL” goal?
There are some requests that depending on headers provided, we ONLY want to serve out of the shield cache - if the shield does not have cache for the request for whatever reason, then we want to return a 404 (which we do by triggering an error in vcl_miss). But, we don’t want to cache this 404 response at the edge for very long, because the shield should have a cached version of these requests. So we’d like to trigger the synthetic 404 from the shield, but not hold onto it very long at the edge so we can retry the request soon.
The thing that was (and is, to some extent!) throwing me off is that while you’re of course right that the synthetic 404s live in my code, I only generate them at the shield layer of the request - but as a result, the edge caches will still hold onto those (in my mind, without me explicitly “telling” the edge a response from fetch is synthetic and special in some way, the edge just sees the response from the shield that happens to be a 404 status - at least this is what I’m currently telling myself).
I think the simple answer to my question (to be fair I was somewhat tired last night when I posted this :)) is that in the shield node, when I generate that synthetic 404 response, I can just attach another response header that will then make it’s way to the edge at which point I can check for that header, e.g. something like
sub vcl_fetch {
...snip snip snip...
# make sure this isn't a request to the origin, i.e. a shield node
if (! req.backend.is_origin ) {
# this header was set when the synthetic response was generated
if (beresp.http.X-Force-404 == "1") {
# so we'll set a shorter TTL so we can try again sooner
set beresp.ttl = 60s;
} else {
# This is a response from the shield node, so just set a flat 10m TTL
set beresp.ttl = 10m;
}
}
...snip snip snip...
}
Another alternative is to have the shield code generate a response with a status code in the 6xx range, then in the edge code use that as a trigger to generate the synthetic 404 response. A 6xx response should not be cached at the shield at all, if I remember correctly.