Keeping a failed backend out until manual intervention


#1

If one of our backends fails, we’d like to have it stay out of the rotation until a human manually returns it to service, even if its health checks start to pass again. Possible?


#2

I am interesting by this feature but I guess it’s no possible easily as it’s inside VCL.
It would be great as well to have the status in real time of each backend (green or red and since how many times)


#3

Part of the issue here is that an origin’s health is from the perspective of each and every POP.

e.g. internet weather events may make a POP briefly unavailable.

This means that sometimes an origin may appear to be unavailable from Aukland but be fine from Seattle, so how to manage this gets a bit more complicated.


#4

Hello Peter,
Thanks. you are right it can be quite complex.
The thing if one of our backend if never used by Fastly (as maybe down from internet) we don’t have any way to know it.
Same question as well from lhankins : Is there a way to put a backend offline/online easily ? (I guess we need to edit/deploy VCL in all case. no ?
Thanks,
Alex


#5

It would entail a vcl deployment or api calls. One way I’ve advocated for taking an origin server offline is use a healthcheck that always fails.


#6

The thing if one of our backend if never used by Fastly (as maybe down from internet) we don’t have any way to know it.
Same question as well from lhankins : Is there a way to put a backend offline/online easily ? (I guess we need to edit/deploy VCL in all case. no ?
joe root
hr@ Online Assignment help


#7

To take a backend offline, you could alternately just chmod the file you use for your healthcheck to something that will cause the healthcheck to fail, if that’s easier than updating your Fastly config.

ssh <host> chmod 600 <healthcheckfile>


#8

hello Peter,
I agree but you need to connect to server and have access to it. If one operation do it as server level it’s manual operation and can do a mistake.
If with your new products Load Balancing https://www.fastly.com/products/load-balancing/ it’s possible to do it in the interface ?
Thanks,
alex


#9

There’s an API for the dynamic servers. You could remove them from the pool:
https://docs.fastly.com/api/dynamicservers#intro