Connection draining directors

Network Services
1

Is it possible to drain connections from an old backend, while still serving them?

Something like:

  1. existing traffic starts at backend v0.0.1
  2. new backend, v0.0.2 starts handling traffic (director weights equally)
  3. weight the traffic to favor the v0.0.2 backend, but still respond to v0.0.1 clients who already have an existing cookie (or something)
  4. once all traffic is on the new backend (possibly only detected by that system), director will only have v0.0.2 as the backend.

thought about having the draining backend respond with 500 errors or similar but I do want fastly to keep serving some responses (say, if they’re in a minor or major version release and need to finish whatever flow they’re on, even if it’s stateless the UI & API change might be large)

maybe I’m overthinking it.

2

Hiya,

Interesting. From what you’re describing it sounds like:

  • The new backend is running a different version of your backend software, it’s not just a new machine instance
  • Your director is using a client-based strategy, ie repeated requests from the same end user should be directed to the same backend.
  • You want users to gradually migrate to the new backend, but only when they complete their transactions.

So assuming I got that right, a few ideas occur to me:

If you change the weight of the backends in your director to incrementally shift traffic, that will of course cause users to move from one backend to the other, and we can’t know if they are mid-transaction or not. So a better option here might be to use some explicit backend assignment in your edge code rather than using a director. You could do something like this:

  1. When Fastly receives the request, if a cookie exists specifying a backend to use, use that, otherwise assign a backend based on current desired traffic split (which you could store in dynamic configuration data)
  2. Backend returns a response, which could include a set-cookie header to delete the backend assignment if the backend knew that the user no longer needed to stick to that backend.
  3. When Fastly delivers the response to the client…
    • If the response from the backend contains a set-cookie, pass that on unchanged.
    • Otherwise, if the request originally had a cookie that specified a backend assignment, renew it, restarting its max-age lifetime (TTL).
    • Otherwise, set a cookie that sticks the client to the backend it got the current response from.

Now let’s say you start with a 90:10 split in favour of the old backend. Things look good so you dial it up to 50:50. Users mid-session will continue to stay on their current backend until either the cookie expires (because they didn’t send a request for a long time), or the backend they’re using explicitly breaks the attachment by deleting the cookie. So even if you don’t implement any kind of session-ending logic on your backend, the cookie timeout will gradually see you move up to 50:50. Now things still look good and you set 0:100 in favour of the new backend. Existing users with a cookie are still going to go to the old one until their sessions end or the backend ends their session, but all new sessions will be on the new backend.

This would be a fantastic code examples to publish on our developer hub, so let us know if you choose to do something like that!

Andrew

1 Like
3

re-reading this.

is it possible to add/delete backends without incrementing the currently submitted service number?

by the looks of it, the fastly cli returns a little header and footer when I pull up the full VCL, which makes scripting “should I change the config” a little weirder. I guess there’s terraform for that…

if I can change the dynamic config data to set the % it would be interesting if I could use dynamics to update whole backends in and out.

4

Hi @nothingwobbles

is it possible to add/delete backends without incrementing the currently submitted service number?

No, this isn’t possible currently as the backends are coupled to specific service versions.

With regards to configuring the percentage for your backends (which is the approach that Andrew described), you could use an Edge Dictionary to store the current ratio split value and then once your service is activated with the dictionary in place and your VCL code is consuming the value from the dictionary, you can push a change in traffic ratio to the dictionary and it should be picked up by new incoming requests without having to clone/activate a new service version.

EDIT: I stand corrected, Fastly does provide ‘versionless’ server pools Dynamic server pools | Fastly Developer Hub