Flicker-Free A/B Testing with GrowthBook's Edge SDK

The Gallery
1

1. Tell us who you are and what you’ve built!

We’re GrowthBook, an open source A/B testing and feature flagging platform, and we’ve built a public SDK for Fastly Compute. The SDK deploys GrowthBook at the edge, allowing teams to run A/B tests without the performance degradation and other negatives usually associated with front-end experimentation.

2. What problem are you solving?

While experimentation in the browser is easier to set up and allows non-technical team members to implement A/B tests, it has its drawbacks:

  • Pages flicker or flash while experiments are fetched and rendered
  • Networking problems and ad blockers can prevent experiments from running
  • Networking / ad blockers can cause event tracking dropoff, leading to underpowered or biased test results

With our Fastly Edge SDK, we solve these problems by rendering experiments at the edge. Users receive the control or variant HTML directly — without the flash, flicker, and latency. We support both URL redirect tests and visual/no-code tests created with our Visual Editor Chrome extension.

Our edge SDK has a few other benefits:

  • Inject our standard JavaScript SDK with hydrated feature and experiment definitions onto your page, allowing the front-end to pick up where the edge left off
  • Synchronize feature and experiment changes to the SDK via Fastly’s KV store, avoiding network overhead

3. Who are the ideal users for this product?

Product teams, growth teams, and anybody interested in front-end experimentation without incurring the common network costs, visual jank, and measurement gaps.

4. How are you using Fastly within the product?

The GrowthBook Fastly SDK works seamlessly with Fastly’s Compute worker to proxy traffic and render experiments on the edge .

Install our turnkey Fastly SDK onto your Compute worker, set up a few config variables, and optionally connect a Fastly KV store (GrowthBook will push changes to your KV to keep features and experiments in sync without request-time overhead).

Once your worker is set up, point your traffic (DNS records) at the worker, let it proxy your public site and automatically run experimentation on edge.

5. How can people get started right now, and what’s a good “hello world” example?

  1. Sign up for GrowthBook. Bring your own data warehouse and event tracking (or just default to using GA4 and BigQuery to get started quickly)
  2. Create a visual experiment or URL redirect test for your website
  3. Create a Compute worker

Steps 4-7: See our walkthrough here:

  1. Install our SDK
  2. Implement our request handler
  3. Configure the edge app using config vars
  4. Optionally connect Fastly KV to your edge app and to GrowthBook

6. Where can people learn more?

7. Do you have any screenshots, gifs, or videos that should be embedded in the post?

image
image1705×2060 502 KB

Video overview of our Edge SDKs (vendor agnostic)

4 Likes
2

This is pretty cool, thanks for sharing! Out of curiosity, are you able to share any latency improvements or before/after data on moving the logic to the edge?

1 Like
3

Sure, some quick benchmarks while dogfooding a “visual experiment” (DOM mutations) on our own landing page (growthbook.io) are as follows:

With the Fastly SDK proxying the site:

  • Total round trip time: approx ~150ms
  • VARIANT rendered

All visual experimentation delivered in page response, all experiment event tracking done “behind the scenes” on edge.

Running experimentation in the browser:

  • Initial page response: ~140ms
  • CONTROL rendered
  • Fetch experiment definitions in via SDK (CDNed): ~380ms
  • Render DOM mutations: ~10ms
  • VARIANT rendered (Totals about 390ms of visible flickering!)
  • Tracking callback (non-blocking): ~150ms

URL Redirect tests incur a much heavier penalty when instrumented in the browser only, sometimes up to 2-3 seconds of flicker!