Have your say on the future of fastly.toml

kailan
Wed, 10 Jul 2024 11:09:40 +0000
Compute

Since we launched Compute (then known as Compute@Edge) in 2019, developers have made use of the fastly.toml file format to describe the properties and requirements of their edge services.

The format, represented in TOML (Tom’s Obvious Minimal Language), has served our developer community well for those almost five years, but as the platform has grown and the way that we want to build Compute services has become clearer, it’s apparent that it is time to provide an easier way to specify the needs of your edge packages.

That’s why I’m pleased to kick off the next iteration of the Compute package manifest format, v4, by reaching out to you, Fastly’s developer community, to find out what you would like to see going forward.

The big question

Defining resources

Compute packages are more than just code, they depend on a variety of external resources – backends, config stores, secret stores, and the rest. One of the biggest challenges we’ve seen developers face is understanding the difference between locally configured resources (in fastly.toml’s [setup] and [local_server] sections) and remote resources configured on the Fastly control panel, CLI, or API.

Let’s start with some context – the [setup] section was designed to provide hints to our developer tooling about what resources are required by a service (but not their values, more on that later), to allow it to prompt developers to create them on the initial deployment of a starter kit. It does that well, but has led to some confusion around future deployments and whether updating values in the [setup] section would cause them to be updated Terraform-style on the remote service. Spoiler: it does not.

Then came Viceroy, our local testing server, which requires more than just knowing which resources are required but also needs their values. It’s not helpful for the server to know that a backend named web_assets is required, it also needs to know where to find it. This led to the introduction of the [local_server] section, which provides a similar (but unfortunately different!) format for defining resources as well as their configuration values.

These have existed in parallel since 2021, and you (our developer community) have made it work for your projects. But it’s clear that there must be a better way to specify this information in one place, rather than twice for each project. How do you imagine this would look and function going forward?

A smaller quality-of-life improvement

The verbosity of TOML

TOML is a fantastic language, and really shines in situations like the Cargo manifest format. However, the complex nature of Fastly service configurations can lead to something that is not very friendly to write by hand:

[setup]

  [setup.backends]

    [setup.backends.httpbin]

      address = "httpbin.org"

      description = "A simple HTTP Request & Response Service."

      port = 443

    [setup.backends.httpme]

      address = "http-me.glitch.me"

      description = "HTTP me is a tiny express app initally designed to replicate the features of HTTPBin.org"

      port = 443

[setup.config_stores]

  [setup.config_stores.service_config]

    description = "Configuration data for my service"

    [setup.config_stores.service_config.items]

      [setup.config_stores.service_config.items.s3-primary-host]

        value = "eu-west-2"

      [setup.config_stores.service_config.items.s3-fallback-host]

        value = "us-west-1"

As a large majority of new developers on Compute are using JavaScript, I wonder if it would be beneficial to switch to a more commonly-known format, such as JSON (à la npm’s package.json) or perhaps its lesser-known but more flexible successor, JSON5, which would retain TOML’s ability to include comments in the file. How do you feel about this? Is there a format that you think would be more suitable?

Anything else?

We want to hear your honest feedback about developing on Compute! This thread is open for your input on the above questions and any other thoughts you may have about the Compute package manifest format. We’ll be taking your discussion seriously and using it to inform the next version.

Also, stay tuned for a special episode of Fastly Developers Live next week where we’ll be bringing together some of the key engineers at Fastly working on Compute and developer tooling to have a chat about the above questions, as well as review your input from this thread. Click “Notify me” on the stream page to be notified when we go live: https://youtube.com/live/WjlRMZQ1CiY

22 posts - 11 participants