Repurposing Fastly's S3 guide for Fastly Object Storage as a VCL backend

Object Storage
1

Fastly has a great guide to using Amazon S3 as an origin for VCL services. Because Fastly Object Storage is S3-compatible, we can repurpose this guide and take advantage of Object Storage’s free egress.

First, you’ll need to sign up for Fastly Object Storage. In the Fastly web UI, visit Resources → Object Storage, or visit this page. You’ll have the opportunity to sign up there. If you’re already signed up, you’ll see a page where you can create an access key, which you’ll need to do.

Of course, to use a Fastly Object Storage bucket as a backend for a VCL service, you’ll need to upload some content to one. We cover a few different ways to do that in other posts, so for this guide we’ll assume you already have a bucket with some data in it.

On the Object Storage page, click the “+ Create Key” button. Give it any description you want, and choose the “Read only” scope. Click the “Create” button and make note of the Access key ID and Secret access key. This the only time you will be able to see the secret key. You will use these in your VCL configuration.

Turning back to the guide, when you create a service, instead of using an amazonaws.com hostname, you’ll use <REGION>.object.fastlystorage.app, where the region is the region where your bucket lives. As of this writing this is either us-east, us-west, or eu-central. You will want to leave “Enable TLS?” set to “Yes.”

All buckets in Fastly Object Storage are private, so we’ll continue to follow the directions for using an Amazon S3 private bucket. Copy the VCL snippet into your service. In the VCL, you’ll find a few configuration variables that need to be set:

set var.awsAccessKey = "YOUR_AWS_ACCESS_KEY";   # Change this value to your own data
set var.awsSecretKey = "YOUR_AWS_SECRET_KEY";   # Change this value to your own data
set var.awsS3Bucket = "YOUR_AWS_BUCKET_NAME";   # Change this value to your own data
set var.awsRegion = "YOUR_AWS_BUCKET_REGION";   # Change this value to your own data
set var.awsS3Host = var.awsS3Bucket ".s3." var.awsRegion ".amazonaws.com";

For use with Fastly Object Storage, you will set:

  • awsAccessKey to the Access key ID you got from the Fastly web UI
  • awsSecretKey to the Secret access key you got from the Fastly web UI
  • awsS3Bucket to the name of the Fastly Object Storage bucket you want to serve content from
  • awsRegion to the Fastly Object Storage region your bucket lives in: us-east, us-west, or eu-central
  • awsS3Host to var.awsRegion ".object.fastlystorage.app";

If you are serving large files out of Object Storage, you may also need to enable Streaming Miss.

Once you’ve deployed your service, you should be able to visit the toplevel to list the files in your bucket, and navigate directly to files within your bucket. You can make further VCL customizations to limit access.

Give it a try and let us know what you think!

3 Likes
2

Enable Segmented Caching too if you are serving large objects (greater than 20MB for most recently-created Fastly accounts).

1 Like