Collecting Counts of Requests without Sampling

Problem Overview

Accountants are busy and our customers need a way to count the number of requests utilized by different departments for internal billing and cost sharing. The requests that are associated with the different departments do not necessarily need to be sampled. The counts by utilization may be available through requests to the Fastly Next-Gen WAF API so that the data collection on the counts may be done programmatically and on a regular recurring basis.

Solution

Create custom corp signals for each department

In this example, we have the product line apples and bananas. The following rules both have request sampling set to None. Requests that match the criteria for these rules will then also have this signal applied. However, the requests themselves are only part of the time series data and should not impact the sampling rate of other requests.


Account for undocumented requests

Applications change and new applications are brought on-board. We can surface these requests that need to be accounted for in a corp level request rule like the following.


Collecting the data

You may use the following API endpoint to get these metrics in a programmatic way.

timeseries_requests_get

Here’s an example request to the API endpoint and filtered output using curl and jq.

curl "https://dashboard.signalsciences.net/api/v0/corps/brooks_cunningham_lab/sites/terraform_ngwaf_site/timeseries/requests?from=-30d&tag=corp.product-line-bananas" \
-H x-api-user:$SIGSCIEMAIL \
-H x-api-token:$SIGSCITOKEN \
| jq .data[].summaryCount
1008

This data MUST be collected within 30 days as the retention for time series data is only 30 days based on the current online documentation (About data storage and privacy | Fastly Next-Gen WAF).

Have you implemented something similar to help calculate internal utilization across teams? Let us know in the discussion.