I was very excited when I learned about Fastly’s new log streaming into Google BigQuery feature:
I am currently working with a large publisher. They don’t use Fastly yet (unfortunately) but are currently rolling out Snowplow Analytics. Like most tracking solutions, Snowplow collects data by requesting a little tracking pixel which’s URL contains a list of key-value parameters, e.g. like this:
In general, any analytics solution needs a server-side collector component that receives GET or POST requests and passes the data along to the subsequent processes, e.g. Snowplow’s enrichment component and eventually a data sink.
Based on Fastly’s new feature, my idea is to point the tracking requests to Fastly and stream the logs into BigQuery to build real-time dashboards. However, the tutorial doesn’t mention a way to extract the URL’s payload and strean not only the default parameters to BigQuery but also the custom URL parameters that Snowplow works with.
What do you think about this idea? Does anyone know if it is possible, maybe using VCL?