workingwithdata_ebook_april21_awc2op 4

TREATING DATA AS A PRODUCT
Our approach to observability at Snowplow
What Ralph and the rest of the business really care about is whether they can
trust the data. Is it complete and is it fresh? Crucially, data observability
should align the more technical data with business outcomes so that
business and engineering teams are talking the same language and moving
in the same direction.
Our approach to observability at Snowplow is to focus on two key metrics;
throughput and latency. These are emitted from each part of the pipeline
and as a result, if a bottleneck occurs at any point then it is far simpler to
diagnose, allowing you to take corrective action immediately.
Our plan at Snowplow is to make it the most observable behavioural data
pipeline available. We have already added observability to our BigQuery
Loader, and we’ll soon be launching it within our RDB loader and Enrich
assets too.
47