workingwithdata_ebook_april21_awc2op 4
TREATING DATA AS A PRODUCTWhat this could look like in practiceWe can define what the events coming to your data warehouse look likebefore they are even sent by writing a set of rules. For example, the ruleset fora click event:{"element_name": {"enum": ["share","like","submit_email","rate",..."close_popup”],"description": "The name of the element that is clicked"},"value": {"type": ["string", "null"],"description": "What is the value associated with the click"},"element_location": {"type": ["string", "null"],"description": "Where on the screen is the button shown eg.Top, left"},"click_error_reason": {"type": ["string", "null"],"description": "If the click resulted in an error, what wasthe reason eg. Invalid character in text field"}}37
TREATING DATA AS A PRODUCTPrior to loading the data to your warehouse, each event is checked to see if itconforms to the rules laid out. There are two ways of doing this:• If you are using a 3rd party data collection vendor such asGA – validate client side• If you have 1st party data collection such as a home-built pipelineor Snowplow – validate in the data collection pipeline, prior towarehouse loadingEither method means the structure of data in the warehouse is controlledstrictly by those consuming it.Subset of properties sentautomatically for every eventCustom properties of the click eventUser ID Platform Timestamp EventNameElement_name Value Element_location Click_error_reasonJoeWeb2019-10-0112:33:21Page_viewJoeWeb2019-10-0112:33:29Click submit_email joe@email.com homepage_footerJoeiOS2019-10-0123:31:03Click rate no_rating_selectedWith this simple change to the setup of introducing an enforced ruleset, yourfront-end devs can finally QA your analytics in the same way as they wouldQA the rest of any build, by adding to their integrated testing suite usingsomething like the open-source tool Micro.38
- Page 1 and 2: TREATING DATAAS A PRODUCT
- Page 3 and 4: CHAPTER 1THE CHALLENGESOF WORKINGIN
- Page 5 and 6: TREATING DATA AS A PRODUCTBut it’
- Page 7 and 8: TREATING DATA AS A PRODUCT‘I want
- Page 9 and 10: TREATING DATA AS A PRODUCTBased on
- Page 11 and 12: TREATING DATA AS A PRODUCTTool eval
- Page 13 and 14: TREATING DATA AS A PRODUCTOur custo
- Page 15 and 16: TREATING DATA AS A PRODUCTAccording
- Page 17 and 18: TREATING DATA AS A PRODUCTAnd while
- Page 19 and 20: CHAPTER 2A GUIDE TODATA TEAMSTRUCTU
- Page 21 and 22: TREATING DATA AS A PRODUCTData is n
- Page 23 and 24: TREATING DATA AS A PRODUCTTourlane
- Page 25 and 26: TREATING DATA AS A PRODUCTA balanci
- Page 27 and 28: TREATING DATA AS A PRODUCTPEBMED is
- Page 29 and 30: TREATING DATA AS A PRODUCTOmio (for
- Page 31 and 32: TREATING DATA AS A PRODUCTHow Snowp
- Page 33 and 34: TREATING DATA AS A PRODUCTAs compan
- Page 35 and 36: TREATING DATA AS A PRODUCTThis appr
- Page 37: TREATING DATA AS A PRODUCT1 A dev n
- Page 41 and 42: CHAPTER 4REDUCING DATADOWNTIME WITH
- Page 43 and 44: TREATING DATA AS A PRODUCTA real-li
- Page 45 and 46: TREATING DATA AS A PRODUCTThe spira
- Page 47 and 48: TREATING DATA AS A PRODUCTTo take o
- Page 49 and 50: CHAPTER 5HOW DATASTORYTELLING CANMA
- Page 51 and 52: TREATING DATA AS A PRODUCTWhat is d
- Page 53 and 54: TREATING DATA AS A PRODUCTWhy shoul
- Page 55 and 56: TREATING DATA AS A PRODUCTAsk yours
- Page 57 and 58: TREATING DATA AS A PRODUCTWrap it u
TREATING DATA AS A PRODUCT
Prior to loading the data to your warehouse, each event is checked to see if it
conforms to the rules laid out. There are two ways of doing this:
• If you are using a 3rd party data collection vendor such as
GA – validate client side
• If you have 1st party data collection such as a home-built pipeline
or Snowplow – validate in the data collection pipeline, prior to
warehouse loading
Either method means the structure of data in the warehouse is controlled
strictly by those consuming it.
Subset of properties sent
automatically for every event
Custom properties of the click event
User ID Platform Timestamp Event
Name
Element_name Value Element_location Click_error_reason
Joe
Web
2019-10-01
12:33:21
Page_view
Joe
Web
2019-10-01
12:33:29
Click submit_email joe@email.com homepage_footer
Joe
iOS
2019-10-01
23:31:03
Click rate no_rating_selected
With this simple change to the setup of introducing an enforced ruleset, your
front-end devs can finally QA your analytics in the same way as they would
QA the rest of any build, by adding to their integrated testing suite using
something like the open-source tool Micro.
38
Change language