workingwithdata_ebook_april21_awc2op 4
TREATING DATA AS A PRODUCTThe same applies for data being used in real time. If your productrecommendations engine isn’t using the freshest data, then your users aregoing to be served outdated recommendations, negatively impacting theuser experience and harming your bottom line.The need for data observabilityThe challenge outlined above is the exact problem that data observabilityaims to fix. Data observability gives you transparency and control over thehealth of your data pipeline, such that when an issue does occur you canquickly understand:1 Where is the problem?2 Who needs to resolve it?Knowing this information makes it possible to find and resolve issues farquicker and minimize data downtime.But, how is data observability any different to monitoring?The best way to describe the difference is that monitoring covers the ‘knownunknowns’, whereas observability covers the ‘unknown unknowns’.MONITORINGKnown unknownsMonitoring tells you whensomething is wrongAssumes you knowwhat questions to askOBSERVABILITYUnknown unknownsDoesn’t assume thatsomething is wrongAssumes we don’t know whatall the questions are to ask45
TREATING DATA AS A PRODUCTTo take one example: as a Data Engineer, I know that I need to monitor theCPU usage of a microservice. But what is the complete landscape of thingsthat could go wrong that could impact the delivery of complete andaccurate data?It is impossible to predict every issue that could arise, and this is whereobservability steps in. Data observability assumes we don’t know what all thequestions are to ask, and instead gives us visibility of the things that reallymatter so that when something does go wrong we can investigate andresolve it quickly.46
- Page 1 and 2: TREATING DATAAS A PRODUCT
- Page 3 and 4: CHAPTER 1THE CHALLENGESOF WORKINGIN
- Page 5 and 6: TREATING DATA AS A PRODUCTBut it’
- Page 7 and 8: TREATING DATA AS A PRODUCT‘I want
- Page 9 and 10: TREATING DATA AS A PRODUCTBased on
- Page 11 and 12: TREATING DATA AS A PRODUCTTool eval
- Page 13 and 14: TREATING DATA AS A PRODUCTOur custo
- Page 15 and 16: TREATING DATA AS A PRODUCTAccording
- Page 17 and 18: TREATING DATA AS A PRODUCTAnd while
- Page 19 and 20: CHAPTER 2A GUIDE TODATA TEAMSTRUCTU
- Page 21 and 22: TREATING DATA AS A PRODUCTData is n
- Page 23 and 24: TREATING DATA AS A PRODUCTTourlane
- Page 25 and 26: TREATING DATA AS A PRODUCTA balanci
- Page 27 and 28: TREATING DATA AS A PRODUCTPEBMED is
- Page 29 and 30: TREATING DATA AS A PRODUCTOmio (for
- Page 31 and 32: TREATING DATA AS A PRODUCTHow Snowp
- Page 33 and 34: TREATING DATA AS A PRODUCTAs compan
- Page 35 and 36: TREATING DATA AS A PRODUCTThis appr
- Page 37 and 38: TREATING DATA AS A PRODUCT1 A dev n
- Page 39 and 40: TREATING DATA AS A PRODUCTPrior to
- Page 41 and 42: CHAPTER 4REDUCING DATADOWNTIME WITH
- Page 43 and 44: TREATING DATA AS A PRODUCTA real-li
- Page 45: TREATING DATA AS A PRODUCTThe spira
- Page 49 and 50: CHAPTER 5HOW DATASTORYTELLING CANMA
- Page 51 and 52: TREATING DATA AS A PRODUCTWhat is d
- Page 53 and 54: TREATING DATA AS A PRODUCTWhy shoul
- Page 55 and 56: TREATING DATA AS A PRODUCTAsk yours
- Page 57 and 58: TREATING DATA AS A PRODUCTWrap it u
TREATING DATA AS A PRODUCT
To take one example: as a Data Engineer, I know that I need to monitor the
CPU usage of a microservice. But what is the complete landscape of things
that could go wrong that could impact the delivery of complete and
accurate data?
It is impossible to predict every issue that could arise, and this is where
observability steps in. Data observability assumes we don’t know what all the
questions are to ask, and instead gives us visibility of the things that really
matter so that when something does go wrong we can investigate and
resolve it quickly.
46
Change language