Data Quality — You’re Measuring It Wrong
This article was first posted on Towards Data Science.One of our shoppers not too way back posed this question:“I wish to organize an OKR for ourselves [the data team] spherical data availability. I’d like to establish a single KPI that will summarize availability, freshness, prime quality.What’s probably the greatest methods to do this?”I can’t let you understand how quite a bit pleasure this request launched me. As anyone who’s obsessive about data availability— yeah, you be taught that correct: in its place of sheep, I dream about null values and data freshness right this moment — it’s a dream come true.Why does this matter?If you’re in data, you’re each presently engaged on a data prime quality enterprise in any other case you merely wrapped one up. It’s the regulation of unhealthy data — there’s on a regular basis further of it.Traditional methods of measuring data prime quality are generally time and resource-intensive, spanning numerous variables, from accuracy (a no brainer) and completeness, to validity and timeliness (in data, there’s no such issue as being fashionably late). But the good news is there’s a larger answer to technique data prime quality.Data downtime— intervals of time when your data is partial, defective, missing, or in some other case inaccurate — is a crucial measurement for any agency striving to be data-driven. It might …