Perception Filters and Data Quality

Apr 25, 2012 by Jim Harris in Data Quality

In the science fiction universe of Doctor Who, a perception filter is an energy field surrounding an object that convinces people to ignore it. Most notably, it’s used to make the Doctor’s TARDIS seem ordinary wherever it lands, which is important because the chameleon circuit of the TARDIS, which is supposed to alter its outside appearance in order for it to blend in with its environment, was faulty, leaving it locked in the blue box shape of a 1960s-style London police box.Thanks for that Doctor Who trivia Jim, but what does this have to do with data quality?

Well, as I have traveled through time in the data management space, I have frequently observed organizations ignoring their data quality issues as if they were surrounded by a perception filter making poor data quality seem ordinary wherever it lands. Data used to support business operations and drive strategic business decisions is often faulty, but despite these obvious data quality issues staring organizations right in the face, no one seems to notice them.

Data denial is the most common perception filter for data quality issues. This is a natural self-defense mechanism for the people responsible for the business processes and technology surrounding data, and understandable since nobody wants to be blamed for causing, or failing to fix, data quality issues.

Just like the Doctor’s TARDIS, the blue box of data quality is bigger on the inside, meaning that the complexities underlying the issues make data quality a much bigger challenge than it first appears to be, and make it much easier to surround data quality issues with a perception filter and ignore them.

However, in Doctor Who, a perception filter does not work if the object it surrounds draws too much attention to itself, or if someone is specifically searching for the object in question. Sometimes, data quality issues draw too much attention to themselves, such as when bad data makes 20,000 British men pregnant. Other times, data quality issues are found because they are specifically searched for, such as when regulators verify data for compliance with requirements like Solvency II.
Is your organization surrounding its data quality issues with a perception filter?

Tags: Data Quality