Distant Early Warnings

Jan 25, 2010 by Phil Simon in Data Governance, Data Quality

One of my favorite phrases is “distant early warning”, defined in Wikipedia as:

The Distant Early Warning Line, also known as the DEW Line or Early Warning Line, was a system of radar stations in the far northern Arctic region of Canada, with additional stations along the North Coast and Aleutian Islands of Alaska, in addition to the Faroe Islands, Greenland, and Iceland. It was set up to detect incoming Soviet bombers during the Cold War, a task which quickly became outdated when intercontinental ballistic missiles became the main delivery system for nuclear weapons.

Ah…that’s wonderful, Phil. Great factoid.

It is, actually. With respect to data quality is, spotting the four following DEWs is critical for consultants and DQ practitioners.

1. Poorly-Trained End-Users

It all starts with the end-user. I’m fairly sure that companies such as Apple follow that mantra. The same concept applies to those who key data into CRM, ERP, and other types of systems as part of their jobs. One would hope that analysts, managers, and senior folks know what they’re doing, although rare is the VP who has to roll up her sleeves and key.

Simon Says

Many entry-level employees or temps may not fully appreciate the DQ ramifications of their actions—or lack thereof. Good apps and systems with rules can certainly minimize the chance of error but it’s important to be realistic. Keep an eye out for less-than-conscientious data entry folks and use audit reports to identify outliers and odd data trends.

2. Redundant or Overlapping Systems

Any time that I see systems that ostensibly do the same thing, I cringe. I can just see different database schema, tables, indexes, and the like. Absent an MDM solution, my question is not, “How’s your data quality?” Rather, it’s “How bad is your data quality?”

I once worked for an organization that had no fewer than nine disparate compensation systems, some of which contained (in theory) the same information. Needless to say, the simplest way of finding out an employee’s rate of pay or bonus was to ask him, “What did we pay you, anyway?”

Simon Says

Temper DQ expectations. While MDM tools can make things less chaotic, there is no silver bullet. Take a long-term viewpoint and seek to phase out overlapping systems. Watch your DQ improve.

3. Data Gaps

The counterpart of the above, data only housed in standalone Excel spreadsheets, Access databases, or even paper need to be integrated into the larger whole.

Simon Says

Do what you have to do. Create temp tables in SQL Server or Oracle. Use views. It doesn’t matter. Get people away from the mentality of keeping “their data” in one place while others’ data lies somewhere else.

4. Internal Politics

The last of the DEWs is arguably the most pernicious. Senior executives who cannot agree on data definitions or a common infrastructure are not terribly likely to adhere to DQ standards. We’ve all seen this before.

Simon Says

We are moving towards a more integrated world; the logical arguments to maintain silos of information seem to be holding less water these days. The CEO, CIO, CMO, or CXO needs to smack everyone in the head. Are the goals of a division, department, or region more important than the goals of the organization as a whole?

Tags: Data Governance, Data Quality