Data has become critical to business. Hence, enterprises are investing time, money and resources in data-centric systems such as data warehouse, MDM, CRM & migration projects. However, all research done by independent agencies indicates that
- There is such a high failure/delays in implementations of data-centric projects
- Users still don’t trust data coming from data warehouses
Having worked in this industry since 1996, mostly on the data side of the software equation, I couldn’t agree more! Of course there are many reasons for the failure/ delay, but the main reason that comes to my mind is the mismanagement of data risk and the lack of experience in managing data centric projects and systems. When was the last time you saw management and architects identify data risk and its mitigation strategy?
If you need further proof, just check the company’s product portfolio for data centric system. You will find the two critical components that mitigate data risk are missing.
- ETL testing software, data testing software, data warehouse testing software, and data migration testing… call it by any name
- Data monitoring and data quality governance software
The only way to mitigate the data risk is by having an ETL testing software and data quality governance in production (ETL process will define your data quality). But people are not even aware of it. Hence the chances of success in development or in production are bleak.
Ask yourself, what do you have in place to test ETL processes? And how do you monitor your production data?
