Many companies are moving away from traditional databases to new database appliances or big data systems. These projects not only help IT, they provide benefits for the business as well. Data Migration projects have high a risk of failure due to constrained budgets, timelines and resources. As per information week article 10 Big Data Migration Mistakes one of the mistakes is failure to validate and test data migration process.
Data migration involves recreating/replicating the schema structure of the source database in the new target database. The inability to correctly replicate the schema, may lead to data truncation or format issues as well as failure of the data migration process.
Incomplete or data loss happens when the data migration process does not migrate the expected data from source to target. Loss of data is the highest risk of any data migration project and the cost involved in resolving these issues are high.
Data migration processes might bring over invalid or inconsistent data to the new target system causing data integrity issues. Data integrity issues include: null values, foreign keys, duplicate data issues and much more.
iCEDQ is a Data Testing software platform which is heavily used by customers for Data Migration Testing. It’s unique In-Memory rules engine allows users to compare schema structures and content inside the database effectively. It helps identify issues generated by the data migration process by comparing complete datasets between source and target.
Schema Compare: Validate complete data structure between source and target databases. User can identify missing tables/ columns/ views or any other object. It can also identify mismatching data types.
Compare Initial Data Load: When the legacy data is migrated into the target system, reconcile source and target data to ensure they match completely and they have same initial state.
iCEDQ can help compare the output generated from legacy and new system to make sure the same data is being generated from both systems. Unless there is a business rule change in which case you test for the business rule change.
Our unique rules engine compares data in-memory, in chunks. This allows users to compare huge volumes of data making sure you test all the data. It can effectively compare data between different databases and files.
In data migrations, sources & target schema structures are the same, so our Rules Wizard can help in creating the rules automatically for the users. This in turn saves huge amounts of time as manual processes are eliminated.
Once the rules and test suites have been created to test in one environment, users can easily point the rules to different environments database or file connections. This make rules reusable across environments in different phases of data migration projects.