Automate Data Migration Testing with iCEDQ

Use iCEDQ to test your data migration to big data or to cloud Redshift, Vertica, Snowflake, Azure and more…

Currently, many companies are moving away from traditional databases to new database appliances, or big databases or in the cloud. Companies are taking these considerable data migration risks because, without database migration, you cannot get the benefits of the fantastic new technologies like performance, big data processing, and lower cost maintenance.  

Bottom line, companies need to migrate data, and for migration to be successful they need to do data migration testing. In the time and resource-constrained world, there is no alternative to automated data migration testing. 

Manual data migration testing has become the Achilles heel for data migration projects because: 

  • Huge data volume 
  • Existence of SaaS databases in the cloud
  • Shrinking project Time, Money and Resources
  • Lack of qualified data testers
  • Inability to reconcile data between legacy data and new databases
  • The number of tables has also increased exponentially
  • The need for quicker turnaround time
  • Regression testing
  • The advent of DevOps/DataOps methodology does not support manual testing 

Data Migration Testing Solution with iCEDQ

Data migration involves recreating/replicating the schema structure of the source database in the new target database. The inability to correctly replicate the schema may lead to data truncation or format issues as well as the failure of the data migration process.

Data Migration Process

Big DataBig data testing challenges & solutions

The Challenge 

Manual testing for a large volume of data involved in Big data migration is nearly impossible. Testers end up doing only summary checks.

The iCEDQ Solution 

iCEDQ has a Big Data Edition than can test billions of migrated records.  Because of iCEDQ’s huge processing capabilities, each migrated rows can be compared with the record in the legacy database. iCEDQ already has built-in Big Data connectors for Hive, Impala, Cassandra, HBase and more. 

Cloud MigrationCloud Migration challenges & solutions

The Challenge 

While migrating the cloud-based database, comparing data between the on-premise database and cloud database is nearly impossible.

The iCEDQ Solution 

iCEDQ works well in hybrid clouds as well. It has connectors for an on-premise database such as Oracle SQL server as well as cloud databases such as Snowflake, Redshift, Azure Etc.

Schema
schema challenges & solutions

The Challenge 

Schema: Ensuring the database schema matches with the source database.

The iCEDQ Solution 

iCEDQ can compare and match the source databases structures to the destination database. It does so by comparing the metadata of the schema. It also has the capabilities to create advanced compariosn rules for equivalent data types.

Regression testing

The Challenge:

It requires recalling past test and recombines them to create regression packs. However, with a manual test, this is nearly impossible to do. Neither the test are organized and stored centrally, nor they are in a format where they are useful.

The iCEDQ Solution 

iCEDQ has  a repository to store test rules at the most granular levels. When the need arises, the rules can be recalled and combined to create a regression pack. This is also possible because the rules are tagged with metadata and this makes it possible to recall existing rules and create data migration regression test packs when needed.

Data Truncation & rounding errors

Data Truncation & rounding errors

The Challenge 

The data migration process can introduce errors in data which can result in data truncation and data errors.  These are very difficult to check manually.

A simple database migration involving 500 tables woth 50 columns each and million record averge colud result in 25,000,000,000 (500 x 50 x 1,000,000) data exceptions.

The iCEDQ Solution 

iCEDQ’s rules engine can easily compare millions and billions of attributes individually without any problems.

Connectivity

The Challenge 

Databases involved in data migration are either ancient or might be brand new. Having the right connectivity is very difficult.

The iCEDQ Solution 

The iCEDQ’s data validation and comparison engine is separate and built independently from the connectivity. This allows iCEDQ to build specialized connectors, for legacy, new, cloud or any other system.

With iCEDQ now you can…

Pre-checks for Target Schema

  • Schema Compare: Data migration involves recreating/replicating the schema structure of the source database in the new target database. The inability to correctly replicate the schema may lead to data truncation or format issues as well as the failure of the data migration process.
  • Validate complete data structure between source and target databases. User can identify missing tables/ columns/ views or any other object. It can also identify mismatching data types. 

Initial Data Migration Testing

  • Compare Initial Data Load: When the legacy data is migrated into the target system, reconcile source and target data to ensure they match entirely, and they have the same initial state. 

Post Data Migration Testing

  • iCEDQ can help compare the output generated from legacy and the new system to make sure the same data is generated from both systems. Unless there is a business rule change in which case you test for the business rule change. 

Benefits of iCEDQ for Data Migration Testing

What was impossible with manual testing, is now possible with iCEDQ’s technology. The data validation and reconcile engine also with cloud enablement is a game changer. Data migration test automation also enables implementation of DevOps for Data.

  • Test 1,000,000,000s records of with full volume testing, no need for sampling or summary checks.
  • 70% of the QA time is saved as compared to manual testing coding.
  • 200% improvement in Test coverage.
  • 33% reduction in timelines.

Data Migration Testing Features

Rules Wizard

This allows users to generate rules for direct comparison between two databases when the schema structure is matching. 

Schema Compare

Compare schema across environments and databases to identify missing objects between source and target.

CI Tool Integration

Integrate iCEDQ with any continuous build integration tool like Jenkins or Bamboo to make iCEDQ a part of your workflow.

Full Volume Testing

With our In-memory rules engine you can compare complete dataset between source and target, no more sampling.

Reusability

Execute same set of rules/ test across different environment and databases by reusing them without changing underlying definition.

Scheduling

Schedule you data migration testing rules to be executed whenever you want and get the results in your mailbox.

Use Cases