ETL & Data Warehouse Testing Software

Test in Dev & Monitor in Production

iCEDQ software is used to completely automate the testing of ETL/ Data Warehouse and Data Migration. It is also used to monitor production data for controls & compliance. iCEDQ not only helps the technical team by allowing full volume, regression and integration testing but also the management, by allowing them better insight into their data issues.

How iCEDQ Works

It allows users to create different types of Rules (tests) to validate individual data sets or compare data sets. Below is the behavior of a Rule created inside iCEDQ.

Step 1. Read Data

The first step is to connect to source(s) and target and read the data from a file or database. It then loads a subset of data in to our In-Memory rules engine.

Step 2. Apply Set Theory

Once the data is in memory, it applies “Set Theory” (diff) to identify below rows based on specified columns.

  • Rows which are present in source not in target (Source minus Target)
  • Rows which are present in target not in source (Target minus Source)
  • Rows which are present in both source & target (Source intersect Target)

Step 3. Evaluate Expression

It evaluates expression for transformation, calculation or any other expression. These expressions are applied on the data, which is common between source & target.

Step 4: Generate Report

After performing the steps above it writes all of the data issues into an Exception Report.

Functionality

Rules Engine

All of the comparison and validation tests are done by our In-Memory Rules Engine, which uses Java & Groovy Scripts and is capable of comparing and testing millions of rows. Users can also easily scale their data warehouse testing with multiple engines.

Integration

iCEDQ has an out-of-the-box integration with HP QC/HP ALM through a Web Services interface and also ships with a Command Line Utility. This allows users to integrate iCEDQ with any scheduling tool, continuous build integration or ETL tool. Below is list of some of the platforms we’ve integrated iCEDQ with,

  • Control – M/ Tidal/ Autosys
  • Jenkins/ Bamboo
  • Informatica/ DataStage/ Talend

Collaboration

iCEDQ has a web based GUI allowing global teams to work together regardless of location. All of the information is stored in a centralized database repository making it easier to access any information at any time.

Reporting

Exception reports generated by the rules (tests) highlight all of the data issues between source & target. It helps users identify the exact rows and columns containing data issues.

iCEDQ also comes with a dashboard and pre-defined tabular reports which allows business users and management to get more insight/transparency across projects and organizations.

Data Connectors

Our rules engine makes it easy to compares data across databases and files. This is achieved by allowing users to connect any database or file format using different adapters. See the complete list of data connectors

Scheduling

ICEDQ provides a scheduling tool so that users can schedule any rules or batches. Users can schedule the jobs to run one time, daily or weekly depending upon their requirement.