iCEDQ Big Data Edition

Test your big data efficiently, with parallel processing on Hadoop clusters.

YouTube player

Organizations who have a big data ecosystem and want to scale testing of billions of rows can use our iCEDQ Big Data Edition. The iCEDQ Big Data Edition uses Java, Groovy, and Apache Spark to process the data for testing. Currently, iCEDQ supports Cloudera/ Hortonworks platform for running Spark Engine. We are in the process of adding following platforms too AWS EMR, Databricks, Azure HD Insight, GCP Cloud Dataflow.

iCEDQ Big Data Edition supports six different types of Rules.

  • Cluster Validation
  • Cluster Recon
  • Validation Rule
  • Recon Rule
  • Checksum Rule
  • Script Rule
iCEDQ Spark Edition - iCEDQ

How does Big Data Edition work?

How does Spark Edition work-iCEDQ

Every Rule or Regression Pack is evaluated using Apache Spark cluster. User can scale the performance by scaling the Spark cluster.

In the Big Data Edition, the Rules engine uses JDBC or native connectors to read data from different sources. Unlike Standard and HT Rules engine, the Spark Rule engine reads all the data into the cluster and sorts it parallelly. It then identifies missing and matching records and evaluate expressions in parallel. The Spark Engine writes all the data issues to an exception report in HDFS.

Big Data Edition gives 100x better performance than Standard edition for High Volume Datasets.


Built in Scheduler - iCEDQ

Built-in Scheduler

Now schedule all your tests in advance, iCEDQ Big Data Edition supports daily, weekly and monthly job scheduling. You can run jobs offline also.

Regression Testing Suite - iCEDQ

Regression Testing Suite

iCEDQ Big Data Edition supports creation of regression testing packs by combining multiple rules. You can also store batches for future use.

Alerts and Notifications - iCEDQ

Alerts and Notifications

It connects with users via email notifications and notifies them about success or failure of a job. It also notifies for scheduling tool alerts.

Advanced User Level Security - iCEDQ

Advanced User Level Security

Apart from system level access, database user level security and LDAP security are also available. You can also limit access to data for specific users.

CICD pipeline integration - iCEDQ

CI/CD pipeline integration

You can implement DevOps for your data centric project with iCEDQ Big Data Edition. It supports Jenkins plugin and has an API for Jira, Bamboo and Git, which results in faster application delivery, enhanced innovation and gives a stable operating environment.

Custom Reporting and Dashboard - iCEDQ

Custom Reporting and Dashboard

It has an embedded custom reporting and dashboarding solution, where you can create custom dashboards, create custom reports, share reports and dashboards. It has HTML-5 Reporting utility, Dashboard utility and is independent of any external tool for this functionality.

Speed - iCEDQ


It can reach up to 14.1M rows/sec speed while testing data with Recon rule.

Centralized-Knowledge-Repository - iCEDQ

Centralized Knowledge Repository

It stores the metadata and rules in database repository which can be PostgreSQL, Oracle or SQL Server. Clients can access the centralized metadata from any PC, build custom reports, store and archive execution log for compliance or use APIs to share with other applications.

New-Connectors - iCEDQ

New Connectors

The Spark engine and the connectors are separate which allows easy addition of new connectors.

Getting started is easy

Be up and running in minutes.