Big Data Edition

iCEDQ now uses the power of Hadoop Cluster – Generally Available
Big Data Edition Page Banner

iCEDQ’s Big Data Edition now utilizes Hadoop Cluster for processing, and thus massively scales the data testing capabilities to Terabyte and Petabytes.

Big Data Edition will help companies successfully migrate to Hadoop or Big Data environments. Now companies can use iCEDQ to test or monitor data in the big data world of data lakes. It supports different flavors of Hadoop like Cloudera, Horton, and Map-R or cloud-based platforms such as Azure or AWS.

  • Scalability, unlimited data processing capabilities
  • Hadoop cluster processing
  • Utilize your existing Hadoop cluster Apache, Cloudera, Horton Works, or Map-R
  • Scale with the number of data nodes
  • No need to extract data out of Hadoop cluster
  • Even tests your legacy data by using Hadoop as a processing engine

Standard Edition - Engine

  • Uses SMP architecture to run rules
  • Designed for Legacy databases and ETL

Big Data Edition - Engine

  • Uses Hadoop architecture to run rules
  • Designed for Big data, HDFS files, Hive

Why Big Data Edition?

There is a shift in the way companies are processing data. Earlier, it was about millions of records but now companies are easily processing billions of records. Hence companies are shifting to Hadoop infrastructure with data lakes, large HDFS files, Hive and other big data processing technologies.

While the data has grown exponentially, the time to processes or test this data have not. While the Standard edition with its in-memory processing is fast the SMP architecture is a limitation. One just cannot get a terabyte of HDFS data out of the multi-node Hadoop cluster and use SMP machine to analyze it.

The iCEDQ Big data edition is a solution that uses the complete power of the existing Hadoop cluster and does on spot processing. This allows the least amount of data over the network and uses the power of Hadoop cluster to give both performance and scalability.

iCEDQ Big Data Edition tested 1.7 Billion rows in less than 2 minutes and Recon Rule with around 20 expressions for 1.7 billion rows in less than 30 minutes