Big Data Testing
iCEDQ’s Big Data Testing Edition now utilizes Hadoop Cluster for processing, and thus massively scales the data testing capabilities to Terabyte and Petabytes.
Big Data Testing Edition will help companies successfully migrate to Hadoop or Big Data environments. Now companies can use iCEDQ to test or monitor data in the big data world of data lakes. It supports different flavors of Hadoop like Cloudera, Horton, and Map-R or cloud-based platforms such as Azure or AWS.
- Scalability, unlimited data processing capabilities
- Hadoop cluster processing
- Utilize your existing Hadoop cluster Apache, Cloudera, Horton Works, or Map-R
- Scale with the number of data nodes
- No need to extract data out of Hadoop cluster
- Even tests your legacy data by using Hadoop as a processing engine
Standard Edition – Engine
Uses SMP architecture to run rules
Designed for Legacy databases and ETL
Big Data Testing Edition – Engine
Uses Hadoop architecture to run rules
Designed for Big data, HDFS files, Hive
Why Big Data Testing Edition?
There is a shift in the way companies are processing data. Earlier, it was about millions of records but now companies are easily processing billions of records. Hence companies are shifting to Hadoop infrastructure with data lakes, large HDFS files, Hive, and other big data processing technologies.
While the data has grown exponentially, the time to processes or test this data have not. While the Standard edition with its in-memory processing is fast the SMP architecture is a limitation. One just cannot get a terabyte of HDFS data out of the multi-node Hadoop cluster and use an SMP machine to analyze it.
The iCEDQ Big data edition is a solution that uses the complete power of the existing Hadoop cluster and does on spot processing. This allows the least amount of data over the network and uses the power of the Hadoop cluster to give both performance and scalability.
iCEDQ Big Data Testing Edition tested 1.7 Billion rows in less than 2 minutes and Recon Rule with around 20 expressions for 1.7 billion rows in less than 30 minutes