iCEDQ High Throughput Edition

Tests your data rapidly, by using parallel processing.

Now efficiently test your complete data pipeline without any hassle using iCEDQ High Throughput Edition. The HT edition is specially designed for organizations looking for better performance than the standard version and has more significant data sets to be evaluated in a shorter time. This is also an in-memory engine built-in Java, but it uses Akka to parallelize the data processing.

iCEDQ HT Edition supports six different types of Rules, which include Standard Edition Rules.

HT Validation
HT Recon

Validation Rule
Recon Rule

Checksum Rule
Script Rule

How does HT Edition work?

In iCEDQ High Throughput Edition any process of a Rule or Regression Pack occupies multiple cores on the server. The HT Rules Engine supports a mixed workload of Standard and HT Rules.
In simple words, HT Edition is Standard Edition on steroids.

This Rules engine also connects to data sources using JDBC. If it is a database, then sorting is pushed down to the database. In case of a file, the Rule engine will do the sorting.
HT Rules engine sorts the file data much faster as the process uses multiple cores.
Then the Rules engine reads the data in larger micro-batch compared to Standard edition. While the data is being read, the engine identifies matching and missing business keys.
After this, for matching records, data is split across multiple cores to be evaluated for the expression. This ability to process data parallelly for expressions gives 4-5 x performance benefit over Standard edition. And as it is identifying issues, it is writing data issues to an exception report.
Once the engine completes micro-batch processing; it reads next micro-batch into the memory. This process continues until all the data is evaluated.

Features

Advanced User Level Security

Apart from system level access, database user level security and LDAP security are also available. You can also limit access to data for specific users.

CI/CD pipeline integration

iCEDQ HT Edition supports Jenkins plugin and has an API for Jira, Bamboo and Git, which results in faster application delivery, enhanced innovation and gives a stable operating environment. You can also implement DevOps for your data centric project with it.

Custom Reporting and Dashboard

It has an embedded custom reporting and dashboarding solution, where you can create custom dashboards, create custom reports, share reports and dashboards. It has HTML-5 reporting utility, dashboard utility and is independent of any external tool for this functionality.