All About ETL Testing, Data Migration & Data Warehouse Testing

Migrating Database-iCEDQ

Migrating Database to Redshift, Snowflake, Azure DW and Test with iCEDQ

How to Migrate to Redshift, Snowflake, Azure DW and Test with iCEDQ Many companies are migrating database actively for multiple reasons  End of life (Ex. Netezza)  Database cannot handle big data  Companies moving the database to the cloud (ex. Redshift, Vertica, Snowflake, Azure, etc.)  In-house database maintenance is becoming expensive   License Cost for on-premise install  The complexity of migration…

Details

What Is BI Testing and the Importance of BI Report Testing-iCEDQ

What is BI Testing and the Importance of BI Report Testing

What Is BI Testing and the Importance of BI Report Testing Reports and dashboards are extensively used to make business decisions. These decisions later form the basis for the company’s growth and success. If BI reports are wrong, then decisions taken based on these reports will also be wrong. Inaccurate reports affect the organization’s credibility…

Details

A Practical Guide for Data Centric Testing Automated ETL Testing-iCEDQ

A Practical Guide for Data Centric Testing: Automated ETL Testing

“Information is the oil of the 21st century, and analytics is the combustion engine”- Peter Sondergaard, Senior Vice President, Gartner Big Data and Business Intelligence are becoming an increasingly important source of statistical information which is used as a vital part of the critical decision-making process of all businesses. Bernard Marr, in his article titled…

Details

Overcome Data Testing Challenges-iCEDQ

Overcome Data Testing Challenges

Today all decisions in an organization are being made on the data available to them. Hence it has become critical to ensure that the does is free of any issues or defects. The way to ensure that there are no data issues and it is fit for the business, is to test, validate and compare…

Details

AML Software Implementation & Production Monitoring with iCEDQ DataOps Platform-iCEDQ

AML Software Implementation & Production Monitoring with iCEDQ DataOps Platform

iCEDQ accelerates AML software implementation and prevents false positive signals in AML operations. AML software is a downstream system that consumes data from multiple sources. AML software analyzes data based on compliance models.  This results in suspicious activity reports. Further, they also monitor data for regulations such as FATCA, trade restrictions, sanctions, and watch list. However, if…

Details

iCEDQ your Gatekeeper for Data Issues-iCEDQ

iCEDQ your Gatekeeper for Data Issues

The Challenges: Today’s organizations have thousands of data integration (ETL) processes constantly moving silos of data from various operational and/or external data sources to downstream applications. Since the downstream system doesn’t have control over incoming data or the process, it can cause serious data issues due to: The quality of the data depends on the upstream…

Details

What Are The Challenges Of A Data Factory-iCEDQ

What Are The Challenges Of A Data Factory

DataOps Platform for Integrated Data Testing & Production Monitoring DataOps is a set of practices and tools used by Big Data teams to increase velocity, reliability, and quality of data analytics. It emphasizes communication, collaboration, integration, automation, measurement and cooperation between data scientists, analysts, data/ETL (extract, transform, load) engineers, information technology (IT), and quality assurance/governance.…

Details

Auditing -The Missing Element in Data Management and Data Governance-iCEDQ

Auditing -The Missing Element in Data Management and Data Governance

Perhaps the most astonishing fact, however, is that IT has been blind for so long to the need for monitoring and metering (Auditing) for data health, and yet this fundamental engineering concept.  For instance, Figure 1 illustrates a centrifugal steam engine governor. Figure 1: The Centrifugal Steam Engine Governor This device, invented by James Watt,…

Details

QA Challenges in Data Integration Projects-iCEDQ

QA Challenges in Data Integration Projects

Download Whitepaper from here QA Challenges Data Integration Projects Quality Assurance (QA) is a very important component of any data-centric application project.  Projects such as data warehouse, data migration, ETL, Data Lakes and MDM are no exception. The Majority of these projects are the multi-year and multi-million dollar in nature due to the amount of work…

Details

iCEDQ Agile Way

Agile Data Warehouse Testing & Data Migration Testing

Automate data warehouse etl testing and migration testing the agile way from Sandesh Gawande   Development of a data warehouse, ETL, data migration or conversion always faces an ever-decreasing timeline. These implementations can take years to complete and users are not ready to wait that long. The waterfall development model has been discarded in favor…

Details

Data Migration Testing Techniques to Migrate Data Successfully-iCEDQ

Data Migration Testing Techniques to Migrate Data Successfully

We discussed the potential risks involved with the data migration process in our last iCEDQ insight. As previously mentioned, data migration is an important process where data from one system is transferred to a new, target system. The threat of data loss, data corruption, extended downtime, and application crashes make the data migration process risky.…

Details

The Data Migration Process & the Potential Risks-iCEDQ

The Data Migration Process & the Potential Risks

What is Data Migration? Data migration is the process of transferring data from one system to another system, known as the target system, using a variety of tools and techniques. Below are the different types of migrations which are encountered in different enterprises: Database Migration: This involves moving from one database software to another. E.g.…

Details

3 Reason To Perform ETL Testing - iCEDQ

3 Reasons Why You Need to Perform ETL Testing

An ETL process is at the heart of any data-centric system/ project be it Data Migration or Data Warehouse. All the data movement, transformation, and conversions are done by the ETL process in order to ensure that all the data is uniform in terms of quantity, quality, and format. So, why is ETL testing so…

Details

ETL Development & ETL Testing – a Pipeline for Data Warehouse Testing

ETL Development & ETL Testing – a Pipeline for Data Warehouse Testing

ETL is a process of extracting (E), transforming (T) and loading (L) the data into the data warehouse or any other data-centric system. The process involved in developing these ETL processes is time consuming and usually follows the steps below: 1. Understand the Business Requirement 2. Create Technical Specification/ Requirement 3. Generate Mapping Document 4.…

Details

iCEDQ vs Other Tool

iCEDQ Platform vs Data Quality Tools

When people hear about iCEDQ, the first reaction is… “Hey, I already have a data quality tool! Why do I need iCEDQ?” In this article let’s contrast iCEDQ with any other data quality tools available in the market. What is iCEDQ? iCEDQ is a Quality Assurance and Test Automation platform for data-centric projects and processes such as…

Details

ETL-Testing-iCEDQ

How to test an ETL Process – Basic Concept

The basic concept of ETL Testing and Data Warehouse Testing The answer lies in the understanding of an ETL process. An ETL process at its core reads data, applies a transformation on it and then loads the data. This can be represented by the following simplistic equation. Input Data + Transformation = Output Data The…

Details

iCEDQ - ETL Testing Vs. Application

ETL Testing Vs. Application Testing – The Fundamental Difference

At the core, quality is the measurement of deviation between what is expected vs. actual. In quality assurance practice, we implement a set of tests to measure this deviation. The extent of the deviation indicates the quality of the software. In any application testing, there are three common terms that we will notice: Requirements Test…

Details

etl-testing-and-data-quality-governance-software-iCEDQ

ETL Testing and Data Quality Governance Software – The Missing Link

Data has become critical to the business. Hence, enterprises are investing time, money and resources in data-centric systems such as data warehouse, MDM, CRM & migration projects. However, all research done by independent agencies indicates that There is such a high failure/delays in implementations of data-centric projects Users still don’t trust data coming from data…

Details