The team is usually divided into development, QA, operations and business users. In almost all Data Integration projects, development teams try to build and test ETL processes, reports as fast as possible and throw the code across the wall to the operations teams and business users. However, when the data issues start appearing in production, business users become unhappy. They point fingers at Operations people, who in turn point fingers at QA people. The QA group then puts the blame on the development teams.
AML software is a downstream system that consumes data from multiple sources. AML software analyzes data based on compliance models. This results in suspicious activity reports. Further, they also monitor data for regulations such as FATCA, trade restrictions, sanctions, and watch list.
However, if the incoming data is not good then these results cannot be relied upon. It is basically at the mercy of upstream systems.
iCEDQ is an in-memory data audit rules engine. It sits between the data sources and downstream systems such as ALM. It can validate and reconciliation data coming from multiple data sources. Thus, managing data issues before it affect the downstream system.
DataOps is a set of practices and tools used by Big Data teams to increase velocity, reliability, and quality of data analytics. It emphasizes communication, collaboration, integration, automation, measurement and cooperation between data scientists, analysts, data/ETL (extract, transform, load) engineers, information technology (IT), and quality assurance/governance. It aims to help organizations rapidly produce insight, turn that insight into operational tools, and continuously improve analytic operations and performance.