Salesforce Migration Testing with iceDQ

The acquisition of a large Brokerage Firm by one of the largest Bank in US, involved migrating millions of customers to a single Salesforce instance. Ensuring proper testing and certification was paramount for business continuity, regulatory compliance, and seamless customer experience.

iceDQ, an automated data testing platform, played a key role in the data migration testing. It had the native data connectivity and reconciliation capabilities to compare data across different environments of Salesforce, SQL Server, Data Lake, and Snowflake. It enabled automated data migration testing by not only performing complex checks but also ensured data accuracy at scale.

Automating with iceDQ led to impressive results:

In recent years, a large Brokerage Firm, with over 5 million customer accounts merged with one of the largest Bank in the US. Post merger the bank identified Salesforce CRM as one of the many redundant systems. It was decided to sunset one of the Salesforce systems and migrate to a single instance. It consisted of 5 million customer records with IDs, communications, and other essential data.

A major part of the project was sunsetting and migration of Salesforce instance, however there were additional projects that involved archiving legacy SF data and re-testing the impacted downstream systems.

Testing was required to reconcile the migrated data across the various target environments, ensuring data consistency and integrity. The migration spanned multiple platforms, including the Firm’s Salesforce instance, Bank’s Salesforce instance, and various target environments Bank’s SQL database, Hive Data Lake, and Snowflake cloud.

iceDQ’s data reconciliation testing scope:

A1. Salesforce vs. Salesforce reconciliation.
B1. Salesforce vs. SQL Server database reconciliation.
C1. Salesforce vs. Data Lake (Hive) reconciliation.
D1. Data Lake (Hive) recon vs Snowflake reconciliation.

The project had additional data testing requirements such as:

1. The tool should offer native connectivity to Salesforce.
2. The data testing tool should be able to connect and reconcile across systems and cloud environments.
3. It should generate proof of testing necessary for regulatory compliance.
4. The tool should scale with the increasing data volume without creating performance and stability issues.
5. For test automation it should support low-code / no-code by automating the test creation, execution, and integration with test management tools such as JIRA.

iceDQ offers native support for Salesforce SOQL, allowing the testing team to write Salesforce’s custom query language within iceDQ. This enabled efficient and accurate testing by leveraging the team’s existing Salesforce expertise. The project involved migrating, testing, and certification of the following Salesforce objects:

For the above SF objects, iceDQ was responsible for automating testing and verifying:

iceDQ simplified data reconciliation by offering native Salesforce support. As illustrated in Figure 4, this functionality empowers users to leverage their existing Salesforce knowledge.

This user-friendly approach streamlines the process, allowing you to quickly query Salesforce and conduct reconciliation tests tailored to your specific business needs.

To maintain data alignment post-migration, Bank’s Salesforce instance synchronizes key objects with the Bank Data Lake (HIVE). This includes user information, customer contacts, tasks, and account information cards, ensuring both environments reflect the latest data. Figure 6 illustrates this process:

Comparison of Salesforce data in the Bank’s Data Lake (HIVE) with the Snowflake environment to ensure data consistency. Figure 7 illustrates this process:

Data Reconciliation Across Systems and Cloud: To achieve seamless integration, iceDQ integrated with various data sources and destinations through its built-in connectors. These include source Salesforce instances, target environments like Bank SQL, Data Lake, and Snowflake, and existing team interfaces.

Data Volume Testing: The migration involved a massive volume of data, including millions of customer records, communications, and other entities. Full volume testing was crucial to validate the accuracy and completeness of the migrated data.

Proof of Testing & Regulatory Compliance: For highly regulated financial firms, ensuring regulatory compliance and audit readiness is paramount during major data migrations. iceDQ successfully documented the testing activities:

Data Test Automation: Leveraging iceDQ’s user-friendly interface, customizable data checks and automated testing ensured accurate data migration and streamlined workflows. Let’s delve deeper into these benefits:

In an M&A, the efficient integration of acquired entities unlocks faster synergy realization, giving you a competitive edge. By leveraging iceDQ’s automated data migration testing, the bank was able to:

Achievements and Gains:

iceDQ empowers organizations to ensure data trust and reliability throughout the data life cycle.

Our comprehensive platform combines data testing, data monitoring, and data observability into a single solution, enabling data engineers to proactively manage data quality and eliminate data issues before they impact business decisions.

Leading companies across industries, including prominent players in banking, insurance, and healthcare, rely on iceDQ to continuously test, monitor, and observe their data-driven systems. This ensures trustworthy data that fuels informed decision-making and drives business success.