One question we commonly get from our clients using Salesforce.com is “how much testing is needed for our data migration effort”. This can be different in every situation and is dependent on several factors. Compliance, industry regulations, number of users and business risk play a significant role in determining the amount of testing needed for the data migration between corporate information systems.
The importance of the data within the systems is usually well known by the executive team but the cost associated with data clean-up from a incomplete data migration is not. As part of the implementation process, the new systems will be populated with legacy data and the compliance and business risks associated with bad data can be very expensive.
Organizations can incur significant expenses if their Saleforce data is loaded incorrectly and then end users begin transacting on the data. Now the job of clean-up and reloading becomes much larger as there is activity history, tasks, notes and attachments associated to the newly migrated records.
This article offers thoughts and recommendations on how to create a more robust and consistent data migration testing strategy.
Cirrius Solutions has tested numerous data and content migrations primarily in the healthcare, manufacturing, retail and mass media industries. The information presented here includes some of the lessons learned from our clients’ quality requirements and the testing of migrations with hundreds of thousands of fields and terabytes of content.
The recommended approach to designing migration testing strategies is to document the acceptable level of risk, the likelihood of occurrence, and then define the means to mitigate risk via testing where appropriate. The identification of risks a thoughtful process and will be specific to the system being migrated.
Testing Methodology for Data Migrations
A common approach to testing data and content migrations relies upon data sampling, where some subset of random data or content is selected and inspected to ensure the migration was completed “as designed”. Those that have tested migrations using this approach are familiar with the typical iterative test, debug and retest method, where subsequent executions of the testing process reveal different error conditions as new samples are reviewed.
This process works but is reliant upon an acceptable level of error and an assumption pertaining to repeatability. Normally we will focus on repeatability and perform three or more iterations of testing and data loads until the client can confirm that we have achieved a reasonable sample.
Salesforce Sandbox Testing
These tests occur early in the migration process, before any migration, even migration for testing purposes, is completed. The pre-migration testing options include:
- Verify scope of source systems and data with the business users. Verification should include data to be included from all business areas.
- Define the source to target and high-level mappings for each category of data and/or content as needed in the destination system.
- Verify destination system data requirements such as the field names, field type, mandatory fields, valid value lists and other field-level validation checks.
- Verify any data transformations that will be needed. There a many instances where the data in the source does not exactly match the target. In this case Salesforce data transformation tables are used to make the needed data conversions.
- Validate the source and target system connections from the migration platform.
- Test tool configuration against the migration specification to verify that a migration specification’s mappings are complete and accurate.
Functional Testing Areas to Consider
1. Unit/System Testing:
Security Model Review: Validate that users have been added to the appropriate profiles with the necessary role(s), sharing rules and field level security.
Object/Field Level Validation: Validate that all objects and fields have been correctly setup for the data loading to begin.
- Run small loads of accounts, contacts, leads, products opportunities, etc into stage to be sure the objects, fields and data mapping has been implemented correctly before we run large data loads.
Load Count Validation: Validate that exported record counts equal imported record counts
- If 50,000 contacts were exported from the legacy system we should have 50,000 contacts added to SFDC (subtract duplicates). The data load team can provide this information. Reports can also be created for this validation.
Error Handling Review: Validate that any errors during the migration are identified, investigated, and their disposition agreed. – Business Users May be Required.
- Ensure adequate (verbose) Error Logging supports error investigation
- Errors can be Informational (This is what happened but no need to worry)
- Warning (these warning messages were generated)
- Fail (the data failed to migrate)
2. Functional/Integration Testing:
Data Integrity Validation: Validate that contacts and opportunities have been associated with the correct accounts.
- Validate that that all duplicates and invalid data have been eliminated.
Workflow Validation: Validate that all new workflow, assignment, approval rules and email notifications are working correctly.
ETL Load Validation: Validate that the information within the new ETL feeds is able to be loaded correctly into Salesforce.com.
- This includes field level validation to be sure data us not truncated and all data types align.
- Need to evaluate the full cycle of data between the systems to be sure duplicates are not created on subsequent cycles. This is the Add, Update and Delete scenarios that can occur in either system.
3. Performance Testing:
Load Performance Validation: Monitor the data load times and raise issues to the team as needed. If needed we will consider a system outage during the final go-live to ensure a successful load process.
4. End to End Testing:
ETL System Load Validation: Validate that the entire process for loading of the ongoing feeds and keeping all data in sync between the “external systems” to SFDC is working correctly.
5. User Acceptance Testing:
UAT Security Validation: Business to validate that all users have been added to the appropriate profiles with the necessary role(s), sharing rules and field level security.
UAT Functional Validation: Business users from Sales and Marketing to log into the testing environment to validate that all of their accounts and contacts are now in SFDC.
- This includes validation of account, contacts, products, pricing, contracts, subscription history, mail templates, custom reports, etc.
UAT Quality Validation: Business users to validate that no duplicates and only valid data has been migrated to Salesforce.com. User acceptance testing provides an opportunity for the user community to interact with legacy data in the destination system prior to production release, and most often, this is the first such opportunity for the users. Attention should be given to reporting, downstream feeds, and other system processes that rely on migrated data.
6. Production Validation:
Production Go-Live Validation: Final validation in production to verify that all configuration was and data-loads were deployed successfully. All of the testing completed prior to the production migration does not guarantee that the production process will be completed without error.
Planning your Data Migration Testing Strategy
In the context of data and content migrations, business risks are a direct result of migration errors. A thorough testing strategy minimizes the likelihood of data and content migration errors.
- Establish a comprehensive migration team, including functional business users, IT and executive stakeholders. Verify the appropriate level of experience for each team member and train as required on data migration principles, the source and the destination system.
- Analyze business risks with the specific systems being migrated. These risks should become the basis for the data migration testing strategy and include legal, environmental and expense related risks.
- Create, formally review and manage a complete migration specification. This is critical to getting all teams on the same page, setting expectations and outlining roles & responsibilities.
- Verify the scope of the migration with the business users and IT teams. Understand that the scope of the migration may be adjusted over time.
- Identify likely sources of migration error and define specific testing strategies to identify and remediate these errors. This is where multiple test loads can be very helpful! .
- Use the field-level source to destination mappings to establish data requirements for the source system. Use these data requirements to complete pre-migration testing.
- Complete an appropriate level of Sandbox testing. For migrations where errors need to be minimized, 100% verification using an automated tool is recommended.
- Look closely at the ROI of your testing effort. It may not make business sense to test every possible scenario. However, you should establish a post go-live process for continued maintenance.
- Perform User Acceptance Testing with migrated data. This approach tends to identify application errors with data that has been migrated as designed. The end users know how the data will be used in day to day operations.
- Perform a final production validation. Most large migrations will occur over the weekend so that the company can do a production checkout to ensure all data has been migrated correctly.
- Build a rollback strategy! Even if all of the right testing and validation steps are followed you can still experience issues in the final migration. If critical issues are discovered, you will need to have a backup and rollback strategy in place to bring the system back to a pre-migration state. You can always fix the issue(s) and take another run at it the following weekend.
For more information contact your Minneapolis Based Salesforce consulting partner at info@cirriussolutions.com.
We have consultants across the US to assist with your integration needs. Chicago Illinois, Austin, Houston, Dallas & Fort Worth Texas, Philadelphia PA, Phoenix AR, New York NY, Portland OR, Milwaukee WI, Las Vegas NV, Atlanta GA, Raleigh NE, Omaha Nebraska, Cleveland Ohio, San Jose, San Francisco,San Diego & Los Angeles CA
