What is Data Migration?
The term data migration is used to describe the movement of data from its stored location to a new location. The location can be from one storage location to another or between application programs. Data migration can also involve a change of format or a change of data type.
Data Migration for Storage Management
Storage management in a data center or cloud needs to be managed to control costs and adhere to regulatory compliance. High-speed storage comes with a premium price tag, so storage management policies should ensure that older or less frequently needed data is regularly archived in more commodity-priced storage. A business may only need 24 months of historical data in its data warehouse for day-to-day operations, after which it can be archived in low-cost storage or a data lake.
Data Migration for Data Protection
A typical use case is to create copies for disaster recovery. In a cloud context, this means shipping data to a secondary location that is not impacted by power failures or natural disasters that may impact the primary site. To ensure applications don’t pay a performance penalty for making remote copies, data is migrated asynchronously by creating a local change log that is pushed to a remote system.
Data Migration for Offloading Analytic Processing
Operational systems generate data that is essential for supporting business decision-making. Transactional systems are performance sensitive and are streamlined to optimize performance for customer interactions. The transaction logs these systems create can be used by data migration solutions to populate data warehouses and data marts to support data analysis and visualization.
Actian transactional databases work with change data capture solutions such as HVR to automate data migration for analytics.
Data Migration Steps
Before beginning, it is important to scope the effort and ensure you have a strong business case for the migration. As timelines often slip, your project hits roadblocks, and costs rise, set milestones to assess progress and descope as needed. The following are some general steps in planning and performing a data migration.
Scoping
- Assess the scope of the data to be migrated.
- Ensure the data is complete and you understand every field to be moved from the source to the target system.
- Understand how applications are accessing the data you want to migrate.
Migration Design
- Map every source data field to the target.
- Understand the format and data type to be moved or transformed.
- Ensure referential integrity is maintained from source to target data set.
Migration Strategy
- Decide if this will be a big bang migration or a less risky phased migration of subsets of the source data.
- Decide if you will lift and shift your data – this is a great time to improve before you move.
- Agree on the test strategy to check the integrity of the data.
- Running both systems in parallel is a sound strategy for catching data divergence.
Testing
- Assess the scope of the data to be migrated.
- Ensure the data is complete and you understand every field to be moved from the source to the target system.
- Understand how applications are accessing the data you want to migrate.
Switchover
- Map every source data field to the target.
- Understand the format and data type to be moved or transformed.
- Ensure referential integrity is maintained from source to target data set.
Post-Migration Audit
- Decide if this will be a big bang migration or a less risky phased migration of subsets of the source data.
- Decide if you will lift and shift your data – this is a great time to improve before you move.
- Agree on the test strategy to check the integrity of the data.
- Running both systems in parallel is a sound strategy for catching data divergence.
Cloud Migration
As part of a digital transformation initiative, businesses are moving to the cloud to gain new levels of agility, manageability, elastic scalability, and reduced operational costs. Major cloud providers, including Amazon, Microsoft, and Google, provide tools to ease data movement from on-premise systems to the cloud. Moving data off a public cloud platform can incur egress charges. Many businesses employ a pragmatic multi-cloud strategy, maintaining data pools in different clouds and deploying applications, including analytics close to the data.
Transforming Data During Migration
Data formats do vary by application. As applications age or evolve, the data formats they use also change. Y2K is a classic example of a forced data migration for aging applications to accommodate the century change. Many applications have evolved to use modern web-service-oriented data formats such as JSON, for example, which involves a transformation of the format from fixed to a more flexible delimited variable-length character string format. Software vendors often use proprietary file formats, making data migration a non-trivial activity.
Actian Data Migration Capabilities
Actian eases migration to the Actian Data Platform with migration tools and services.
Below are some considerations for your next managed database cloud service:
- Provides transparent management of database software.
- Ensures undisrupted operations with consistently high performance.
- Database instances can be spun up with ease.
- Supports your data security policies.
- Allows elastic compute resources to match your database usage.
- Provides resilience using dynamic multi-region failover and disaster recovery.
You can learn more about data migration by reading our article Database Migration to the Cloud: Things You Should Consider and the migration capabilities of the Actian Data Platform.