As Ralph Waldo Emerson once said, “it’s not the destination it’s the journey”. Unfortunately, this wise saying is not nearly applied as often as it should be to the contemporary topic of data warehouse modernization project design and execution. The result, thousands of data warehouse modernization projects unnecessarily end up in failure. Gartner’s Adam Ronthal, suggests that over 60% of all database migrations fail. That is, a migration is started, then starts slipping, overruns the original budget by multiples, until finally management pulls the plug. Time is lost, money wasted, and careers wrecked. How do you make sure yours is not added to this dubious and growing list?
There are some key steps to designing the optimal journey:
- Migrate or Offload: A first step that many organizations take that mitigates risk and accelerates value delivery is the off-load journey. In this situation several demanding workloads are identified to offload in a phased manner rather than migrate the entire data warehouse. Unless an organization needs to completely get off its existing data warehouse, e.g. your vendor has announced EOL plans as is happening with Netezza today, taking a phased approach with your migration usually makes sense and lets you get started sooner. Great candidates may include workloads with these characteristics: large data sets, ad-hoc query tool diversity or requests for new unsupported tools, hybrid data from multiple diverse data sources and complex queries.
- Baby step or giant leap to the cloud: Some organizations, as part of a cloud-first strategy move all of their data warehouse to the cloud. In other cases, pragmatic organizations often chose to conduct their migrations in stages. For example, moving a data landing and staging area to the cloud provides useful elasticity and agility while reducing the risk of disruption.
- Automate, Automate, Automate: Leading solution providers now offer sophisticated migration assessment tools that can identify SQL queries that can be translated to the new target system automatically. Typically, data warehouse systems that fully support the entire current SQL industry standard will do better at supporting automation of query migration. It is not untypical for industry standard SQL systems to support 95% and higher levels of automatic conversion from the source to the target system.
- Replicate, Augment or Breakthrough: Most organizations take a 3-step approach to the migration: first replicate the source system report generation, then utilizing the upgraded performance of the target system augment the query base with additional dashboards and other ad-hoc analytics. Finally, in the third step, forward-thinking organizations look to develop new composable analytics applications such as real-time offers and fraud analysis that they were prohibited for cost and/or performance reasons from considering before.
- Think “Business” with a Big “B”: A successful data migration project is always dependent on understanding and addressing the current and future needs of the business. This requires proactive collaboration with the direct and indirect users of the insight that you hope to deliver with your modernized data warehouse project. Especially important will be to identify and prioritize discovery-oriented analytics including ad-hoc analysis that the business side seeks and values.
How to get started?
Your first step is to seek out and short-list a set of next generation solution vendors that can offer you a truly hybrid data warehouse journey that runs both in the cloud and on-premises with zero changes to your query stream and easily reroutes your ETL connections to your source data warehouse apps and external data sources, supports multiple cloud platforms to eliminate lock-in, and excels at workloads that demand high levels of scale and concurrency.
Solutions such as Actian’s Avalanche cloud data platform designed to run seamlessly in the cloud and on-premises represent a potential breakthrough worth checking out. As the above diagram conveys an organization can charter its modernization journey through multiple paths with different combinations of cloud and on-premises deployments. A glass of hybrid with your slice of cloud data warehouse computing sir? Drink up – It’s a hybrid world we live in, today and tomorrow.
Learn modernization best practices from industry experts and insiders
If you are thinking about modernizing your enterprise data warehouse, watch our on-demand webinars featuring leading industry analysts and former executives from Teradata and Netezza.
Rethinking Teradata Migration: 7 real-world secrets to success, featuring Raghu Chakravarthi, SVP of R&D at Actian (former Head of Big Data at Teradata)
Top 7 tips for a successful migration from Netezza, featuring Paul Wolmering, VP of Sales Engineering at Actian (former Director of Tech Services at Netezza)