Cloud-Oriented Data Integration Requires Technology not Mythology

Survey after survey shows that data integration is an issue with the majority of enterprises, despite the fact that best practices are understood, and mature data integration technology is available and cost effective.  This is particularly the case as it relates to cloud-based applications.  Because the current state of data integration severely impacts businesses, those charged with enterprise IT are becoming more aware of data integration as strategic technology.

Over half of the respondents in a survey commissioned by Oracle admitted they had missed deadlines due to poor integration between disparate cloud applications.  Indeed, 64% found themselves unable to integrate cloud and enterprise applications, and actually leveraged manual processes as “back-up” measures to work around the lack of a data integration strategy and data integration technology.

This is a transitional time for enterprise IT.  The cloud has decentralized infrastructures, including data storage and compute.  However, as these islands of data become more distributed, there is not parallel effort to solve the larger data integration issues.  This causes the investment in cloud computing to fall short in many instances, and leads enterprises to actually take a step back in their move to cloud-based platforms, in terms of productivity.

The downsides of not having a data integration strategy and technology working within cloud deployments include:

  • Data quality and synchronization issues that lead end users to distrust or not accept applications that exist within public or private clouds.
  • The operational cost of supporting cloud-based platforms is much higher than expected because data integration issues require more time and attention from the staff.
  • The use of ad-hoc data integration approaches, such as shipping data on USB drives between on-site systems and public cloud-based data storage, is both costly and hugely problematic.
  • Reactive data integration implementations are typically not as well thought out, thus not as effective, as those where more advanced planning around data integration occurred.

A global perspective of corporate data is still critical to preserving a shared knowledge within the enterprise.  The use of cloud computing, and other emerging technology such as Big Data, is changing the game.  IT managers must seize the opportunity to lead through the creation and management of a strategic data integration strategy, both considering the movement to cloud-based platforms and the consolidation of data within Big Data systems.

I’m out there consulting with the many Global 2000 companies.  A few common problems frustrate me around the use of data integration.  They can be best described as myths that have gone unchallenged for years, and thus many in enterprise IT now consider them facts.  Let’s take some time to dispel the myths.

Myth:   If I migrate my core business applications and data to the cloud, there is no need to integrate that data since it all resides in clouds.

Data integration needs to occur inter- or intra-platform.  Just collocating your systems and data on the same public or private cloud does not provide data integration.  The same challenges exist as if the systems were widely distributed and heterogeneous.

Myth:  Batch-oriented data integration is typically the way to go when supporting applications and data stores that exist both locally, as well as on public clouds.

This is the way to go at times, depending upon the requirements of the business.  However, in most of the cases that I see, event-driven and real-time data integration is typically the best approach to meet the needs of the business.  This includes the ability to see data as it happens, no matter if it’s local to your business applications, or if it’s consumed through data integration technology.

Myth:   Data security, master data management (MDM), and data governance are not as important when leveraging cloud-based platforms to host data.

While you would think the opposite assumptions would be true, given the paranoia around the use of cloud-based systems, many enterprises that deploy data and applications to public clouds often overlook the need for a sound data security, MDM, and data governance plan and technology solution.  Perhaps iit’s because they were not deployed within the traditional environments, and thus there is no precedence or experience.  Just to be clearly, these approaches and technology are almost always a requirement.

No matter where you are in your cloud deployments, the need to create and leverage a sound data integration strategy is always on the critical path.  Also, the use of key strategic data integration technology is always a step that you need take.  Follow this advice, and you’ll have the plan, the best practices, and the technology in place to drive data integration between systems that are becoming both more numerous, as well as distributed.  This is just the outcome of the use of public and private cloud based systems.

About David Linthicum

Dave Linthicum is the CTO of Cloud Technology Partners, and an internationally known cloud computing and SOA expert. He is a sought-after consultant, speaker, and blogger. In his career, Dave has formed or enhanced many of the ideas behind modern distributed computing including EAI, B2B Application Integration, and SOA, approaches and technologies in wide use today. For the last 10 years, he has focused on the technology and strategies around cloud computing, including working with several cloud computing startups. His industry experience includes tenure as CTO and CEO of several successful software and cloud computing companies, and upper-level management positions in Fortune 500 companies. In addition, he was an associate professor of computer science for eight years, and continues to lecture at major technical colleges and universities, including University of Virginia and Arizona State University. He keynotes at many leading technology conferences, and has several well-read columns and blogs. Linthicum has authored 10 books, including the ground-breaking "Enterprise Application Integration" and "B2B Application Integration."

View all posts by David Linthicum →

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>