How to Effectively Prepare Your Data for Gen AI
Actian Corporation
March 20, 2024
Many organizations are prioritizing the deployment of generative AI for a number of mission-critical use cases. This isn’t surprising. Everyone seems to be talking about Gen AI, with some companies now moving forward with various applications.
While company leaders may be ready to unleash the power of Gen AI, their data may not be as ready. That’s because a lack of proper data preparation is setting up many organizations for costly and time-consuming setbacks.
However, when approached correctly, proper data prep can help accelerate and enhance Gen AI deployments. That’s why preparing data for Gen AI is essential, just like for other analytics, to avoid the “garbage in, garbage out” principle and to prevent skewed results.
As Actian shared in our presentation at the recent Gartner Data & Analytics Summit, there are both promises and pitfalls when it comes to Gen AI. That’s why you need to be skeptical about the hype and make sure your data is ready to deliver the Gen AI results you’re expecting.
Data Prep is Step One
We noted in our recent news release that comprehensive data preparation is the key to ensuring generative AI applications can do their job effectively and deliver trustworthy results. This is supported by the Gartner “Hype Cycle for Artificial Intelligence, 2023” that says, “Quality data is crucial for generative AI to perform well on specific tasks.”
In addition, Gartner explains that “Many enterprises attempt to tackle AI without considering AI-specific data management issues. The importance of data management in AI is often underestimated, so data management solutions are now being adjusted for AI needs.”
A lack of adequately prepared data is certainly not a new issue. For example, 70% of digital transformation projects fail because of hidden challenges that organizations haven’t thought through, according to McKinsey. This is proving true for Gen AI too—there are a range of challenges many organizations are not thinking about in their rush to deploy a Gen AI solution. One challenge is data quality, which must be addressed before making data available for Gen AI use cases.
What a New Survey Reveals About Gen AI Readiness
To gain insights into companies’ readiness for Gen AI, Actian commissioned research that surveyed 550 organizations in seven countries—70% of respondents were director level or higher. The survey found that Gen AI is being increasingly used for mission-critical use cases:
- 44% of survey respondents are implementing Gen AI applications today.
- 24% are just starting and will be implementing it soon.
- 30% are in the planning or consideration stage.
The majority of respondents trust Gen AI outcomes:
- 75% say they have a good deal or high degree of trust in the outcomes.
- 5% say they do not have very much or not much trust in them.
It’s important to note that 75% of those who trust Gen AI outcomes developed that trust based on their use of other Gen AI solutions such as ChatGPT rather than their own deployments. This level of undeserved trust has the potential to lead to problems because users do not fully understand the risk that poor data quality poses to Gen AI outcomes in business.
It’s one issue if ChatGPT makes a typo. It’s quite another issue if business users are turning to Gen AI to write code, audit financial reports, create designs for physical products, or deliver after-visit summaries for patients—these high value use cases do not have a margin for error. It’s not surprising, therefore, that our survey found that 87% of respondents agree that data prep is very or extremely important to Gen AI outcomes.
Use Our Checklist to Ensure Data Readiness
While organizations may have a high degree of confidence in Gen AI, the reality is that their data may not be as ready as they think. As Deloitte notes in “The State of Generative AI in the Enterprise,” organizations may become less confident over time as they gain experience with the larger challenges of deploying generative AI at scale. “In other words, the more they know, the more they might realize how much they don’t know,” according to Deloitte.
This could be why only four percent of people in charge of data readiness say they were ready for Gen AI, according to Gartner’s “We Shape AI, AI Shapes Us: 2023 IT Symposium/Xpo Keynote Insights.” At Actian, we realize there’s a lot of competitive pressure to implement Gen AI now, which can prompt organizations to launch it without thinking through data and approaches carefully.
In our experience at Actian, there are many hidden risks related to navigating and achieving desired outcomes for Gen AI. Addressing these risks requires you to:
- Ensure data quality and cleanliness
- Monitor the accuracy of training data and machine learning optimization
- Identify shifting data sets along with changing use case and business requirements over time
- Map and integrate data from outside sources, and bring in unstructured data
- Maintain compliance with privacy laws and security issues
- Address the human learning curve
Actian can help your organization get your data ready to optimize Gen AI outcomes. We have a “Gen AI Data Readiness Checklist” that includes the results of our survey and also a strategic checklist to get your data prepped. You can also contact us and then our experts will help you find the fastest path to the Gen AI deployment that’s right for your business.
