Why AI Alone isn’t Good Enough for Business Analytics

AI-driven analytics are a tempting prospect: a few keystrokes and your questions are answered, hassle-free.

AI (specifically, LLMs like Claude, ChatGPT, and Gemini) is great at generating answers based on the internet. And your business has much less data than the internet does. So set up some MCP servers, hook up your LLM to your data, and it might feel like you’re in business.

Unfortunately, you’re not.

There are a host of specific challenges standing between you and success with AI-driven business analytics. These challenges are very real, but not insurmountable. Here’s what they are and how you can mitigate them.

Business Analytics Can’t Rely on General Knowledge

Ask an ungoverned AI to analyze “churn rate,” and there’s a non-zero chance that its answer is about butter production instead of customer retention.

And even if the AI model does go with the same definition of “churn rate” that you mean, there are still layers and layers of meaning that need to be defined:

• Metric definitions: How does your business, specifically, calculate “churn rate”?
• Business definitions: If churn rate is based on active customers, are free trial users considered customers? Is an individual seat within an organizational license a customer, or is the organization the customer? Is “active” determined by the customer’s account status or by their product usage?
• Data relationships: Which data source is storing the customer information? How should those customer records be matched up with the product activity records? 
• Organizational rules: Who is responsible for defining a term or maintaining a dataset? 
• Access permissions: Are you even authorized to know the company’s churn rate?

Each of these questions can have a different but completely valid answer depending on the organization.

There’s no way to assure that your LLM will infer your company’s specific correct answer to every single one of these questions, based on general training from the internet at large. And there’s no way to tell what inferences your LLM did make. The precision and transparency that business analytics need simply isn’t there.

Data Access Alone isn’t Enough

If “general training from the internet at large” doesn’t provide enough context to the LLM, will connecting it to your own data get it there?

Nope. Your raw data doesn’t have enough context for your LLM to infer the right answers.

Ambiguity

When you find yourself asking where and why and how, there’s ambiguity in your organization’s data.

It shows up in all kinds of ways.

For instance, metric calculations might be inconsistent because:

• A metric has no documentation about how it should be calculated.
• A metric has been documented by multiple business units, and the documented calculations are all different.
• A metric is documented in one place, and different business units are pulling the data for the calculation from different places.

Or your organization might have conflicting interpretations of business concepts. Think back to the “are free trial users active customers?” example from earlier: it’s clear how one business unit might say yes, one might say no, they aren’t customers, and another might say no, they aren’t active.

Ambiguity in business concepts can be even more basic: is a reference to “this year” about this calendar year, or this fiscal year?

Simply connecting an LLM to your database isn’t going to clear any of this up.

In the long term, it will actually make things worse: AI is, famously, consistently confident in its responses. That false confidence, taken at face value, will initially mask the uncertain and inconsistent reality of your data and analytics, then undermine trust as decisions based on wrong results fail.

Gartner estimates that within the next two years, 60% of agentic analytics projects that rely solely on MCP will fail due to lack of a consistent layer—in other words, they’ll fail because the AI-to-data connection doesn’t have anything to direct it. 

Incorrect joins

One of the reasons that business analytics is so difficult is that getting the data you need almost always means pulling it from multiple tables, usually across multiple systems.

If only every record had a perfect primary key and every secondary key was comparable across systems. Since we and our data both reside in reality, reliably joining data from multiple sources isn’t nearly so simple.

Instead, every time you bring data together you have to know:

• What’s the correct key for this join?
• What cardinality is the join?
• Is the cardinality directional?
• What type of join is appropriate to the analytics you’re about to do?

Giving an AI agent access to your column names won’t answer these questions.

Security concerns

Data is one of your company’s most valuable assets, and it has security concerns to match. When you give an AI agent access to your data sources, you need to consider a range of factors.

User access control 

Part of AI analytics’ appeal is its potential to significantly expand access to analytics within your company. But a consequence of making analytics more widely available is making your data more widely available, and it’s a rare organization that gives every employee unfettered access to every system.

It’s much more common for employees to have different levels of access to different kinds and sources of data. But unless you’re going to create a separate AI deployment for every different variation of data access—which is a bad idea for a variety of reasons—your AI model is going to have access to data that a given user isn’t authorized to see.

Connecting an AI agent to your data source doesn’t give it any context about what a specific user is and isn’t allowed to access.

Compliance & contractual issues

If you’re subject to GDPR, you need to ensure that your new AI-driven analytics won’t land you in hot water for having an unauthorized subprocessor.

And it’s not just regulations like GDPR, BCBS 239, or the EU AI Act that you need to worry about: there’s also any data storage, use, and retention agreements you’ve made with your customers, partners, and vendors.

Nothing about connecting an AI agent to a data source has the inherent information needed to help you meet your regulatory and legal obligations.

Data sovereignty

Data sovereignty is more than just “what’s the geographical location of the servers my data is on,” though that’s certainly still a part of it. You also need to consider your data’s movement. Plenty of parts of your data ecosystem—including data pipelines, SaaS applications, AI tools, and BI tools—have the potential to make your data available in places other than the source(s) you store it.

Giving an AI agent access to your data sources creates enormous potential for data sovereignty issues, unless you take additional steps.

How to Make AI Usable for Analytics

You can’t rely on an LLM’s general, internet-based training, and connecting it to your data doesn’t solve it. So what else do you have to do to get the AI-driven analytics of your dreams?

The answer is context.

Context about what business terms mean. How metrics should be calculated. Who should access your data, and where, and when.

You can characterize this context in terms of five key requirements:

• Shared business definitions
• Governed metrics
• Defined models and relationships
• Access controls
• Traceability

By building this context and then using it to ground your AI model, you can create an AI analytics solution that actually works.

Shared business definitions and governed metrics

If your AI can’t infer the right calculation, definition, or relationship from the internet at large, the solution is to not let it use the internet at large.

The foundation of a business term definition is a dictionary-style plain language explanation, but that’s only the start. To create real context, you also need to record:

• A list of synonyms
• Hierarchical relationships (if any) to other business terms
• The term’s owner
• Whether the term has been approved by any groups or processes, and if so, when
• The term’s domain
• The criticality of the term
• Places where the term is in use

Similarly, a governed metric is more than a mathematical expression. It also needs to include:

• Specified decimal places for the response
• Units of measurement
• The model that anchors the metric
• Required joins, if any
• Allowed group-by definitions
• Filters, both default and optional
• The time dimension that scopes the metric’s calculation, and the time grains allowed for this scoping

At this point, you might be thinking: is this just describing a data catalog and glossary? The answer is yes. This part of your context won’t just benefit your AI analytics: it can improve all of your data initiatives and communications by giving your entire organization a single shared reference point. 

Defined models and relationships

Once your business terms and metrics are defined, you have to tell your AI how those concepts relate to the actual data in your systems, or you’re only a little better off than you were before.

Comprehensive data models structure data with the same business concepts that your organization uses to measure performance.

It’s critical that your models include explicitly defined relationships. It’s not uncommon for a single entity to have data in multiple source systems, and to avoid all the hazards we discussed in “Incorrect joins,” you need to tell your AI what the correct joins are.

Access controls

Creating an entirely separate AI deployment for each permutation of user access permissions isn’t feasible, but luckily that isn’t necessary.

Instead of giving users unfettered access to your AI model at large, filter users’ access to AI analytics through defined, scoped agents.

A good way to know what agents you need is to mirror the primary business roles and/or analytical functions that you expect your users to have. Scope them by giving each agent permission to access only the data source and columns that are permissible for that role or function.

Now, to customize permissions for specific individuals, you only need to specify which agents they’re allowed to work with—and let the agent configuration work you’ve already done handle the rest of the security lifting.

Traceability

For everything each of your AI agents does.

Summary

As we saw in this blog, all of the problems with AI-driven analytics have solutions. They’re just not solutions that you can find with a generic LLM subscription or even with a few customizations.

Instead, you need an AI analytics solution that offers:

• Shared business context. Codifying human expertise about your business, its operations, and its data ensures that AI answers are based on the reality of your business. It benefits your non-AI data work, too: as you build this semantic layer, you’ll have to resolve internal conflicts about semantic definitions and metric calculations.
• Governed data. Having a catalog of what data you have, how reliable it is, and what it means (beyond “these are dates”) is critical for analytics to be trusted no matter who or what is calculating them. Without access to your governance information, an AI agent can’t produce analytical results you can rely on.
• AI reasoning. Specifying preferred methodologies, data sources to prioritize, and go-to reference materials guides your AI solution to apply your organizational knowledge to your governed data in the way that you intend.

By codifying human expertise and knowledge so that it’s accessible to AI, and by restricting the AI model to use only that reference point, you can benefit from the computational speed and widespread accessibility that an AI analytics solution offers, while also ensuring that the results are something you can use and trust.

AI Analyst has all of these benefits built in from the ground up:

• A semantic layer with governed models, metrics, glossary terms, and relationships.
• Consistent answers grounded in shared business definitions.
• Transparent query generation and explainable analytical workflows.
• Fine-grained access controls and governed data access.
• Conversational analytics, Reports, and Scheduled Insights built on the same business context.

Learn More About AI Analyst

Book a demo or take the product tour to see how Actian AI Analyst delivers consistent answers using conversational analytics and governed business context.

About Author

About Actian Corporation

Actian empowers enterprises to confidently manage and govern data at scale. Organizations trust Actian data management and data intelligence solutions to streamline complex data environments and accelerate the delivery of AI-ready data. Designed to be flexible, Actian solutions integrate seamlessly and perform reliably across on-premises, cloud, and hybrid environments. Learn more about Actian, the data and AI division of HCLSoftware, at actian.com.