Data quality is the measure of how well data meets the standards required for its intended use — whether that is making a business decision, generating a regulatory report, training an AI model, or serving a customer.
High-quality data is accurate, complete, consistent, timely, valid, and unique. Poor-quality data costs organizations at least $12.9 million per year on average in operational inefficiencies, bad decisions, failed projects, and compliance penalties, according to Gartner research from 2020.
This guide covers what data quality is, the six dimensions used to measure it, how quality management works, who is responsible for it, how it connects to data governance and AI, and how to build a program that improves it systematically. Throughout, implementation examples reflect how Actian Data Observability and the Actian Data Intelligence Platform approach quality measurement, monitoring, and remediation across hybrid-cloud data estates.
What is Data Quality?
Data quality is the degree to which data is fit for its intended use. A dataset that is high quality for one purpose may be insufficient for another. Customer records accurate enough for billing may not meet the completeness requirements for a predictive churn model. Financial transaction data sufficient for internal reporting may not meet the lineage and accuracy requirements for a regulatory submission.
Quality is not a single attribute — it is a composite of six measurable dimensions, each of which matters differently depending on the use case.
The Six Dimensions of Data Quality
| Dimension | Definition | Example of failure | How it is measured |
|---|---|---|---|
| Accuracy | Data correctly represents the real-world value it describes | A customer’s address is recorded as a previous address after they moved | Comparison against authoritative source systems; error rate per field |
| Completeness | All required fields are populated with non-null, non-empty values | 15% of customer records are missing an email address | Null rate and missing value rate per required field |
| Consistency | The same data value is represented identically across all systems where it appears | “Active” in the CRM and “A” in the billing system both mean active customer, but systems disagree on count | Cross-system comparison of shared fields |
| Timeliness | Data is current and available when needed for its intended use | Yesterday’s inventory data is used to fulfill today’s orders | Data age at time of use; refresh latency vs. SLA |
| Validity | Data conforms to defined formats, ranges, and business rules | A date field contains “13/2026” which is not a valid date format | Conformance rate against defined validation rules |
| Uniqueness | Each real-world entity is represented exactly once with no duplicates | The same customer appears 47 times in the CRM with slightly different name spellings | Duplicate detection rate; entity resolution match rate |
Every data quality program should define acceptable thresholds for each dimension by data domain. A financial transactions dataset might require 99.9% accuracy and 100% completeness on required fields. A marketing contact list might tolerate higher null rates on optional fields. Thresholds defined by domain make quality measurable and certifiable.
Why Data Quality Matters
Decision-making: Every business decision is as reliable as the data behind it. Forecasts built on inaccurate sales data produce wrong projections. Marketing campaigns targeted with incomplete customer data miss segments. Operations optimized on stale inventory data create fulfillment failures. Data quality is not a technical concern — it is a business performance concern.
Regulatory compliance: GDPR, HIPAA, SOX, BCBS 239, and CCPA all carry data accuracy and completeness requirements. BCBS 239 requires banks to demonstrate that risk data is accurate and complete. HIPAA requires that patient records be accurate and current. SOX requires that financial reporting data be reliable and traceable. Compliance failures caused by poor data quality carry penalties, litigation exposure, and reputational damage.
AI and machine learning: AI models learn from training data. Inaccurate, incomplete, or inconsistent training data produces models that make poor predictions. A fraud detection model trained on data with 20% duplicate transactions learns to recognize patterns that do not exist. A recommendation model trained on stale product data surfaces items that are out of stock. Data quality is the foundation of trustworthy AI.
Operational efficiency: Poor data quality generates rework: engineers fixing pipelines that fail because of unexpected nulls, analysts rerunning reports after discovering source errors, and customer service teams correcting billing errors caused by inconsistent records. Gartner research from 2020 puts the average cost of poor data quality at $12.9 million per year. Most of that cost is invisible — it is embedded in the time people spend working around data they do not trust.
Customer experience: Duplicate customer records produce duplicate communications. Inaccurate address data causes failed deliveries. Stale preference data produces irrelevant recommendations. The customer experience downstream of a business is shaped by the data quality upstream.
For continuous monitoring of data health across pipelines, see our guide to data observability.
How Data Quality is Measured
Quality measurement requires defining what good looks like for each domain before measuring against it. The measurement process has four components.
Data profiling: Automated scanning of data assets to assess current quality characteristics: null rates, value distributions, format patterns, duplicate rates, and referential integrity. Profiling establishes the baseline and identifies where quality gaps exist without requiring manual inspection.
Validation rules: Business rules that define what valid data looks like for each field: acceptable value ranges, required formats, referential constraints, and cross-field dependencies. Validation rules run continuously against incoming and stored data and flag records that fail.
Quality scoring: Aggregation of profiling results and validation outcomes into a single quality score per asset. Scores make quality comparable across assets and over time. A dataset with a quality score of 94% is more trustworthy than one scoring 71%, and a dataset whose score has declined from 94% to 87% over two weeks signals a pipeline or source quality issue that needs investigation.
Quality monitoring and alerting: Continuous monitoring of quality scores with automated alerts when scores fall below defined thresholds or when anomalies appear: unexpected row count drops, sudden null rate increases, schema changes that break validation rules. Monitoring catches quality issues before they reach production reports or model training pipelines.
Quality scores are one of the primary evaluation signals in data discovery. Users assessing whether a dataset is appropriate for their use case rely on profiling results and quality dimensions surfaced in the catalog. For definitions of quality and the other concepts that make discovery reliable, see the data discovery glossary.
The Cost of Poor Data Quality
Organizations rarely measure the cost of poor data quality directly, which is why it persists. The costs are distributed across teams and budgets in ways that make the root cause invisible.
| Cost category | How poor data quality generates it |
|---|---|
| Engineering rework | Pipelines fail because of unexpected nulls, format violations, or referential integrity breaks. Engineers spend time debugging and fixing rather than building. |
| Analyst rework | Reports are rerun after quality issues are discovered in source data. Analysts spend time validating data rather than analyzing it. |
| Bad decisions | Strategies based on inaccurate data produce wrong outcomes: missed revenue targets, overstocked inventory, failed product launches. |
| Compliance penalties | GDPR, HIPAA, and SOX violations caused by inaccurate or incomplete data carry direct financial penalties. |
| Customer churn | Duplicate communications, billing errors, and failed deliveries caused by poor data quality damage customer relationships. |
| AI failures | Models trained on poor-quality data make unreliable predictions, requiring costly retraining and delayed deployment. |
| Audit preparation | Manual reconstruction of data quality evidence for audits costs weeks of compliance team time when quality records are not maintained automatically. |
Who is Responsible for Data Quality
Data quality is a shared responsibility distributed across roles with distinct accountabilities.
| Role | Accountability | Day-to-day responsibility |
|---|---|---|
| Data owner | Ultimate business accountability for quality in their domain | Defines quality standards for the domain, approves certification criteria, sponsors stewardship |
| Data steward | Operational quality management within the domain | Monitors quality scores, resolves flagged issues, certifies assets that meet thresholds, manages exceptions |
| Data engineer | Technical quality infrastructure | Builds quality checks into pipelines, implements validation rules, executes technical remediation |
| Data governance lead | Organization-wide quality standards | Defines the quality dimensions and thresholds that apply across domains, tracks program health |
| Data analyst | Responsible use of quality-scored assets | Reports quality issues encountered in analytical work, uses certified assets for production reporting |
| CDO | Executive accountability for data quality posture | Reviews quality metrics, allocates resources to address persistent quality failures, communicates data quality health to leadership |
The most common failure in data quality programs is treating quality as a purely technical responsibility owned by data engineering. Engineers can build quality checks and fix technical issues, but they cannot define what “accurate” means for a business field, certify whether a dataset is fit for a specific business purpose, or resolve cross-domain definitional inconsistencies. Those require business ownership and stewardship accountability.
The Six Stages of Data Quality Management
1. Assess
Profile every data asset in scope to establish baseline quality across the six dimensions. Identify the domains with the highest business risk or regulatory exposure and establish quality thresholds for those domains first.
2. Define standards
Write the quality standards that apply to each priority domain: the acceptable thresholds for each dimension, the validation rules that enforce them, and the certification criteria that make an asset trustworthy enough to use in production reporting and AI training.
3. Monitor continuously
Deploy automated quality monitoring that checks every asset against its defined standards continuously. Configure alerts for threshold violations and anomalies. Connect monitoring to stewardship workflows so issues are routed to the right person automatically rather than sitting in a monitoring dashboard that nobody acts on.
4. Remediate
Resolve quality issues at the source wherever possible. Downstream cleansing — fixing data after it has entered the warehouse — treats symptoms rather than causes and creates ongoing maintenance burden. When source-level remediation is not immediately possible, implement downstream checks that prevent poor-quality data from reaching production reporting or AI pipelines.
5. Certify
Apply certification status to assets that meet defined quality thresholds and have been reviewed by a steward. Certified assets appear with a verified badge in the data catalog. Users trust certified assets without independent validation. Certification is the mechanism that translates quality measurement into user confidence.
6. Improve
Track quality score trends over time by domain. Identify persistent quality failures and address their root causes: source system issues, pipeline logic errors, definitional inconsistencies, or missing validation rules. Report quality health metrics to governance leadership monthly. Programs that cannot show quality improvement over time are not improving.
Choosing Data Quality Software for Hybrid Environments and Live Pipelines
Data quality software ranges from rule engines that check data after it lands to platforms that monitor pipelines continuously and route issues to owners. The right choice depends on where your data lives, how fast it moves, and who has to act when a check fails. Match the software to the job you need done.
What to look for, by need
| If you need to… | Look for | Ask the vendor |
|---|---|---|
| Block bad data before it spreads | Validation rules enforced at ingestion and inside pipelines, with the option to reject or quarantine failing records | Can a check stop a pipeline run, or does it only report afterward? |
| Catch errors in customer data | Profiling, duplicate detection, and format and completeness checks on customer fields | How are duplicates and near-duplicates found, and who reviews merges? |
| Protect live pipelines | Anomaly detection for freshness, volume, schema changes, and distribution shifts | How fast does an alert fire, and who receives it? |
| Catch errors before they reach reports | Quality scores and certification shown in the catalog, linked to lineage for downstream reports | Can I see which dashboards a failing dataset feeds? |
| Reduce rule-writing effort | Automated profiling that proposes validation rules for steward approval | Who approves a suggested rule before it is enforced? |
| Clean existing data | Standardization and matching, with fixes at the source where possible | Does it fix data at the source or only downstream? |
Comparing cost and value
License price is one part of the cost. Compare these items across options:
- Licensing. What the software costs alone, and inside a larger platform.
- Setup. Connector configuration, rule building, and steward time to cover the first priority domains.
- Integration. A standalone checker needs work to connect to your catalog, lineage, and stewardship workflows.
- Growth. How pricing changes as data volume, sources, and users increase.
- Repeat cost. Cleaning tools that only fix data downstream have to run again every time the source repeats the error. Tools that help you fix the cause at the source cost less over time.
Ask every vendor for a three-year estimate that includes implementation and support. For practical steps on cleaning existing data, see How to Clean Up Messy or Inconsistent Company Data.
Judging reviews and proof
Ratings and recommendations help you build a shortlist. They do not show how a tool performs on your data. Before you decide:
- Ask for reference customers with a similar environment and data volume.
- Run a proof of concept on a real pipeline with known problems, and count how many the tool catches.
- Check how quickly an issue reaches the right owner, not only how quickly it is detected.
- Review analyst research as one input alongside your own test.
Data quality in hybrid environments
Quality checks lose value when they cover only part of your estate. For hybrid environments, confirm that the software:
- Connects to your on-premises and cloud sources.
- Applies the same standards, scores, and workflows in both.
- Offers deployment options that fit your security and data residency rules.
- Shows quality and lineage together, so a failing on-premises source is traced to the cloud reports it feeds.
Data Quality by Role: What it Means in Practice
Data analyst: Before quality management: spends 30 to 40 percent of working time finding data, validating it, and confirming with engineers that the version found is the right one. After: searches the catalog, checks the quality score and certification status of candidate assets, and uses certified ones without escalation. Time spent on data validation drops to under 10 percent.
Data engineer: Before: discovers data quality issues when pipelines fail in production. Root cause investigation takes hours. Fixes are reactive and often incomplete. After: quality checks run in the pipeline before data reaches production. Failures route to stewardship workflows automatically. Engineers address issues proactively rather than reactively.
Data steward: Monitors quality scores for every asset in their domain through a single catalog interface. When a score drops below threshold, an alert fires and the issue is assigned. The steward investigates, coordinates the fix with the data engineer, and either resolves the issue or holds the asset out of certified status until resolution.
Compliance officer: Quality certifications and validation histories are maintained as governance records in the data catalog. Audit requests for quality evidence are answered from these records rather than assembled manually. BCBS 239, HIPAA, and SOX audits requiring data quality documentation take hours rather than weeks.
Data scientist: Training datasets carry quality scores, validation histories, and certification status. The data scientist selects certified datasets with quality scores above defined thresholds and can document their quality evidence for model reproducibility and regulatory purposes without additional manual work.
Data Quality and Data Governance
Data quality and data governance are interdependent. Governance defines the quality standards, assigns the accountability for enforcing them, and creates the policies that make quality requirements operational. Data quality management executes those policies through profiling, monitoring, and remediation.
| Governance provides | Data quality management executes |
|---|---|
| Quality standards and thresholds per domain | Continuous monitoring against those standards |
| Stewardship assignments | Operational quality issue resolution and certification |
| Certification criteria | Certification status applied to assets that meet criteria |
| Compliance requirements | Quality evidence maintained for audit purposes |
| Data ownership accountability | Owner-level reporting on domain quality health |
A governance program without quality management produces standards that are never measured. A quality management program without governance produces metrics that nobody acts on because accountability is unclear.
For guidance on data contracts as a quality enforcement mechanism, see our data contracts guide.
Data Quality and AI
Data quality is the most important infrastructure requirement for trustworthy AI. Every data quality problem that exists in training data is inherited by the model that learns from it.
Accuracy failures in training data: A fraud detection model trained on transaction records with 5% inaccurate merchant category codes learns to associate legitimate transactions with fraud patterns. The model flags clean transactions as fraudulent and misses actual fraud.
Completeness failures in training data: A customer churn prediction model trained on records where 20% of the tenure field is null learns to predict churn without one of the strongest predictive signals. Model performance is systematically lower than it should be, and the cause is invisible without lineage back to the training data quality records.
Consistency failures in training data: A recommendation model trained on data from two source systems that define “active customer” differently learns two contradictory patterns simultaneously. Recommendations for customers who match one definition are systematically different from recommendations for customers who match the other, producing inconsistent user experiences that cannot be explained without tracing the training data quality issue.
What AI-ready data quality requires: Every dataset used for AI training must carry a quality certification: profiling results confirming current quality scores, validation history confirming the data met defined standards at the time of training, and a steward sign-off confirming the data is appropriate for the model’s intended use. Without this certification, model training cannot be audited, reproduced, or defended under AI governance regulations.
Data Quality in Regulated Industries
Financial services: BCBS 239 requires banks to demonstrate that risk data is accurate, complete, and traceable from source to regulatory submission. Quality certifications maintained through a governance program satisfy this requirement continuously rather than through periodic manual assessments. SOX requires reliable financial reporting data with documented quality controls.
Healthcare: HIPAA requires that patient records are accurate and current. Inaccurate patient data causes clinical errors, billing disputes, and privacy violations. Data quality programs in healthcare environments must cover PHI datasets with the strictest quality standards and the most rigorous monitoring.
Pharmaceuticals: FDA 21 CFR Part 11 and GxP regulations require data integrity for clinical and manufacturing data. Quality management programs must maintain complete validation histories for all data used in regulatory submissions, demonstrating that data was not altered without documentation and that every quality check passed.
Retail and e-commerce: Customer data quality directly affects revenue: duplicate records produce duplicate communications that damage brand trust, inaccurate address data causes failed deliveries, and stale preference data produces irrelevant recommendations. Inventory data quality affects fulfillment: stale stock data causes overselling and customer disappointment.
How Actian Approaches Data Quality
Actian Data Observability provides AI-based anomaly detection built for an open architecture, so teams catch pipeline problems early across hybrid-cloud data estates. Actian Data Intelligence Platform shows quality scores and certification in the data catalog, connects them to data lineage, and routes issues to owners through data stewardship workflows. Monitoring, ownership, and trust signals work together, so a quality issue is detected, traced to its source, and assigned to the person who can fix it.
FAQ
Data quality is how well data does its job. If you ask data a question and the answer is wrong because the data is inaccurate, incomplete, outdated, or inconsistent, you have a data quality problem. High-quality data gives you correct answers reliably.
Accuracy (data correctly represents the real-world value), completeness (all required fields are populated), consistency (the same value is represented identically across systems), timeliness (data is current when needed), validity (data conforms to defined formats and rules), and uniqueness (each real-world entity is represented exactly once with no duplicates).
The most common causes are: manual data entry errors, system migrations that transform data incorrectly, schema changes that break existing validation rules, integration failures that cause records to sync incorrectly, inconsistent definitions across systems (two systems define “active customer” differently), and the absence of validation rules that would catch errors before they enter the system.
Through four mechanisms: data profiling (automated scanning to assess current quality characteristics), validation rules (business rules that define what valid data looks like), quality scoring (aggregation of profiling and validation results into a single score per asset), and continuous monitoring (automated alerts when scores fall below thresholds or anomalies appear).
Data certification is the formal process of marking a data asset as approved for use after it has met defined quality thresholds and been reviewed by a data steward. Certified assets appear with a verified badge in the data catalog. Users trust certified assets without independent validation, which is what makes data quality programs operationally valuable rather than just bureaucratically correct.
Enforce validation rules at ingestion and inside pipelines, and configure failing records to be rejected or quarantined instead of loaded. Add continuous monitoring so problems that pass the rules, such as a sudden drop in volume or a schema change, still raise an alert.
Look for a tool that profiles customer fields, detects duplicates and near-duplicates, checks formats and completeness, and flags anomalies, with a steward reviewing merges and corrections. Quality results that appear in your catalog and link to lineage are easier to act on than results kept in a separate tool.
Choose software that monitors freshness, volume, schema changes, and data distribution as data moves, and that can run checks inside your orchestration tools. Alerts should reach an assigned owner, with a record of how the issue was resolved.
Yes. Profiling can reveal typical formats, ranges, and null rates, and software can propose validation rules based on them. A data steward or domain owner should approve each rule before it is enforced, because only the business can say what valid means for a field.
The best software fits where your data lives and what you need to do: block bad data at ingestion, monitor live pipelines, catch errors in customer records, or certify data for reports and AI. Choose software that covers your on-premises and cloud sources, connects issues to owners, and links quality to lineage and your catalog. Compare total cost, not only the license: setup, rule building, integration, steward time, and growth. Favor tools that fix problems at the source, because downstream cleaning has to be repeated. Test it on a real pipeline before you commit.
