AI runs on data plumbing — readiness beats model choice
The data backbone for AI: quality, governance, pipelines, and unstructured data
The question: What data capabilities must exist before AI can create reliable business value?
Business teams routinely underestimate data plumbing. Enterprise AI depends on data quality, access rights, lineage, metadata, and governance — the unglamorous layer where most AI projects actually succeed or die.
What the lesson covers
Data is not a neutral raw material. It is produced by processes with incentives, errors, missingness, and shifting definitions ("active customer" means three different things in three systems). Before any AI discussion, ask: where does this data come from, who touches it, what breaks it, and what does each field actually mean?
Data quality has named dimensions: accuracy, completeness, timeliness, consistency, uniqueness, and fitness for purpose. The last one is decisive — data can be perfectly accurate and useless for your question. Quality is also a moving target: pipelines silently rot as source systems change, which is why quality checks belong in the pipeline, not in a one-off audit.
The modern architecture vocabulary in business terms: pipelines move and transform data; warehouses store structured, query-ready data; lakes store everything raw; lakehouses blend both. Metadata (what the data means) and lineage (where it came from, what transformed it) are what make any of it trustworthy — and auditable when a regulator or executive asks "why did the model say that?".
Generative AI adds the unstructured backbone: documents, contracts, emails, calls, wikis. These need their own governance — ownership, freshness/retention rules, access permissions, and chunk-ready structure — because a RAG assistant is only as good as the documents it retrieves. Stale policy documents don't just fail: they confidently misinform.
Governance is roles, not documents: data owners (accountable for a domain), stewards (quality day-to-day), and consumers with defined rights. The AI Data Readiness Scorecard turns this into practice: for every AI use case, score purpose, source, quality, permissions, lineage, update frequency, and evaluation labels before you build.
Key points
- Data is produced by processes with incentives and errors — interrogate the process, not just the table.
- Quality dimensions: accuracy, completeness, timeliness, consistency, uniqueness, fitness for purpose.
- Metadata and lineage make data trustworthy and auditable; pipelines rot silently without checks.
- Unstructured data (documents, calls) needs ownership, freshness rules, and permissions before RAG.
- Score readiness per use case BEFORE building — most "AI failures" are data failures discovered late.
Framework — AI Data Readiness Scorecard
Per use case, score 1–5: purpose fit, source reliability, quality, permissions/rights, lineage, update frequency, and evaluation labels. Anything scoring ≤2 is a project risk with a named remediation owner — before the model conversation starts.
The lab
Clean the messy sales dataset with AI assistance — and catch what the AI misses.
Open this lesson, its lab and its quiz
Sources and further reading
- DAMA-DMBOK — data quality & governance — DAMA International
- NIST AI RMF — data quality concepts — NIST