Analytics only matters when a named decision changes
AI for analytics and decision intelligence
The question: How can AI improve analysis without replacing critical thinking?
AI accelerates every step of analysis — cleaning, exploring, charting, narrating. It also fabricates patterns and overstates confidence fluently. The discipline that protects you is decision intelligence: connect every analysis to a decision, an owner, and a feedback loop.
What the lesson covers
Decision intelligence links data → model/analysis → decision → action → outcome → feedback. An analysis that doesn't name its decision and decision owner is entertainment. Start every analytical task with: what decision will this change, who makes it, and what would make them act differently?
The analytics ladder: descriptive (what happened), diagnostic (why), predictive (what will happen), prescriptive (what should we do), causal (what happens IF we act). Generative AI helps at every rung — drafting code, summarising patterns, generating hypotheses and counter-arguments — but the rungs still have different evidence requirements, and AI does not change that.
AI-assisted analysis fails in characteristic ways: it fabricates patterns in noise, overstates confidence, produces plausible-but-wrong aggregations, and writes persuasive narratives for whatever the chart seems to say. The controls: reproducible steps (code or logged operations, not vibes), an assumptions log, data-quality checks before interpretation, sensitivity analysis (does the conclusion survive different assumptions?), and source-linked numbers.
Manager's discipline when consuming AI analysis: require assumptions, uncertainty, counterfactuals, and "what would change this conclusion". The economics of analytics: value scales with decision frequency × stakes × current error rate — automate the frequent low-stakes analyses fully, keep humans on rare high-stakes calls with AI as the challenger (red-team, pre-mortem).
Where AI should not be trusted without expert verification: causal claims, legal/medical/financial advice, and anything involving confidential data pasted into public tools. "The AI found that X causes Y" is not a sentence — it is a hypothesis with a marketing department.
Key points
- Name the decision and the decision owner before analysing anything.
- The analytics ladder (descriptive → causal) has rising evidence requirements AI does not waive.
- AI fabricates patterns and narrates noise persuasively — reproducibility, assumptions logs, and sensitivity checks are the controls.
- Automate frequent low-stakes analysis; keep humans + AI-as-challenger on rare high-stakes calls.
- Causal claims need experiments, not confidence.
Framework — Decision Brief Template
question · decision owner · data used (+ quality notes) · options considered · analysis/model · uncertainty & sensitivity · recommendation · next experiment. One page. If a field is empty, the analysis is not done.
The lab
Run an AI-assisted analysis sprint on real messy data — and interrogate the charts.
Open this lesson, its lab and its quiz
Sources and further reading
- Decision intelligence selections — Harvard Business Review
- NIST AI RMF — measurement & monitoring — NIST