A prediction is not a decision — and never a cause
Machine learning for business decisions: prediction, classification, and causality traps
The question: When should a manager use machine learning, and when is ordinary analysis enough?
Most enterprise AI still depends on supervised learning, ranking, anomaly detection, and forecasting — even when the interface is generative. Managers who cannot read a model's output economically will either over-trust it or waste it.
What the lesson covers
Traditional programming encodes rules; machine learning infers patterns from labelled examples. Supervised learning maps inputs to labels (churn: yes/no; loan: default/repay). Unsupervised learning finds structure without labels (segmentation, anomaly detection). Reinforcement learning optimises actions through reward. Most business prediction problems — churn, demand, fraud, credit risk, maintenance — are supervised learning, and many don't need an LLM at all.
The workflow that matters: data → features → model → threshold → decision → action → outcome → feedback. Models are trained on one slice of data, validated on another, and tested on data they have never seen. Two classic failures: overfitting (memorising the training data — great in the lab, useless live) and leakage (information from the future or from the label sneaking into features, inflating scores that collapse in production).
Model evaluation is a management problem, not just a technical one. A confusion matrix is really a cost matrix: a false positive (flagging a loyal customer as a churner, denying a good loan) and a false negative (missing a churner, approving a future default) have different economic costs. Choosing the decision threshold is therefore a business decision about which mistake is cheaper — one worth arguing about in a boardroom, not defaulting to 0.5.
The causality trap: prediction is not explanation. A churn model may "predict" perfectly and still mislead action — the retail case where high-scoring customers received retention offers they never needed, because the model identified who would churn, not who would be saved by an offer. Causal answers (what happens if we intervene?) require experiments, quasi-experiments, or strong assumptions — this is why churn prediction is not churn prevention.
Fairness and drift complete the management picture: models inherit bias from data and proxies, and the world changes under a deployed model (drift), so monitoring, retraining plans, and named owners are part of any serious deployment.
Key points
- Supervised / unsupervised / reinforcement learning solve different problems; most business predictions are supervised — and don't need an LLM.
- Overfitting and leakage are the two lab-to-production killers; held-out test data is the defence.
- The confusion matrix is a cost matrix; the threshold is a business decision about which mistake is cheaper.
- Prediction is not causality: churn prediction is not churn prevention. Interventions need experiments or uplift logic.
- Deployed models drift — monitoring, retraining triggers, and a named owner are part of the system.
Framework — Prediction Value Chain
data → features → model → threshold → decision → action → outcome → feedback. Every link can break: bad labels, leaky features, wrong threshold, no action, no measured outcome, no feedback loop. Audit the chain, not just the model.
The lab
Read a real churn dataset like a manager: baseline, drivers, threshold economics.
Open this lesson, its lab and its quiz
Sources and further reading
- Machine Learning Crash Course — Classification — Google
- Interpretable Machine Learning (free online) — Christoph Molnar
- The Hundred-Page Machine Learning Book — Andriy Burkov