2026-07-13 · Article
What the 5% Did
The GenAI failure statistics are now boring. The interesting question is what the small working minority did differently, and it fits on one page.
For the Record: AI Governance Tools for SA Boards, Article 4 of 4
The numbers have stopped being controversial and started being wallpaper. The most-cited enterprise study found 95 percent of GenAI pilots produced no measurable financial return within six months. Economic research found roughly 90 percent of firms reporting no productivity impact at all, even as executives kept projecting gains. The latest global survey work says 88 percent of organisations now use AI somewhere, while fewer than one in ten can point to measurable, sustained value at scale.
Boards have noticed. The clearest signal in this year's data is not another failure rate; it is a change in what gets counted. In a survey of 830 IT decision-makers, "direct financial impact" nearly doubled as the primary success metric for AI initiatives while "productivity gains" fell as a leading measure. Translated out of survey language: the era of justifying an AI project with "it saves the team four hours a week" is over. Somebody now asks four hours of what, worth how much, measured against which baseline, and the room goes quiet.
South Africa is running the same experiment in public. A major grocery retailer launched a conversational AI shopping assistant this month, an explicit competitive answer to a rival's app; one of the big banks is piloting autonomous agents alongside its relationship managers. Some of these will produce numbers their CFOs can defend. Statistically, most will not. The difference will not be the model.
The 5% did not have better AI
Look across the post-mortems of the failures and the case studies of the successes and the pattern is almost insultingly consistent. The projects that produced defensible value did three unglamorous things before any technology decision.
They asked a narrower question. Not "deploy AI in claims" but "reduce the manual handling time of one document type in one claims process." A narrow scope is not a lack of ambition; it is what makes measurement possible at all. The failures tried to boil the department; the successes picked the one pot with a lid.
They fixed the data before the demo. The working projects ran on production-quality data from a stable, repeatable process. The failed ones discovered in week seven that the data feeding the model was incomplete, inconsistent or legally unusable, a discovery that was available in week one to anyone who asked.
They wrote the baseline down first. This is the one that separates the 5 percent most cleanly. Before deployment, they recorded the current number: minutes per transaction, error rate, cost per case, whatever the process truthfully cost. Which means that after deployment, "is this working" was an arithmetic problem, not a debate. The 95 percent, by and large, cannot answer "compared to what," because nobody measured the before.
None of this requires a consultancy to say it. What a methodology adds is the discipline of not skipping it under enthusiasm: in our own delivery work, no build starts until the scope is a single bounded use case and the baseline is signed, and no engagement ends without the after-numbers measured against it. The failure statistics above are, mostly, a census of projects that skipped that step.
The artefact: the ROI Baseline Memo
File this. One page, completed and signed before any AI project starts build. If a project cannot fill this in, that is not a paperwork problem; the project does not yet know what it is for. Filed memos make the quarterly "is our AI working" question answerable in minutes.
Project: ______________________ Date: __________ Sponsor (signs the numbers): ______________________
| Section | Content |
|---|---|
| The one process | The single, bounded process this project changes, in one sentence. If the sentence contains "and", split the project. |
| Baseline, measured | The current numbers, measured not estimated: volume per week, minutes per item, error or rework rate, cost per item. State the measurement window and who measured it. |
| Target | Which of those numbers moves, by how much, by when. |
| Measurement method | Exactly how the after-numbers will be captured, using the same definitions as the baseline, and who owns that measurement. |
| Data reality | Where the data lives, its quality as verified (not assumed), and any POPIA basis it requires. |
| Payback | Engagement plus run cost, divided by the monthly value of the target movement, in months. If it exceeds 18, say why the project still stands. |
| Review dates | The date the after-measurement happens (no later than 90 days post go-live) and the date the Sponsor reports the result to exco or board. |
| Kill criteria | The result at review that stops or reworks the project. Written now, while nobody is defending sunk cost. |
The memo's power is entirely in its timing. Completed before build, it turns every later status meeting into a comparison against a number nobody can retro-fit. Completed after build, it is fiction with a signature.
For the record, and the end of this series
Four articles, four documents now in your file: a vetting checklist that catches machine-fabricated content before it ships, a questionnaire that gets you through the insurance renewal that is coming, an agent inventory that answers the auditor's first question, and this memo, which decides whether the next AI project can prove it was worth doing.
None of them requires a transformation programme. All of them require a name in a box and the discipline to stop when a row cannot be honestly filled. That, in the end, is what the working 5 percent actually bought: not better models, but the unfashionable habit of writing things down before believing them. It is also, not coincidentally, what a board can defend, on the record, when someone eventually asks.
Sources: MIT "GenAI Divide" study on enterprise GenAI project outcomes; NBER working-paper research on firm-level AI productivity impact; McKinsey, The State of AI 2026 (adoption versus scaled-value figures); Futurum Group survey of 830 IT decision-makers on AI success metrics, 2026; Pick n Pay conversational shopping assistant launch, July 2026, via CNBC Africa; Absa agentic-AI pilot disclosures, 2026; Tech Sight, TechSight 90 methodology (Phase 1 scope discipline and the Value Realization Index), techsight.co.za.
Book the two-hour diagnostic