"We finished the security work. The project should run smoothly now." That's what a DX lead at a large manufacturer said last month, coffee in hand. His team had actually spent six months building out audit logs, PII masking, and role separation. Then the H1 review put the project "on hold."
The reasons weren't singular. The data was still scattered, cases that worked in PoC broke down under real user traffic, and two months after launch nobody was opening the dashboard. To put it precisely, security was a pass condition, not a success condition. If Episode 6 covered security and governance, this episode is about the wall you hit right after that.
In a July 2025 survey by MIT Project NANDA, 95% of organizations running generative AI pilots reported no measurable P&L impact. RAND's enterprise AI analysis found 80.3% failed to deliver promised business value, with 33.8% scrapped before ever reaching production. The numbers look bleak, but the ways projects fail are surprisingly repetitive. Watching a range of projects up close, they converge on five patterns.
1. Unprepared Data — The Moment You Layer RAG on Scattered Documents
In one first meeting, the team lead said "Our internal docs are on SharePoint." We opened it: 34 folders, 1,200 duplicate documents, the latest manual and a three-year-old version living side by side under the same name. What happens when you run RAG indexing on that state? A user asks about the "return policy" and gets an answer blending the 2022 and 2025 versions. Convincingly, too.
One industry survey pointed to data quality as the root cause in 85% of failed projects. Only 12% of organizations had data ready to support AI applications. Which means the other 88% are stuck in the "let''s just start with what we have" trap.
- Audit source documents for freshness, duplicates, and version drift for at least three weeks
- Tag PII and confidentiality levels at the document level
- Measure accuracy in advance with a 100-query "gold set" before RAG indexing
- If accuracy is below 70%, finish data cleanup before anything else
2. The PoC Trap — What Works in the Demo Collapses Under Real Traffic
In an S&P Global survey, 46% of AI PoCs were scrapped before reaching production. My first reaction was "really, that much?" — but after a few projects, it made sense.
A PoC demo tends to run like this: showcase with 20 prepared questions, applause from the room, "let''s build for real." But real users ask things you didn''t predict. Typos, casual phrasing, slang, repeated questions, three questions crammed into one sentence. PoC accuracy of 92% drops to 61% in production. The cause is clear: the demo assumed an "ideal user." Real users aren''t ideal.
- Evaluate from the PoC stage using real user logs (anonymized)
- Maintain 50 "deliberately awkward queries" as a stress set
- Define PoC success as "real-usage accuracy above X%" rather than "passing the demo"
3. No Operations — Two Months In, Nobody Opens the Dashboard
Honestly, this is the pattern I see most often. Launch is an event. Operations is a habit. Most organizations budget only up to the event.
What''s needed after launch is obvious: weekly accuracy metric checks, failure case reviews, prompt improvement PRs, model update response, usage reports. Without someone owning this, three months later the tool is unchanged but the users are gone. A natural ending.
In an April 2026 Gartner report, only 28% of AI infrastructure projects hit their promised ROI. Many in the miss group pointed to a lack of operational resources. Building well and being used long-term are different games. ROI metric design itself was covered in Episode 3. Whether the KPIs you defined then are actually reviewed by someone every week — that''s what decides operational success or failure.
- Before launch, put the operations owner and weekly metrics review meeting on the calendar
- Sustain a rhythm of at least two prompt/data update PRs per month
- Document alert and response processes for usage drops
4. Overreach — The Cost of Assuming "AI Will Handle Everything"
This pattern shows up more with adoption sponsors than approvers. The picture is "let AI handle it instead of staff." Attractive at first, but automated execution without human checkpoints leads to incidents.
One 2026 report cites the most common cause of agentic system failure as "missing human-in-the-loop checkpoints." Overtrust autonomy and bad judgments execute without human intervention. A wrong refund approval in customer service, a factual error deployed as-is in auto-generated docs. Once an incident hits, trust inside the organization breaks — and the project itself gets suspended.
- Define human approval stages for automated actions based on amount, scope, and criticality
- Run all actions in "suggestion mode" for the first three months and accumulate review logs
- Check trust metrics (approval rate, correction rate) before rolling out automation in stages
5. Security as an Afterthought — Rollbacks at the Audit and Legal Review Stage
This was covered in an earlier episode, but a quick note on how "security later" comes back around in the late stages of a project.
One case. Service development 90% complete, successful internal demo. Then legal review flags that "audit logs are storing raw personal data." The refactor scope was so large it required a four-month rollback. An extra 40% of the original budget went in. One survey shows 62% of organizations struggle with data governance — this kind of late-stage rollback is what that statistic actually looks like.
The cost-side blowup was covered in Episode 5. The budget overruns created by trailing security align exactly with that episode''s case.
- Assign audit and legal reviewers from the architecture design stage
- Include PII handling, audit logs, and role separation in the MVP scope
- Register "legal sign-off" as a sprint completion condition before service launch
Wrapping Up — Failures Share "Sustaining" as the Problem, Not "Building"
Line up those five, and one thing stands out. They''re all problems of "sustaining," not "building." Much of failure is decided in organizational and process design, not technology choice. More than half of the early meetings for in-house LLM projects at 5years+ are spent walking through which of these five risks is the biggest for the client. If you''re evaluating one, feel free to share your situation via a free project risk review — no pressure.
The next episode is the series'' final execution piece. It lays out a phased plan from diagnosis → PoC → operations and follow-on development. It''s a look at calendar, roles, and milestones — how to actually keep a project running while avoiding this episode''s five patterns.