The AI support projects that fail and the ones that work are usually running the same technology. What separates them is everything that happened before the tool was switched on: whether the team knew which questions to automate, whether there was anything worth training the AI on, and whether a human was still standing behind it.
The pressure to buy is real. In a Gartner survey of 321 customer service and support leaders, 91% reported pressure from executive leadership to implement AI. That pressure is exactly why so many projects launch before they're ready.
Why AI customer service so often disappoints: what the numbers say
Three findings are worth knowing before you sign anything.
- Most failures are data failures, not technology failures. Gartner predicts that through 2026, organisations will abandon 60% of AI projects that aren't supported by AI-ready data. In the same research, 63% of organisations either didn't have the right data management practices for AI or weren't sure whether they did.
- Agent projects get cancelled at a high rate. Gartner expects over 40% of agentic AI projects to be cancelled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls. The same analysis warns about "agent washing": existing chatbots and automation rebranded as AI agents.
- The replace-your-team plan keeps getting reversed. Gartner predicts that by 2027, 50% of organisations that expected to significantly reduce their customer service workforce will abandon those plans. In a poll of 163 service leaders, 95% said they planned to keep human agents to define AI's role.
Read together, those three say something useful: the projects that fall over aren't the ones that bought the wrong software. They're the ones that had nothing solid to build on and no one left to catch what the AI dropped.
What the projects that work have in common
The same research points at the pattern. Among leaders planning AI in 2026, 58% aim to upskill agents into knowledge management specialists, because both AI and self-service need accurate, continually updated content. Nearly 80% are moving agents into new roles rather than removing them, and 84% are adding new skills to the agent role.
In other words, the organisations getting this right are the ones treating AI as a change to how knowledge is written and how people work, not as a purchase.
A word of caution on the success figures you'll see quoted elsewhere. Most published resolution-rate benchmarks come from the AI vendors themselves, and they're self-reported. Treat them as direction, not proof, and ask any vendor how their numbers were measured before you plan around them.
The readiness checklist
Work through these before you buy. Most take days, not months, and they're all useful even if you never deploy AI at all.
-
01Find out what your customers actually ask
Tag a few hundred recent tickets by reason. You'll usually find that a small number of question types cover most of your volume. That list is the only sensible starting point for automation, and it's the same first step as scaling a support team.
-
02Build the knowledge base your AI agent will answer from
This is the step that decides the outcome. AI answers from your documentation, so thin, contradictory or out-of-date help content produces thin, contradictory or out-of-date answers, at speed and at scale. Build the articles from real tickets, in plain language, one question per article. Our guide to knowledge bases that deflect the easy stuff covers how.
-
03Decide what AI is allowed to do
Write it down explicitly: what it may answer on its own, what it may draft for a human to approve, and what it must never touch. Refunds, account changes, cancellations, complaints, anything legal or safety-related, and anything involving an upset customer belong in the last group.
-
04Design the handoff before go-live
Most of the damage from support AI happens at the moment it gives up. If the customer has to repeat everything to a person, you've made the experience worse than no AI at all. Decide what triggers a handoff, what context travels with it, and who picks it up. We've written about designing that escalation in detail.
-
05Agree how you'll measure it
Deflection rate on its own is a vanity metric: a customer who gives up looks identical to one who was helped. Measure resolution without human involvement, reopen rate, satisfaction on AI-handled conversations specifically, and time to resolution end to end. Those are the metrics that show whether it's working.
-
06Start narrow and read the transcripts
Launch on one or two question types, on one channel, with a human reviewing conversations weekly. Widen only when the numbers hold. The weekly read-through is what catches the confidently wrong answer before your customers do.
-
07Keep your people, and change their job
Someone has to own the knowledge base, review AI conversations and handle what escalates. That's a real role, and it's the one the research says organisations are creating rather than cutting. The hybrid model isn't a compromise; it's what works.
Tick them off as you go — nothing is saved or tracked, it's just here to work through.
Questions to ask any AI vendor
- What does it answer from? If the answer is "your help centre", your help centre is the project.
- What happens when it doesn't know? Ask to see the handoff, with the conversation history that travels with it.
- How do we stop it answering something? There should be a straightforward way to put a topic out of bounds.
- How is resolution measured? Insist on the definition, not the percentage. Deflection and resolution are not the same thing.
- Can we read the transcripts? All of them, easily, and export them.
- What's genuinely new here? A fair question, given how much existing automation has been relabelled as AI agents.
- What does it cost when volume doubles? Per-resolution pricing behaves very differently from per-seat in a busy month.
The bottom line
AI in customer support works when it's answering well-documented questions inside clear limits, with people owning the knowledge and catching what it can't handle. That's not a technology decision, and almost all of the work happens before you buy anything. If your knowledge base is thin, start there. It's the step that turns AI from a risk into a genuine improvement, and it pays for itself even if you never automate a single conversation.
Not sure your support is ready for AI?
In a free review we'll look at your ticket mix and your documentation, and tell you honestly which questions are safe to automate and which aren't. No obligation, and you keep the findings either way.