If you work in a mid-sized company, there is a reasonable chance you have experienced this: someone proposes implementing a chatbot for customer service or to handle internal queries, the project gets approved, it launches with enthusiasm, it works more or less for the first few months, and then it gradually falls into disuse until no one mentions it. At some point it is quietly deactivated or simply left without updates until it gives incorrect answers to half the questions.
This is not a technology problem. It is a problem of design, expectations and maintenance. And it is avoidable.
The decision tree trap
The first generation of chatbots — and many of those still being deployed today — are based on decision trees: if the user says A, respond with X; if they say B, respond with Y. The problem is that users don’t speak to a chatbot as though it were a multiple-choice form. They say things like "I have a problem with my order" instead of clicking "Order tracking". And when the chatbot doesn’t understand the question, or gives a response that doesn’t match what the user actually asked, the frustration is immediate.
Decision-tree chatbots have a low utility ceiling. They work for highly predictable flows — booking an appointment in a known availability calendar, checking an account balance, tracking a shipment with a tracking number — and they break the moment a query steps outside the script.
The solution is not to make the decision tree bigger. It is to use a language model that understands the intent behind what the user writes, regardless of how they phrase it.
The problem of the outdated knowledge base
The second most common failure: the chatbot worked well when it was implemented because the answers were correct. Six months later, prices have changed, opening hours are different, there are new products the chatbot knows nothing about and policies have been updated. The chatbot keeps giving out the old information with complete confidence.
A static knowledge base requires active maintenance. Every time something changes in the company — a price, a schedule, a policy, a procedure — someone has to update the chatbot. If that process is not formalised and assigned to someone with explicit responsibility, the knowledge base becomes outdated and the chatbot turns into a problem rather than a solution.
Modern chatbots solve this by connecting the language model directly to the company’s sources of truth — the ERP, the management system, the product database — so that when the chatbot answers a question about a product’s price, it queries the current price in the system, not the one that was there when it was set up.
The expectations nobody manages well
The third failure factor is the hardest to resolve because it is human, not technical.
Chatbots create expectations in two groups with opposing interests. Management wants the chatbot to resolve the highest possible percentage of queries without human intervention, because that justifies the investment. Users want personalised attention, exact answers to their specific situations, and the ability to escalate to a person when the chatbot cannot help them.
The tension between these two expectations leads to configurations where the chatbot tries to resolve more than it can resolve well, producing incorrect or frustratingly generic responses that damage the user experience. Or configurations where the chatbot is so conservative in what it tries to resolve that the percentage of escalations to human agents is so high that the business case disappears.
The right balance requires defining honestly which types of queries the chatbot can resolve well and which should always be escalated. A chatbot that resolves 60% of queries well and correctly escalates the remaining 40% is far more valuable than a chatbot that attempts to resolve 90% and does so poorly in half of those cases.
What sets a working chatbot apart
A chatbot that works in the long term has three characteristics that failing chatbots rarely have simultaneously.
First, it has context memory. It remembers what has been said in the conversation and adjusts its responses accordingly. The user who already explained that they have the premium plan does not have to repeat it when they ask the next question in the same conversation.
Second, it has access to real-time data from the company’s systems. It does not respond with static information that may be out of date, but by querying the company’s sources of truth directly at the moment the question is asked.
Third, it has a clear escalation process. When it cannot resolve a query — or when it detects that the user is frustrated — it escalates intelligently to a human agent, passing the full conversation context so the agent does not have to start from scratch.