Your AI Support Bot Just Promised A Refund Policy That Doesn't Exist

A customer asks your chatbot if they can return an item after 60 days. Your real policy says 30. The bot, trying to be helpful, says "yes, we offer a 60 day window for loyal customers." It made that up. There is no loyalty exception. There never was. Now the customer has a screenshot, and in a few jurisdictions and in most people's sense of fairness, a company is on the hook for what its own representative told them, bot or not.

This isn't a hypothetical. It's the natural failure mode of handing a language model a support role without constraints. The model's job, by design, is to produce a plausible, fluent answer. It was never told that "I don't know, let me check" is an acceptable output. So when it doesn't have the exact policy memorized, it writes one that sounds right.

Why this keeps happening

Most teams bolt a chatbot onto their help docs and assume the model will stick to what's written. It won't, not reliably. A few patterns we see over and over:

• The bot is given a huge, loosely organized knowledge base instead of a small set of exact, current policy text, so it blends fragments into something new.
• Nobody tells it what to do when the answer isn't in its source material. Silence gets filled with a guess.
• The policy itself lives in three places (a PDF, a help center article, an internal wiki) and they've drifted out of sync, so even a careful model is choosing between two correct-looking answers that disagree.

None of this is an argument against using AI for support. It's an argument for treating the bot like a junior employee who needs a script, not a philosophy. Retrieval that pulls from one canonical, versioned source. Explicit instructions to say "I'm not sure, here's how to reach a person" instead of inventing specifics. And logging, so when something does go wrong you can see exactly what the bot said and why, instead of taking the customer's word for it after the fact.

The fix is boring, which is the point

Nobody wants to hear that the solution to "my AI said something false" is "write better guardrails and test them," but that's genuinely most of it. The interesting work is deciding which questions a bot should ever answer unsupervised, and which ones should route straight to a human with full context attached.

This is the kind of system we build at Cedrix: support automation that knows the edges of its own knowledge and hands off instead of improvising. If you want to see what that looks like for your business before committing to anything, our free MVP offer is a low risk way to find out. Or just get in touch and tell us what your bot keeps getting wrong.