Loading
RAG chatbot
Retrieval-augmented generation is the standard way to make a chatbot answer from your content instead of from memory. Here is how it actually works, where it fails in production, and what we do about each failure.
Most explanations of RAG stop at the happy path. The failure column is the part that decides whether your chatbot is trustworthy in month six.
Your documents are read and normalised: PDFs, help centre articles, spreadsheets, database records.
Where it failsGarbage in. Superseded pricing sheets and half-current policy PDFs get ingested alongside the good ones, and nothing downstream can tell them apart.
Long documents are split into passages small enough to retrieve precisely and large enough to still make sense alone.
Where it failsChunks cut mid-clause, so a retrieved passage says 'this does not apply if' and the condition is in the next chunk that never got fetched.
Each passage becomes a vector, so the system can find text by meaning rather than by keyword.
Where it failsSemantic similarity is not truth. Two passages about refunds (one current, one from 2023) look nearly identical to a vector index.
The question is embedded the same way, and the closest passages are pulled back as context.
Where it failsThe top result is close but wrong, or the right passage ranks fourth and gets cut. This is where most RAG chatbots quietly fail.
The model answers using the retrieved passages, rather than from whatever it absorbed during training.
Where it failsGiven weak context, a model will still produce a confident answer. Retrieval does not prevent invention. It only makes invention less likely.
The answer carries the passages it used, and is checked against them before it reaches anyone.
Where it failsSkipping this is the single most common shortcut, and it is why so many RAG deployments cannot tell you why they said something.
Anyone telling you RAG solves hallucination is selling something. Given weak context, a model will still produce a confident answer. That is what it is built to do. What closes the remaining gap is checking the answer against the passages it retrieved, and refusing when the support is thin. That is a system design decision, not a property you get for free by adding a vector database.
See what we build on top of itFAQ
No, and anyone claiming otherwise is overselling. It substantially reduces it by giving the model the right text to answer from. What closes the remaining gap is checking the answer against its sources and refusing when support is weak. That is a system design decision, not a property of RAG.
For factual questions over content that changes, yes. You update a document rather than retrain. Fine-tuning is the better tool for teaching tone, format, or a narrow task. They solve different problems and are not really competitors.
Less than people expect, and quality matters far more than volume. Fifty accurate, current, well-structured pages outperform a thousand pages of mixed vintage, and the thousand pages are usually worse than nothing, because they retrieve confidently.
Yes, and for most businesses that is where the valuable answers live. Order status, availability and account state are lookups, not retrievals. A good build uses both and knows which to reach for.
It depends on volume and how much retrieval each answer needs. We quote a build price and a flat monthly run before any work starts, so you are not exposed to a per-token bill you cannot forecast.
Book a free call
Thirty minutes, no pitch. You leave with a straight answer on whether an agent helps your business, whether or not you build it with us.
We will not put you on a nurture sequence. If it is not a fit we say so and that is the end of it.