RAG: how an assistant finds the answer in your documents

When I build an assistant for a business, the first question is usually "do we need to train the AI on our data?". Almost never. What we do is simpler and more reliable: on every question, the system searches your documents for the fragments that help and hands them to the model along with the query. It's called RAG (retrieval-augmented generation) and it's the difference between a bot that makes things up and one that answers with your real policies.

How it works, in three steps

1. Split. Your documents (FAQs, policies, catalog) are cut into short fragments, one topic each.
2. Search. When a question arrives, its meaning is compared with each fragment's and every fragment gets a similarity score. It doesn't look for matching words: "how long until it arrives?" finds the delivery-times fragment even if they share no words.
3. Answer. The top three fragments go to the model with a clear instruction: "answer only with this information".

Try it with a sample store. Pick a query and watch the fragments reorder:

Where it breaks

Almost every RAG problem lives in the first step, not in the model:

• Fragments that mix topics. If shipping and payments share a paragraph, the score gets confused and the model receives information nobody asked for.
• Stale information. The assistant confidently repeats whatever the document says, even if prices changed two months ago.
• Questions with no answer. If no fragment scores well, the right move is "I don't know, let me connect you with someone", not improvising. That's configurable, and it's the first thing I configure.

What you need to start

Less than you think: a tidy document with the questions you get the most and their real answers. That's enough for a first version. The rest (connecting it to stock, a calendar, WhatsApp) comes later, once the base answers well.