ML//RAG

RAG (retrieval-augmented generation) is a technique for answering with a language model in which relevant documents are retrieved first and put into the context window alongside the question, so that the answer rests on those sources. The knowledge enters at inference, as expanded context with no change to the weights, where pre-training bakes it into the weights for good.

RAG (retrieval-augmented generation) is a technique for answering with a language model in which relevant documents are retrieved first and put into the context window alongside the question, so that the answer rests on those sources. The knowledge enters at inference, as expanded context with no change to the weights, where pre-training bakes it into the weights for good.

The sources are web search, databases and PDFs, retrieved from a vector database by cosine similarity. The search is powered by sentence transformers: BERT fine-tuned with contrastive learning so that [CLS] captures semantic similarity instead of just predicting masks.

GPT does not use [CLS]: it takes the last token as the embedding, which causal masking trains to carry the full meaning. RAG is only as smart as its search engine. If the retrieved documents are garbage, the model will confidently synthesize garbage.

In industry it is the standard way to put a large language model to work on a plant's own documents: answering a maintenance technician from the pump's manual, the site procedures and the standards, with the fragments it used shown beside the answer so they can be checked. The documents can be updated without retraining anything, and they stay in the company's index instead of being baked into a vendor's weights.

It is one of three places a system keeps knowledge. Parametric memory is in the weights and frozen at training; the context is the working memory of one call; RAG draws on external memory, which can be edited any day (agent memory). Updating the index changes what the system can answer without anything being learned, so it is the usual answer to a fact that changes, and fine-tuning the usual answer to a behaviour that should (fine-tuning).

Before any of it, plain keyword search and rules often answer the question (BM25, hybrid search); retrieval quality decides most of the result, and a wrong passage retrieved is an error the generator then dresses in fluent prose (hallucination).