86. RAG: Giving Language Models Long-Term Memory
Large language models are limited by their training cutoff dates and lack knowledge of recent events or internal company data. When asked about such topics, they often hallucinate plausible but incorrect answers. Retrieval-Augmented Generation (RAG) solves this by providing the model access to an external knowledge base at query time. Relevant documents are retrieved and included in the prompt as context, enabling the model to answer based on that specific, current information. This approach dramatically reduces hallucinations, allows the model to cite its sources, and enables knowledge base updates without retraining. The article includes a code example using SentenceTransformer and cosine similarity to implement a basic RAG system.
RAG enables developers to build LLM applications grounded in up-to-date, proprietary data without costly retraining.