Topic 03 / 8

Retrieval-Augmented Generation (RAG)

~20 min read  //  LLM Engineering Series  //  Coding India

The Problem: Knowledge Cutoffs

LLMs are frozen in time at the end of their training. They don’t know about yesterday’s news or your company’s internal documents. Fine-tuning an LLM on new data is expensive and doesn’t solve the problem of real-time updates.

The Solution: RAG

Retrieval-Augmented Generation solves this by searching your data before asking the LLM. The workflow looks like this:

  1. User asks a question.
  2. Retrieve: Search your database (often a vector database) for documents relevant to the question.
  3. Augment: Combine the user’s question with the retrieved documents into a single prompt.
  4. Generate: Send the augmented prompt to the LLM to generate an answer based only on the provided documents.

Vector Databases and Embeddings

To find relevant documents, we use Embeddings. An embedding is an array of numbers that represents the meaning of a piece of text. We store these embeddings in a Vector Database (like Pinecone, Qdrant, or pgvector). When a user asks a question, we embed the question and ask the database to find the closest matching document embeddings.

Building a RAG Pipeline

Tools like LangChain or LlamaIndex provide the building blocks for RAG pipelines. You’ll need an embedding model (e.g., OpenAI’s text-embedding-ada-002), a vector store, and a generation model (e.g., GPT-4).