RAG Retrieval Augmented Generation: A Complete Practical Guide with Code Examples



RAG Retrieval Augmented Generation: A Complete Practical Guide with Code Examples

Quick Answer / TL;DR

RAG (Retrieval Augmented Generation) is a powerful AI framework that significantly improves Large Language Models (LLMs) by enabling them to access and integrate up-to-date, external information before generating responses. This process reduces "hallucinations" and provides highly accurate, context-specific answers. This guide will walk you through building a practical RAG system using Python, covering data indexing, retrieval with vector databases, and augmenting LLM prompts.

In the rapidly evolving landscape of Artificial Intelligence, Large Language Models (LLMs) have demonstrated incredible capabilities in understanding and generating human-like text. However, they often face limitations: they can “hallucinate” incorrect information, lack knowledge of recent events, or struggle to provide highly specific answers based on proprietary data. This is where RAG Retrieval Augmented Generation steps in as a game-changer. RAG combines the strengths of information retrieval systems with the generative power of LLMs, allowing models to fetch relevant external data before formulating a response, leading to more accurate, reliable, and contextually rich outputs.

This comprehensive guide will equip you with a practical understanding and the necessary code examples to build your own RAG system from the ground up.

What You Will Learn

Table of Contents

Understanding RAG: Why Retrieval Augmented Generation is Crucial for LLMs

LLMs are trained on vast amounts of data, but their knowledge is effectively frozen at the time of their last training update. This presents several challenges:

RAG Retrieval Augmented Generation directly addresses these issues. Instead of relying solely on its internal, static knowledge, a RAG system dynamically retrieves relevant information from an external, up-to-date, and potentially proprietary knowledge base. This retrieved context is then provided to the LLM, guiding it to generate a more accurate, informed, and traceable answer. It’s like giving the LLM an open-book exam every time it answers a question.

By integrating real-time data and specific domain knowledge, RAG transforms LLMs from general knowledge generators into powerful, context-aware assistants tailored to specific tasks and information sets. Let’s delve into how this magic happens.

The Core Components of a RAG System

A typical RAG architecture consists of four main stages: Data Ingestion and Indexing, Retrieval, Augmentation, and Generation.

1. Data Ingestion and Indexing

This initial phase involves preparing your custom data (e.g., documents, articles, databases) for efficient retrieval.

2. Retrieval

When a user poses a query, the retrieval phase kicks in:

3. Augmentation

This is where the “augmented” part of RAG comes into play.

4. Generation

Finally, the augmented prompt is sent to the LLM:

This structured approach ensures that the LLM is always informed by the most relevant and up-to-date information available in your knowledge base, making RAG Retrieval Augmented Generation an incredibly powerful technique. Now, let’s get our hands dirty and build one.

Setting Up Your Development Environment

Before diving into the code, ensure you have Python (3.8+) installed. We’ll primarily use langchain and ollama (or openai) for this tutorial, along with numpy and faiss-cpu for vector database functionality.

First, create a new project directory and set up a virtual environment:

1
2
3
4

mkdir rag-guide
cd rag-guide
python -m venv venv
source venv/bin/activate # On Windows: .\venv\Scripts\activate

Next, we’ll install the required libraries.

Building a Practical RAG System: Step-by-Step with Code

For this practical guide, we’ll build a RAG system that can answer questions based on a fictional company’s internal documentation. We’ll use langchain as our orchestration framework, Ollama for a local LLM (you can substitute with OpenAI or other models), and FAISS as our vector store.

Step 1: Install Necessary Libraries

We’ll install langchain for orchestrating the RAG pipeline, ollama for running a local LLM (e.g., llama2), faiss-cpu for our vector database, and pypdf for document loading.

1

pip install langchain langchain-community langchain-chroma faiss-cpu pypdf numpy python-dotenv

For the LLM, we’ll use Ollama. If you don’t have it installed, follow the instructions on ollama.com and then pull a model, e.g., llama2:

1

ollama run llama2

This will download and run the llama2 model locally.

Step 2: Prepare Your Knowledge Base

Let’s create a dummy document. In a real scenario, this would be your company’s PDFs, internal wikis, or other data sources. Create a file named company_policy.pdf (or company_policy.txt if you prefer, then adapt the loader). For simplicity, let’s assume it’s a txt file for direct text loading without pypdf for the example.

company_policy.txt:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

---
Policy Title: Remote Work Policy
Policy Version: 1.2
Last Updated: 2026-07-20
---

**1. Introduction**
This policy outlines the guidelines and expectations for employees working remotely at CodeCrux Innovations. Our goal is to provide flexibility while maintaining productivity and team cohesion.

**2. Eligibility**
All full-time employees are eligible for remote work, subject to manager approval and job role compatibility. Part-time employees may be eligible on a case-by-case basis. Employees must have a stable internet connection and a conducive home office environment.

**3. Remote Work Agreement**
An annual Remote Work Agreement must be signed by the employee and their manager, detailing work hours, communication expectations, and equipment responsibilities.

**4. Equipment and Support**
CodeCrux Innovations will provide essential equipment (laptop, monitor). Employees are responsible for maintaining a secure and functional workspace. IT support is available during standard business hours via our internal ticketing system.

**5. Communication and Collaboration**
Regular check-ins with managers and teams are mandatory. We utilize Slack for instant messaging, Google Meet for video conferencing, and Asana for project management.

Step 3: Create Embeddings and Index Data

Now, we’ll write a Python script to load this document, split it into chunks, create embeddings, and store them in a FAISS vector database.

Create rag_system.py:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

import os
from dotenv import load_dotenv
from langchain_community.document_loaders import TextLoader
from langchain_community.embeddings import OllamaEmbeddings
from langchain_community.vectorstores import FAISS
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chains import RetrievalQA
from langchain_community.llms import Ollama

# Load environment variables (for API keys if using OpenAI, etc.)
load_dotenv()

# --- Configuration ---
VECTOR_DB_PATH = "faiss_index"
DOCUMENT_PATH = "company_policy.txt"
EMBEDDING_MODEL = "nomic-embed-text" # A good local embedding model from Ollama
LLM_MODEL = "llama2" # A good local LLM from Ollama

def setup_rag_database(document_path: str, vector_db_path: str):
    """
    Loads documents, splits them, creates embeddings, and stores them in a FAISS vector database.
    """
    if os.path.exists(vector_db_path):
        print(f"Vector database already exists at {vector_db_path}. Loading existing index.")
        embeddings = OllamaEmbeddings(model=EMBEDDING_MODEL)
        vectorstore = FAISS.load_local(vector_db_path, embeddings, allow_dangerous_deserialization=True)
        return vectorstore

    print(f"Loading document from {document_path}...")
    loader = TextLoader(document_path)
    documents = loader.load()

    print("Splitting documents into chunks...")
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
    texts = text_splitter.split_documents(documents)

    print(f"Creating embeddings using {EMBEDDING_MODEL} and indexing into FAISS...")
    embeddings = OllamaEmbeddings(model=EMBEDDING_MODEL)
    vectorstore = FAISS.from_documents(texts, embeddings)
    vectorstore.save_local(vector_db_path)
    print(f"Vector database created and saved to {vector_db_path}")
    return vectorstore

if __name__ == "__main__":
    # Ensure embedding model is available via Ollama
    print(f"Ensuring Ollama embedding model '{EMBEDDING_MODEL}' is available...")
    os.system(f"ollama pull {EMBEDDING_MODEL}")
    print(f"Ensuring Ollama LLM model '{LLM_MODEL}' is available...")
    os.system(f"ollama pull {LLM_MODEL}")

    # Set up the vector database
    vectorstore = setup_rag_database(DOCUMENT_PATH, VECTOR_DB_PATH)

    # Initialize the Ollama LLM
    llm = Ollama(model=LLM_MODEL)

    # Create a RAG chain
    qa_chain = RetrievalQA.from_chain_type(
        llm=llm,
        chain_type="stuff", # 'stuff' combines all retrieved documents into one prompt
        retriever=vectorstore.as_retriever(),
        return_source_documents=True
    )

    print("\nRAG System Ready! Ask a question about the company policy.")
    print("Type 'exit' to quit.")

    while True:
        query = input("\nYour question: ")
        if query.lower() == 'exit':
            break

        print("Searching and generating response...")
        result = qa_chain.invoke({"query": query})

        print("\n--- Answer ---")
        print(result["result"])
        print("\n--- Sources ---")
        for doc in result["source_documents"]:
            print(f"- {doc.metadata.get('source', 'Unknown source')}: '{doc.page_content[:100]}...'")

Before running rag_system.py, make sure you have pulled nomic-embed-text and llama2 using Ollama:

1
2

ollama pull nomic-embed-text
ollama pull llama2

Now, run the script:

1

python rag_system.py

The script will first download the Ollama models (if not present), then load your company_policy.txt, chunk it, create embeddings, and save the FAISS index locally. If the index already exists, it will load it.

Step 4: Implement the Retrieval Mechanism

In the rag_system.py script, vectorstore.as_retriever() does the heavy lifting for retrieval. When qa_chain.invoke() is called, LangChain:

  1. Takes your query.
  2. Embeds the query using the OllamaEmbeddings model.
  3. Performs a similarity search against the FAISS vector store.
  4. Retrieves the top k (default often 4) most relevant document chunks.

This seamlessly integrates the retrieval step into the overall RetrievalQA chain.

Step 5: Augment the LLM Prompt and Generate Response

The RetrievalQA.from_chain_type with chain_type="stuff" handles the augmentation and generation:

  1. Augmentation: It takes the original query and the source_documents (retrieved chunks). It then “stuffs” these into a single prompt template, typically structured like: 
    1
2
3
4
5
6
7
8

    "Use the following pieces of context to answer the user's question.
If you don't know the answer, just say that you don't know, don't try to make up an answer.

Context:
{context}

Question: {question}
Helpful Answer:"

    where {context} is populated by the page_content of the retrieved documents and {question} is your original query.

  2. Generation: This constructed prompt is sent to the Ollama LLM (llama2 in our case), which then generates the final answer based on the provided context. The return_source_documents=True allows us to inspect which documents were used.

Example interaction:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

Your question: What are the eligibility requirements for remote work?
Searching and generating response...

--- Answer ---
All full-time employees are eligible for remote work, subject to manager approval and job role compatibility. Part-time employees may be eligible on a case-by-case basis. Employees must also have a stable internet connection and a conducive home office environment.

--- Sources ---
- company_policy.txt: 'All full-time employees are eligible for remote work, subject to manager approval and job role compatibility.'...

Your question: What did I eat for breakfast today?
Searching and generating response...

--- Answer ---
I apologize, but I cannot answer that question as the provided company policy document does not contain any information about your breakfast.

--- Sources ---
- company_policy.txt: 'This policy outlines the guidelines and expectations for employees working remotely at CodeCrux Innovations.'...

Notice how the RAG system provides an accurate answer when the information is available in its knowledge base, and gracefully indicates it doesn’t know when the question is out of scope. This demonstrates the power of RAG Retrieval Augmented Generation in grounding LLM responses.

Real-World Use Cases for Retrieval Augmented Generation

RAG Retrieval Augmented Generation isn’t just a theoretical concept; it’s being deployed across various industries to solve critical problems:

The ability of RAG to combine the breadth of LLMs with the depth of specific, up-to-date knowledge makes it invaluable for applications where accuracy and trustworthiness are paramount.

Advanced RAG Techniques and Optimizations

While our basic RAG system is functional, there are many ways to enhance its performance and robustness:

By exploring these advanced techniques, you can build even more intelligent and reliable RAG Retrieval Augmented Generation systems tailored to complex challenges.

FAQ: RAG Retrieval Augmented Generation

What is RAG (Retrieval Augmented Generation) and how does it work?

RAG is an AI framework that enhances Large Language Models (LLMs) by allowing them to retrieve relevant information from an external knowledge base before generating a response. It works by first converting a user query into an embedding, searching a vector database for semantically similar document chunks, and then feeding these retrieved chunks along with the original query to an LLM, which uses this context to formulate a more accurate and informed answer.

Why is RAG important for LLMs?

RAG is crucial because it addresses key limitations of standalone LLMs: it mitigates “hallucinations” (generating incorrect information), provides access to up-to-date or proprietary data beyond the LLM’s training cutoff, and enables LLMs to offer more specific, verifiable, and contextually relevant answers.

What are the main components of a RAG system?

The main components of a RAG system are: 1. Data Ingestion & Indexing (loading, chunking, embedding, and storing data in a vector database), 2. Retrieval (finding relevant data chunks based on query similarity), 3. Augmentation (combining the query and retrieved context into a prompt), and 4. Generation (the LLM producing an answer using the augmented prompt).

Can I use RAG with my own custom data?

Yes, absolutely! One of RAG’s primary benefits is its ability to integrate custom, proprietary, or domain-specific data. You can ingest documents like PDFs, internal reports, web pages, or database records into your RAG system’s vector database, making that information accessible to your LLM.

What are some real-world applications of RAG?

RAG is used in diverse applications such as customer support chatbots (answering product questions), enterprise search (finding internal company policies), legal research (retrieving case precedents), healthcare information systems (accessing medical literature), and educational tools (providing specific course material explanations).

Further Reading

  1. Original RAG Paper: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks by Lewis et al. (2020) - The foundational paper that introduced the RAG concept.
  2. LangChain Documentation on RAG: LangChain RAG Overview - Excellent resource for understanding various RAG patterns and implementations with LangChain.
  3. Ollama Documentation: Ollama.com - Learn more about running open-source LLMs and embedding models locally.

Conclusion

RAG Retrieval Augmented Generation stands as a pivotal advancement in the field of AI, transforming how Large Language Models interact with information. By providing LLMs with a dynamic window into external, real-time, and domain-specific knowledge bases, RAG effectively addresses the critical challenges of accuracy, relevance, and traceability. The practical guide and code examples presented here offer a solid foundation for you to start building intelligent applications that leverage the power of RAG. As you continue your journey, exploring advanced techniques will unlock even greater potential, allowing you to create truly authoritative and contextually aware AI solutions.

Ready to dive deeper into AI and machine learning? Explore more cutting-edge tutorials and guides on CodeCrux Innovations’ blog or check out our services for building robust AI systems.



Empower Your Business with Our Expert Solutions

Unlock the full potential of your projects with our professional services!

Get Started Today