GraphRAG Tutorial: Build Knowledge Graph Retrieval for Complex Questions



GraphRAG Tutorial: Build Knowledge Graph Retrieval for Complex Questions

Quick Answer / TL;DR

This tutorial provides a hands-on guide to building a GraphRAG system. GraphRAG enhances traditional Retrieval-Augmented Generation (RAG) by using knowledge graphs to retrieve highly structured and contextual information, enabling Large Language Models (LLMs) to answer complex, multi-hop questions with greater accuracy and explainability. You'll learn how to construct a knowledge graph, perform graph-aware retrieval, and integrate it with LLMs for superior question answering.

Large Language Models (LLMs) have revolutionized how we interact with information, but they often struggle with complex, multi-hop questions that require synthesizing information from various sources or understanding nuanced relationships. Traditional Retrieval-Augmented Generation (RAG) helps ground LLMs in external data, but even RAG can fall short when the required context is buried in unstructured text or demands intricate logical inference. This is where GraphRAG shines, leveraging the power of knowledge graphs to provide structured, interconnected context for superior question answering.

In this comprehensive tutorial, we’ll walk through the process of building a GraphRAG system from the ground up. You’ll discover how knowledge graphs can unlock deeper insights, enhance retrieval accuracy, and provide explainability for LLM responses, especially for those challenging questions that stump vanilla RAG.

What You Will Learn

Table of Contents

  1. Understanding GraphRAG: Why Knowledge Graphs for RAG?
  2. Setting Up Your GraphRAG Environment
  3. Data Ingestion and Knowledge Graph Construction
  4. Implementing Graph-Aware Retrieval
  5. Integrating with Large Language Models for Complex Question Answering
  6. Frequently Asked Questions (FAQ)
  7. Further Reading

Understanding GraphRAG: Why Knowledge Graphs for RAG?

Vanilla RAG systems typically rely on vector databases to store chunks of unstructured text. When a query comes in, the system retrieves top-k similar chunks and feeds them to the LLM. While effective for many direct questions, this approach has limitations:

GraphRAG addresses these challenges by replacing or augmenting the vector store with a knowledge graph. A knowledge graph stores information as a network of interconnected entities (nodes) and relationships (edges). This structured representation offers several advantages:

By transforming unstructured data into a rich, semantic graph, GraphRAG empowers LLMs to navigate a landscape of interconnected knowledge, leading to more accurate, relevant, and transparent responses.

Now that we understand the ‘why,’ let’s prepare our environment for building this powerful system.

Setting Up Your GraphRAG Environment

Before we dive into building our knowledge graph and integrating it with an LLM, we need to set up our development environment. We’ll primarily use Python for this tutorial.

1. Prerequisites

Ensure you have Python 3.9+ and pip installed.

2. Install Necessary Libraries

We’ll need libraries for interacting with LLMs (LangChain), a graph database driver (Neo4j), and potentially a local LLM or API client.

1

pip install langchain langchain-community langchain-openai neo4j tiktoken python-dotenv

3. Set Up a Graph Database (Neo4j)

For this tutorial, we’ll use Neo4j, a popular graph database. You have a few options:

Option A: Neo4j AuraDB (Cloud Service)

This is the easiest for production or cloud-based development. Sign up for a free tier at Neo4j AuraDB. Once created, you’ll get connection details (URI, username, password).

Option B: Local Docker Container

For local development, Docker is highly recommended.

1

docker run --name neo4j-graphrag -p 7474:7474 -p 7687:7687 -e NEO4J_AUTH=neo4j/password -e NEO4J_apoc_export_file_enabled=true -e NEO4J_apoc_import_file_enabled=true -e NEO4J_apoc_import_file_use__neo4j__config=true -e NEO4J_dbms_security_procedures_unrestricted=apoc.* -e NEO4JLABS_PLUGINS='["apoc", "graph-data-science"]' neo4j:5.16

This command starts a Neo4j 5.x container with APOC and GDS plugins, which are useful for various graph operations.

4. Environment Variables

Create a .env file in your project root to store sensitive API keys and database credentials.

OPENAI_API_KEY="your_openai_api_key_here"
NEO4J_URI="bolt://localhost:7687" # or your AuraDB URI
NEO4J_USERNAME="neo4j"
NEO4J_PASSWORD="password"

Then, load them in your Python script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

import os
from dotenv import load_dotenv

load_dotenv()

openai_api_key = os.getenv("OPENAI_API_KEY")
neo4j_uri = os.getenv("NEO4J_URI")
neo4j_username = os.getenv("NEO4J_USERNAME")
neo4j_password = os.getenv("NEO4J_PASSWORD")

if not all([openai_api_key, neo4j_uri, neo4j_username, neo4j_password]):
    raise ValueError("Missing one or more environment variables. Check your .env file.")

print("Environment variables loaded successfully.")

With our environment ready, the next crucial step is getting our data into a graph format that Neo4j can understand.

Data Ingestion and Knowledge Graph Construction

The heart of GraphRAG lies in transforming unstructured text into a structured knowledge graph. This process typically involves identifying entities (nouns, concepts) and the relationships between them (verbs, predicates).

1. Choose a Sample Dataset

For this tutorial, let’s consider a hypothetical dataset of internal company project documentation. Each document describes projects, team members, technologies used, and project outcomes.

Example Document Snippets:

1
2
3

Document 1: "Project Phoenix, led by Alice Johnson, aimed to develop a new AI-powered recommendation engine. Bob Smith and Carol White were key contributors. It utilized Python and TensorFlow, and resulted in a 15% increase in user engagement."

Document 2: "The Atlas Initiative, overseen by David Lee, focused on migrating our legacy systems to a cloud-native architecture using Kubernetes and Go. Alice Johnson also consulted on its security aspects. This project achieved a 20% cost reduction."

2. Extract Entities and Relationships

We can use LLMs or specialized NLP tools to extract structured data from these snippets. LangChain provides excellent tools for this.

Let’s define a schema for our knowledge graph. We might want entities like Person, Project, Technology, Outcome, and relationships like LED_BY, CONTRIBUTED_TO, USED_TECHNOLOGY, ACHIEVED_OUTCOME.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

from langchain_openai import ChatOpenAI
from langchain.chains import create_tagging_chain_pydantic
from langchain.graphs import Neo4jGraph
from pydantic import BaseModel, Field
from typing import List, Optional

# Initialize LLM
llm = ChatOpenAI(model="gpt-4o", temperature=0, api_key=openai_api_key)

# Connect to Neo4j
graph = Neo4jGraph(url=neo4j_uri, username=neo4j_username, password=neo4j_password)

# Define our knowledge graph schema for extraction
class Person(BaseModel):
    name: str = Field(description="Name of the person")

class Project(BaseModel):
    name: str = Field(description="Name of the project")
    description: Optional[str] = Field(description="Brief description of the project")

class Technology(BaseModel):
    name: str = Field(description="Name of the technology or tool")

class Outcome(BaseModel):
    description: str = Field(description="Description of the project outcome")

class Relationship(BaseModel):
    source_entity_type: str
    source_entity_name: str
    relation_type: str
    target_entity_type: str
    target_entity_name: str

class KnowledgeGraphSchema(BaseModel):
    people: List[Person] = Field(default_factory=list)
    projects: List[Project] = Field(default_factory=list)
    technologies: List[Technology] = Field(default_factory=list)
    outcomes: List[Outcome] = Field(default_factory=list)
    relationships: List[Relationship] = Field(default_factory=list)

# Create a tagging chain for extracting structured data
# Note: LangChain offers more advanced GraphDocument and entity extraction tools,
# but this demonstrates a simpler Pydantic-based approach.
tagging_chain = create_tagging_chain_pydantic(KnowledgeGraphSchema, llm)

# Example text
text1 = "Project Phoenix, led by Alice Johnson, aimed to develop a new AI-powered recommendation engine. Bob Smith and Carol White were key contributors. It utilized Python and TensorFlow, and resulted in a 15% increase in user engagement."
text2 = "The Atlas Initiative, overseen by David Lee, focused on migrating our legacy systems to a cloud-native architecture using Kubernetes and Go. Alice Johnson also consulted on its security aspects. This project achieved a 20% cost reduction."

# Process text and extract entities/relationships
extracted_data1 = tagging_chain.invoke({"input": text1})["parsed"]
extracted_data2 = tagging_chain.invoke({"input": text2})["parsed"]

# You would typically loop through many documents and aggregate
print("Extracted from Document 1:", extracted_data1)
print("\nExtracted from Document 2:", extracted_data2)

The output for extracted_data1 might look something like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

Extracted from Document 1: KnowledgeGraphSchema(
    people=[Person(name='Alice Johnson'), Person(name='Bob Smith'), Person(name='Carol White')],
    projects=[Project(name='Project Phoenix', description='new AI-powered recommendation engine')],
    technologies=[Technology(name='Python'), Technology(name='TensorFlow')],
    outcomes=[Outcome(description='15% increase in user engagement')],
    relationships=[
        Relationship(source_entity_type='Project', source_entity_name='Project Phoenix', relation_type='LED_BY', target_entity_type='Person', target_entity_name='Alice Johnson'),
        Relationship(source_entity_type='Person', source_entity_name='Bob Smith', relation_type='CONTRIBUTED_TO', target_entity_type='Project', target_entity_name='Project Phoenix'),
        Relationship(source_entity_type='Person', source_entity_name='Carol White', relation_type='CONTRIBUTED_TO', target_entity_type='Project', target_entity_name='Project Phoenix'),
        Relationship(source_entity_type='Project', source_entity_name='Project Phoenix', relation_type='USED_TECHNOLOGY', target_entity_type='Technology', target_entity_name='Python'),
        Relationship(source_entity_type='Project', source_entity_name='Project Phoenix', relation_type='USED_TECHNOLOGY', target_entity_type='Technology', target_entity_name='TensorFlow'),
        Relationship(source_entity_type='Project', source_entity_name='Project Phoenix', relation_type='RESULTED_IN', target_entity_type='Outcome', target_entity_name='15% increase in user engagement')
    ]
)

3. Loading into the Graph Database

Now, we’ll convert the extracted Pydantic objects into Cypher queries to create nodes and relationships in Neo4j.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

def ingest_extracted_data(extracted_data: KnowledgeGraphSchema, graph: Neo4jGraph):
    for person in extracted_data.people:
        graph.query(f"MERGE (p:Person {{name: '{person.name}'}})")
    for project in extracted_data.projects:
        graph.query(f"MERGE (pr:Project {{name: '{project.name}'}}) SET pr.description = '{project.description}'")
    for tech in extracted_data.technologies:
        graph.query(f"MERGE (t:Technology {{name: '{tech.name}'}})")
    for outcome in extracted_data.outcomes:
        graph.query(f"MERGE (o:Outcome {{description: '{outcome.description}'}})")

    for rel in extracted_data.relationships:
        # Example for a relationship: (Source)-[RELATION]->(Target)
        # We need to ensure the nodes exist before creating the relationship.
        # This example uses a simplified merge pattern.
        source_label = rel.source_entity_type
        target_label = rel.target_entity_type
        relation_type = rel.relation_type

        # Construct the Cypher query carefully to handle different entity types
        cypher_query = f"""
        MATCH (s:{source_label}) WHERE s.name = '{rel.source_entity_name}' OR s.description = '{rel.source_entity_name}'
        MATCH (t:{target_label}) WHERE t.name = '{rel.target_entity_name}' OR t.description = '{rel.target_entity_name}'
        MERGE (s)-[:{relation_type}]->(t)
        """
        # Note: The above MATCH for name/description is a simplification.
        # In a real system, you'd use a unique ID or more robust matching.
        graph.query(cypher_query)
    print(f"Ingested {len(extracted_data.relationships)} relationships.")

# Ingest data
ingest_extracted_data(extracted_data1, graph)
ingest_extracted_data(extracted_data2, graph)

# Verify by running a simple query
# print(graph.query("MATCH (p:Person)-[r]->(pr:Project) RETURN p.name, type(r), pr.name LIMIT 5"))

After running this, your Neo4j database will contain a network of interconnected nodes representing people, projects, technologies, and outcomes, along with their explicit relationships.

Once our knowledge graph is populated, we can leverage its power for intelligent retrieval, moving beyond simple keyword matching to contextual graph traversal.

Implementing Graph-Aware Retrieval

With our knowledge graph populated, the next step is to retrieve relevant information based on user queries. Graph-aware retrieval goes beyond vector similarity to find connected facts and paths within the graph that directly address a question.

LangChain offers Neo4jGraph as a retriever, which can interpret a natural language query and translate it into Cypher queries to traverse the graph.

1. Connecting LangChain to the Neo4j Graph

We’ve already initialized Neo4jGraph in the previous section. Let’s make it more explicit as a retriever:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

from langchain_community.graphs import Neo4jGraph
from langchain.chains import GraphCypherQAChain
from langchain_openai import ChatOpenAI

# Re-initialize graph if not already
graph = Neo4jGraph(url=neo4j_uri, username=neo4j_username, password=neo4j_password)

# Define the schema of the graph to help the LLM generate accurate Cypher queries
# This is crucial for GraphCypherQAChain to understand your graph structure.
# You can auto-generate this using graph.get_schema, but for a clear tutorial,
# we'll use a simplified version based on our ingestion.
graph.refresh_schema() # Get the latest schema from the graph

# The LLM will use this schema to understand which labels and relationships exist.
# Example schema content after refresh:
# Nodes: Person, Project, Technology, Outcome
# Relationships: LED_BY, CONTRIBUTED_TO, USED_TECHNOLOGY, RESULTED_IN, CONSULTED_ON

2. Crafting Graph Retrieval Queries

The GraphCypherQAChain uses an LLM to translate natural language questions into Cypher queries, executes those queries against the Neo4j database, and then uses another LLM call to synthesize the results into a human-readable answer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

# Initialize the LLM for generating Cypher queries and answering
llm_cypher = ChatOpenAI(model="gpt-4o", temperature=0, api_key=openai_api_key)

# Create the GraphCypherQAChain
cypher_chain = GraphCypherQAChain.from_llm(
    graph=graph,
    llm=llm_cypher,
    verbose=True # Set to True to see the generated Cypher queries and graph responses
)

# Example: Complex Multi-Hop Question
complex_question = "What technologies were used in projects Alice Johnson was involved in, and what outcomes did those projects achieve?"

print(f"\nQuerying GraphRAG for: '{complex_question}'")
response = cypher_chain.invoke({"query": complex_question})

print("\nGraphRAG Response:")
print(response["result"])

When verbose=True, you’ll see the LLM’s generated Cypher query, the raw results from Neo4j, and then the final synthesized answer. A plausible Cypher query generated by the LLM might be:

1
2
3
4
5

MATCH (p:Person)-[:LED_BY|CONTRIBUTED_TO|CONSULTED_ON]->(proj:Project)
WHERE p.name = "Alice Johnson"
MATCH (proj)-[:USED_TECHNOLOGY]->(tech:Technology)
MATCH (proj)-[:RESULTED_IN]->(o:Outcome)
RETURN proj.name, COLLECT(DISTINCT tech.name) AS technologies_used, COLLECT(DISTINCT o.description) AS outcomes_achieved

And the synthesized response from the LLM based on the graph results might be:

1

"Alice Johnson was involved in 'Project Phoenix', which utilized Python and TensorFlow and resulted in a 15% increase in user engagement. She also consulted on 'The Atlas Initiative', which used Kubernetes and Go, leading to a 20% cost reduction."

This clearly demonstrates the power of GraphRAG in answering questions that require traversing multiple relationships. The LLM didn’t “guess”; it used the explicitly retrieved structured information from the graph.

Retrieving context is only half the battle; integrating it effectively with an LLM for final answer generation is the critical next step.

Integrating with Large Language Models for Complex Question Answering

After retrieving relevant subgraphs or structured data, the final step in the GraphRAG pipeline is to present this information to an LLM in a way that allows it to generate accurate, comprehensive, and well-grounded answers.

1. Feeding Graph-Retrieved Context to the LLM

The GraphCypherQAChain we used earlier already handles the integration: it takes the graph query results and feeds them into the LLM for final answer generation. The prompt structure inside GraphCypherQAChain is designed to guide the LLM to synthesize these results.

However, for more custom scenarios, you might retrieve the graph data explicitly and then construct your own prompt.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

# --- Hypothetical Custom Retrieval (if not using GraphCypherQAChain directly) ---
# For demonstration, let's assume we have raw graph results:
raw_graph_results = """
[
    {"project_name": "Project Phoenix", "technologies": ["Python", "TensorFlow"], "outcomes": ["15% increase in user engagement"]},
    {"project_name": "Atlas Initiative", "technologies": ["Kubernetes", "Go"], "outcomes": ["20% cost reduction"]}
]
"""
# In a real scenario, this would come from graph.query() or a more sophisticated graph traversal.
# --- End Hypothetical Custom Retrieval ---

# Define a custom prompt template to include the graph context
prompt_template = ChatPromptTemplate.from_messages(
    [
        ("system", "You are an expert assistant for company knowledge. Answer the question based ONLY on the provided context. If the answer is not in the context, state that you don't have enough information."),
        ("user", "Context from knowledge graph: {graph_context}\n\nQuestion: {question}")
    ]
)

# Initialize the LLM for final answer generation
llm_answer_gen = ChatOpenAI(model="gpt-4o", temperature=0.2, api_key=openai_api_key)

# Create a RAG chain
custom_rag_chain = (
    {"graph_context": lambda x: x["graph_context"], "question": RunnablePassthrough()}
    | prompt_template
    | llm_answer_gen
    | StrOutputParser()
)

# Simulate answering the complex question with custom RAG
question_for_custom_rag = "What technologies were used in projects Alice Johnson was involved in, and what outcomes did those projects achieve?"

print(f"\nAnswering with Custom RAG for: '{question_for_custom_rag}'")
final_answer_custom = custom_rag_chain.invoke({
    "graph_context": raw_graph_results, # Use the assumed raw graph results
    "question": question_for_custom_rag
})

print("\nCustom RAG Final Answer:")
print(final_answer_custom)

The output from custom_rag_chain should be similar to the GraphCypherQAChain output, demonstrating how the LLM leverages the structured graph_context to formulate a precise answer.

2. Prompt Engineering for GraphRAG

Effective prompt engineering is crucial. When presenting graph data to an LLM, consider:

3. Evaluating Responses

Evaluating GraphRAG responses involves:

Tools like Ragas can help automate some aspects of RAG evaluation, often requiring ground truth answers or human annotations for complex queries. The inherent structure of knowledge graphs, however, often simplifies manual verification.

To solidify your understanding, let’s look at common questions developers encounter.

Frequently Asked Questions (FAQ)

Further Reading

  1. LangChain Documentation on Graph Integrations: Dive deeper into LangChain’s capabilities for interacting with various graph databases and building graph-aware chains. LangChain Graph Integrations
  2. Neo4j Developer Resources: Explore comprehensive guides, tutorials, and documentation for Neo4j, including advanced Cypher queries and data modeling. Neo4j Developer Resources
  3. The “Knowledge Graphs and LLMs” Paper Series: For a deeper academic understanding of the synergy between knowledge graphs and large language models, search for recent research papers on the topic.

Conclusion

This GraphRAG tutorial has guided you through the critical steps of building an advanced RAG system that leverages the power of knowledge graphs. By transforming unstructured data into an interconnected network of facts, we’ve enabled LLMs to perform superior knowledge graph retrieval, tackling complex, multi-hop questions with unprecedented accuracy and explainability. From setting up your environment and constructing the graph to implementing intelligent retrieval and integrating with LLMs, you now have a solid foundation to build sophisticated AI applications.

The ability to provide LLMs with structured, contextualized information is a game-changer for enterprise knowledge management, scientific discovery, and decision support systems. As LLMs continue to evolve, the combination of their generative power with the precise, structured reasoning capabilities of knowledge graphs will undoubtedly lead to even more intelligent and reliable AI systems.

Ready to build more robust AI solutions? Explore our services for custom AI/ML development and advanced RAG implementations. Contact CodeCrux Today!



Empower Your Business with Our Expert Solutions

Unlock the full potential of your projects with our professional services!

Get Started Today