How to Apply RAG Chatbot with LangChain + Ollama
In this post, you'll learn how to build a powerful RAG (Retrieval-Augmented Generation) chatbot using LangChain and Ollama. We'll also show the full flow of how to add documents into your agent dynamically!
Let's go step-by-step.
What is RAG Chatbot?
RAG stands for Retrieval-Augmented Generation. Instead of a chatbot replying only from what it "knows" internally, it first retrieves relevant documents and then generates an intelligent, customized answer.
This approach makes your chatbot:
- More accurate
- Up-to-date with external knowledge
- Better at answering domain-specific questions
Tools We'll Use
- LangChain: Framework for chaining LLMs with retrieval, tools, agents.
- Ollama: Local LLM runner (models like
Mistral
,Llama3
, etc.). - FAISS: Local vector search for fast document retrieval.
Step 1: Setup
First, install the libraries:
pip install langchain ollama faiss-cpu
Make sure you have Ollama installed and running:
- https://ollama.com/
- Download a model (example:
mistral
,llama3
,phi3
, etc.)
Step 2: Basic RAG Chatbot Code
from langchain_community.embeddings import OllamaEmbeddings
from langchain_community.llms import Ollama
from langchain.vectorstores import FAISS
from langchain.chains import RetrievalQA
from langchain.text_splitter import CharacterTextSplitter
from langchain.docstore.document import Document
# 1. Documents
my_docs = [
Document(page_content="Python is a programming language created by Guido van Rossum."),
Document(page_content="LangChain helps build applications powered by language models."),
Document(page_content="The capital of Thailand is Bangkok."),
]
# 2. Split Documents
splitter = CharacterTextSplitter(chunk_size=500, chunk_overlap=50)
split_docs = splitter.split_documents(my_docs)
# 3. Embedding
embedder = OllamaEmbeddings(model="nomic-embed-text")
# 4. FAISS Vector Store
vectorstore = FAISS.from_documents(split_docs, embedder)
# 5. Retriever
retriever = vectorstore.as_retriever()
# 6. LLM (Ollama)
llm = Ollama(model="mistral")
# 7. RetrievalQA Chain
rag_chain = RetrievalQA.from_chain_type(
llm=llm,
chain_type="stuff",
retriever=retriever,
return_source_documents=True
)
# 8. Ask a Question
query = "Who created Python?"
result = rag_chain({"query": query})
print("Answer:", result["result"])
print("Sources:", result["source_documents"])
Step 3: Add New Documents to Your Agent
You can dynamically add documents to your chatbot without restarting everything.
Here is the conceptual workflow in Mermaid.js:
graph TD
A[User Uploads New Document] --> B[Split into Chunks]
B --> C[Embed Chunks]
C --> D[Add Embeddings to Vectorstore]
D --> E[Retriever Automatically Updated]
E --> F[Chatbot Now Uses New Knowledge]
Python Code to Add New Document:
# Suppose you get a new document
new_doc = Document(page_content="Django is a popular Python web framework.")
# 1. Split
new_splits = splitter.split_documents([new_doc])
# 2. Embed
new_vectors = embedder.embed_documents([doc.page_content for doc in new_splits])
# 3. Add to FAISS
vectorstore.add_documents(new_splits)
# Now your retriever has fresh knowledge instantly!
Conclusion
Building a RAG chatbot with LangChain + Ollama is powerful and flexible. You can:
- Control your own models (no external APIs)
- Add new knowledge live
- Build super domain-specific chatbots
In production, you can scale this by connecting to:
- PDF loaders
- Website scrapers
- Database retrievers
The future is open-source, private, and customizable.
Stay tuned for Part 2 where we'll make the chatbot stream responses and keep conversation memory! 🚀
Get in Touch with us
Related Posts
- 下一个前沿:面向富裕人群的数字私人俱乐部
- The Next Frontier: A Digital Private Club for the Affluent
- Thinking Better with Code: Using Mathematical Shortcuts to Master Large Codebases
- Building the Macrohard of Today: AI Agents Platform for Enterprises
- Build Vue.js Apps Smarter with Aider + IDE Integration
- Yo Dev! Here’s How I Use AI Tools Like Codex CLI and Aider to Speed Up My Coding
- Working With AI in Coding the Right Way
- How to Select the Right LLM Model: Instruct, MLX, 8-bit, and Embedding Models
- How to Use Local LLM Models in Daily Work
- How to Use Embedding Models with LLMs for Smarter AI Applications
- Smart Vision System for Continuous Material Defect Detection
- Building a Real-Time Defect Detector with Line-Scan + ML (Reusable Playbook)
- How to Read Source Code: Frappe Framework Sample
- Interface-Oriented Design: The Foundation of Clean Architecture
- Understanding Anti-Drone Systems: Architecture, Hardware, and Software
- RTOS vs Linux in Drone Systems: Modern Design, Security, and Rust for Next-Gen Drones
- Why Does Spring Use So Many Annotations? Java vs. Python Web Development Explained
- From Django to Spring Boot: A Practical, Visual Guide for Web Developers
- How to Build Large, Maintainable Python Systems with Clean Architecture: Concepts & Real-World Examples
- Why Test-Driven Development Makes Better Business Sense