Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Latest commit

Β 

History

3 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 

Repository files navigation

RAG using Llama2, LangChain, and ChromaDB

A comprehensive Retrieval-Augmented Generation (RAG) implementation combining Llama2 language model with LangChain framework and ChromaDB vector database for advanced document question-answering capabilities.

πŸ“‹ Overview

This project demonstrates a complete RAG pipeline using:

  • Llama2: Open-source large language model for generation
  • LangChain: Framework for building LLM applications
  • ChromaDB: Vector database for semantic search
  • Local Models: Fully local setup without external APIs (optional)

Perfect for building private, offline-capable document QA systems.

🎯 Objectives

  • Implement complete RAG system
  • Use open-source models (Llama2)
  • Demonstrate LangChain capabilities
  • Leverage ChromaDB for vector storage
  • Build offline-capable applications
  • Create scalable document indexing
  • Enable multi-document QA

πŸ—‚οΈ Project Structure

RAG-using-llama2-langchain-and-chromadb/
β”œβ”€β”€ RAG using llama2 langchain and chromadb.ipynb  # Main notebook
β”œβ”€β”€ README.md                                       # Documentation
└── data/                                           # Documents (optional)
    β”œβ”€β”€ documents/
    β”œβ”€β”€ chroma_db/
    └── models/

πŸ› οΈ Technologies & Libraries

  • LLM: Llama2 (via Ollama or HuggingFace)

  • Framework: LangChain

  • Vector DB: ChromaDB

  • Embeddings: Sentence-Transformers or HuggingFace

  • Document Processing:

    • PyPDF2
    • LangChain Document Loaders
    • TextSplitter
  • Utilities: NumPy, Pandas

πŸ“Š Key Features

  • Complete RAG Pipeline:

    • Document ingestion
    • Text chunking and splitting
    • Embedding generation
    • Vector storage
    • Semantic search
    • Answer generation
  • LangChain Integration:

    • Document loaders
    • Text splitters
    • Vector stores
    • LLM chains
    • Memory management
    • Prompt templates
  • ChromaDB Features:

    • Persistent storage
    • Similarity search
    • Metadata filtering
    • Collection management
    • Scalable indexing
  • Llama2 Support:

    • Local inference (via Ollama)
    • HuggingFace integration
    • GGML quantized models
    • CPU and GPU support

πŸš€ Getting Started

Prerequisites

  • Python 3.8+
  • Ollama (for local Llama2) - Install
  • 8GB+ RAM (16GB+ for Llama2)
  • Document files for indexing

Installation

  1. Clone the repository:

    git clone <repository-url>
    cd RAG-using-llama2-langchain-and-chromadb
  2. Install dependencies:

    pip install -r requirements.txt
  3. Set up Llama2 locally (optional):

    # Install Ollama from ollama.ai
    ollama pull llama2
    ollama serve
  4. Open the notebook:

    jupyter notebook "RAG using llama2 langchain and chromadb.ipynb"

πŸ“ˆ RAG Pipeline Architecture

Document Upload
    ↓
Load Documents (PDF, TXT, etc.)
    ↓
Split Text into Chunks
    ↓
Generate Embeddings
    ↓
Store in ChromaDB
    ↓
User Query
    ↓
Generate Query Embedding
    ↓
Semantic Search in ChromaDB
    ↓
Retrieve Top-K Documents
    ↓
Build Prompt with Context
    ↓
Llama2 Generation
    ↓
Answer + Source Citation

πŸ’Ύ Configuration

ChromaDB Setup

import chromadb
from langchain.vectorstores import Chroma

# Initialize ChromaDB
chroma_client = chromadb.Client()
collection = chroma_client.create_collection(
    name="documents",
    metadata={"hnsw:space": "cosine"}
)

# Or use persistent client
persistent_client = chromadb.PersistentClient(
    path="/path/to/data"
)

Llama2 via Ollama

from langchain.llms import Ollama

# Use local Ollama
llm = Ollama(model="llama2", base_url="http://localhost:11434")

# Or via HuggingFace
from langchain.llms import HuggingFacePipeline
llm = HuggingFacePipeline(model_name="meta-llama/Llama-2-7b-chat-hf")

LangChain Setup

from langchain.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import Chroma
from langchain.chains import RetrievalQA

# Load documents
loader = PyPDFLoader("document.pdf")
documents = loader.load()

# Split text
splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200
)
texts = splitter.split_documents(documents)

# Create embeddings
embeddings = HuggingFaceEmbeddings()

# Create vector store
vectorstore = Chroma.from_documents(
    texts,
    embeddings,
    persist_directory="./chroma_db"
)

# Create QA chain
qa = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vectorstore.as_retriever()
)

πŸ“ Usage Examples

Basic Question Answering

# Simple query
question = "What are the main topics discussed?"
answer = qa.run(question)
print(f"Answer: {answer}")

Advanced Configuration

from langchain.chains import RetrievalQA
from langchain.prompts import PromptTemplate

# Custom prompt
template = """Use the provided context to answer the question.
Context: {context}
Question: {question}
Answer: """

prompt = PromptTemplate(
    template=template,
    input_variables=["context", "question"]
)

# Create QA chain with custom prompt
qa = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="refine",  # Better for complex docs
    retriever=vectorstore.as_retriever(k=5),
    chain_type_kwargs={
        "prompt": prompt,
        "verbose": True
    }
)

# Query
result = qa({"query": "What is the document about?"})
print(result)

Multi-Document Indexing

from langchain.document_loaders import DirectoryLoader

# Load all PDFs from directory
loader = DirectoryLoader(
    "documents/",
    glob="**/*.pdf",
    loader_cls=PyPDFLoader
)
documents = loader.load()

# Process and store
texts = splitter.split_documents(documents)
vectorstore = Chroma.from_documents(
    texts,
    embeddings,
    persist_directory="./chroma_db"
)

Source Citation

from langchain.chains import RetrievalQAWithSourcesChain

# Create QA chain with sources
qa_with_sources = RetrievalQAWithSourcesChain.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vectorstore.as_retriever()
)

# Query returns answer with sources
result = qa_with_sources("What is the main topic?")
print(f"Answer: {result['answer']}")
print(f"Sources: {result['sources']}")

βš™οΈ Key Parameters

# Text splitting
CHUNK_SIZE = 1000
CHUNK_OVERLAP = 200

# Embedding
EMBEDDING_MODEL = "sentence-transformers/all-MiniLM-L6-v2"

# Retrieval
TOP_K = 5
SIMILARITY_THRESHOLD = 0.5

# Generation
TEMPERATURE = 0.7
MAX_TOKENS = 500

πŸ“Š Model Options

Model Size Speed Quality Notes
Llama2 7B 3.8GB Fast Good Via Ollama
Llama2 13B 7.3GB Medium Better Needs more VRAM
Llama2 70B 39GB Slow Excellent Requires A100 GPU

πŸ” Best Practices

  • Save ChromaDB periodically
  • Use proper chunk sizes for your domain
  • Test retrieval quality
  • Include metadata in documents
  • Monitor embedding quality
  • Validate LLM outputs
  • Use appropriate chain types
  • Store API keys securely

🎯 Advanced Features

  • Metadata Filtering: Filter by date, author, etc.
  • Hybrid Search: Combine BM25 + semantic search
  • Conversation History: Maintain context across turns
  • Multi-turn QA: Complex reasoning chains
  • Answer Confidence: Score answer reliability
  • Document Versioning: Track document updates

πŸš€ Optimization Tips

  • Quantization: Use GGML quantized Llama2
  • Batch Processing: Process multiple queries
  • Caching: Cache embeddings
  • Chunking: Optimize chunk size
  • GPU Acceleration: Use CUDA if available
  • Model Compression: Use smaller variants

🀝 Contributing

Contributions are welcome! Please:

  1. Fork the repository
  2. Create a feature branch
  3. Commit your changes
  4. Push to the branch
  5. Submit a pull request

πŸ“š References

πŸ“„ License

This project is open source and available under the MIT License.

πŸ’‘ Advantages of This Setup

  • βœ… Fully local - no API costs
  • βœ… Privacy-first - data stays on your machine
  • βœ… Open-source - full transparency
  • βœ… Customizable - modify any component
  • βœ… Offline-capable - works without internet
  • βœ… Scalable - handle large document collections

⚠️ Limitations

  • Slower than cloud APIs
  • Requires local compute resources
  • Model quality depends on size/version
  • Setup complexity vs. managed solutions

βœ‰οΈ Contact

For questions or suggestions, please open an issue or contact the project maintainers.


Built for privacy-first, offline-capable RAG systems using open-source models πŸ”’

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages