RAG Document Q&A System

A Retrieval-Augmented Generation system for document question answering. Built with LangChain, ChromaDB, and FastAPI, it supports hybrid retrieval (BM25 + dense embeddings), multiple embedding models, streaming responses, and a real-time WebSocket chat interface.

Architecture

                        +------------------+
                        |  Document Upload |
                        |  (PDF/DOCX/TXT)  |
                        +--------+---------+
                                 |
                        +--------v---------+
                        |  Document Loader |
                        |  & Chunking      |
                        +--------+---------+
                                 |
                   +-------------+-------------+
                   |                           |
          +--------v---------+       +--------v---------+
          |  Dense Embeddings|       |   BM25 Index     |
          |  (BGE / OpenAI)  |       |  (Sparse Tokens) |
          +--------+---------+       +--------+---------+
                   |                           |
          +--------v---------+       +--------v---------+
          |    ChromaDB      |       |  In-Memory BM25  |
          |  Vector Store    |       |     Index        |
          +--------+---------+       +--------+---------+
                   |                           |
                   +-------------+-------------+
                                 |
                        +--------v---------+
                        | Hybrid Retriever |
                        | (RRF Merging)    |
                        +--------+---------+
                                 |
                        +--------v---------+
                        |    QA Chain      |
                        |  (LangChain +    |
                        |   LLM + Memory)  |
                        +--------+---------+
                                 |
                        +--------v---------+
                        |   FastAPI Server |
                        |  REST + WebSocket|
                        +------------------+

Features

• Hybrid Retrieval: Combines dense vector search (ChromaDB) with sparse BM25
• Reciprocal Rank Fusion: Intelligent merging of results from multiple retrieval strategies
• Multiple Embedding Models: Support for BGE, all-MiniLM, and OpenAI embeddings
• Document Processing: Handles PDF, DOCX, and TXT files with intelligent chunking
• Streaming Responses: Real-time token streaming via Server-Sent Events and WebSocket
• Conversation Memory: Maintains chat context with configurable window size
• Source Attribution: Every answer includes cited source documents and page numbers
• Collection Management: Create, list, and delete document collections via API
• Embedding Cache: Hash-based deduplication to avoid redundant embedding computation

Setup

Prerequisites

• Python 3.10+
• OpenAI API key (for LLM inference; embeddings can use local models)

Installation

git clone https://github.com/ln-bodicherla/rag-document-qa.git
cd rag-document-qa
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Configuration

Copy and edit the config file:

cp configs/config.yaml configs/config.local.yaml
# Edit configs/config.local.yaml with your settings

Set your API key:

export OPENAI_API_KEY="sk-..."

Usage

Start the API Server

uvicorn src.api:app --host 0.0.0.0 --port 8000 --reload

Upload Documents

curl -X POST http://localhost:8000/upload \
    -F "[email protected]" \
    -F "collection_name=my_docs"

Ask Questions

curl -X POST http://localhost:8000/query \
    -H "Content-Type: application/json" \
    -d '{"question": "What is the main topic of the document?", "collection_name": "my_docs"}'

Python Client

import httpx

client = httpx.Client(base_url="http://localhost:8000")

# Upload
with open("report.pdf", "rb") as f:
    response = client.post("/upload", files={"files": f}, data={"collection_name": "reports"})

# Query
response = client.post("/query", json={
    "question": "Summarize the key findings",
    "collection_name": "reports",
    "top_k": 5
})
print(response.json()["answer"])

API Endpoints

Method	Endpoint	Description
POST	/upload	Upload and index documents
POST	/query	Ask questions against a collection
GET	/collections	List all document collections
DELETE	/collections/{name}	Delete a collection
GET	/health	Health check endpoint
WS	/ws/chat	WebSocket real-time chat

Project Structure

rag-document-qa/
├── configs/
│   └── config.yaml            # Default configuration
├── src/
│   ├── __init__.py
│   ├── api.py                 # FastAPI application
│   ├── document_loader.py     # Document loading and chunking
│   ├── embeddings.py          # Embedding model management
│   ├── qa_chain.py            # LangChain QA chain with memory
│   ├── retriever.py           # Hybrid retriever with RRF
│   └── vector_store.py        # ChromaDB + BM25 management
├── tests/
│   └── test_retriever.py      # Retriever unit tests
├── data/
│   └── chroma_db/             # Persistent vector store (gitignored)
├── .gitignore
├── README.md
└── requirements.txt

Tech Stack

• LangChain - LLM orchestration and chain composition
• ChromaDB - Vector database for dense retrieval
• FastAPI - Async web framework with WebSocket support
• sentence-transformers - Local embedding models
• rank-bm25 - BM25 sparse retrieval
• PyPDF / python-docx - Document parsing
• tiktoken - Token counting for chunk sizing

License

MIT License