Skip to content

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository
BranchesTags

Repository files navigation

Local RAG Chatbot Workspace

This repository contains a full local RAG chatbot application with:

  • a TypeScript React frontend that matches the provided mockup layout
  • a FastAPI backend for chat, sessions, ingestion, preview, topics, and graph APIs
  • a local-first retrieval pipeline built on ChromaDB, SQLite, FastEmbed, NVIDIA NIM, and Celery
  • OpenDataLoader PDF parsing for digital text, layout, lists, and tables
  • topic clustering, a directed knowledge graph, chat memory, and optional web fallback

The repository started from the phased plan in rag_chatbot_phases.md and the UI reference in chatbot_ui_mockup.html. The implementation now covers the full Phase 1-6 scope that was planned across those documents.

What The App Does

At a high level, the app lets you:

  • upload PDFs into a local knowledge base
  • ask questions in a chat UI with SSE progress and validated grounded responses
  • cite exact PDF passages and open them in a preview panel
  • organize retrieval by topic collections
  • visualize topic relationships in a knowledge graph
  • persist chat sessions and retrieve memory from past conversations
  • fall back to web search when the local corpus is not enough
  • parse text, lists, tables, and layout from digital PDFs through OpenDataLoader

Current Feature Set

Chat

  • persistent sessions with auto-generated titles
  • grouped session list in the sidebar
  • SSE progress and keepalive updates while provider output is buffered, validated, and grounded before display
  • local semantic sentence selection, with deterministic cited evidence fallback, when the answer provider fails
  • single-flight send protection while a model response is in progress
  • safe rich-text message rendering for paragraphs, lists, headings, inline code, and fenced code blocks
  • citation chips for PDF and web sources
  • collection-scoped retrieval
  • web-search toggle

Retrieval And Ingestion

  • OpenDataLoader PDF parsing for digital text, layout, lists, and tables
  • repeated margin header/footer cleanup using OpenDataLoader block provenance
  • semantic chunking
  • LLM-first keyword and topic tag extraction with KeyBERT fallback
  • vector retrieval + BM25 + reciprocal rank fusion + reranking
  • query rewriting for follow-up questions
  • per-topic Chroma collections
  • topic reclustering
  • directed knowledge graph edges
  • background ingestion through an in-process runner by default, with Celery available for external brokers

Memory

  • SQLite-backed session/message persistence
  • vectorized chat memory in a dedicated chat_history collection
  • same-session retrieval memory
  • cross-session memory support keyed by x-user-id

Pipeline UI

  • file list with chunk counts, topic chips, and overlap summaries
  • upload flow with live status progression
  • manual recluster action
  • knowledge graph view
  • PDF preview panel with highlighted cited text

Architecture

Frontend

The frontend is a route-based React app with a clear separation between app shell, page layer, widgets, and shared utilities:

The main widget areas are:

Backend

The backend is a FastAPI app with routers and service modules separated by responsibility:

Important service modules:

Storage Model

The app stores data locally inside backend/data:

  • app.db: SQLite data for sessions, messages, ingested documents, and page text
  • chroma/: ChromaDB vector store
  • kg.pkl: persisted directed knowledge graph
  • uploads/: uploaded PDF files
  • celery/: filesystem transport directories for Celery

Primary storage responsibilities:

  • SQLite holds session metadata, messages, PDF metadata, and page text
  • ChromaDB holds chunk embeddings, topic collections, and chat memory embeddings
  • NetworkX holds the directed topic graph and is serialized to kg.pkl

Repository Layout

chat/
|-- backend/
|   |-- app/
|   |   |-- core/
|   |   |-- models/
|   |   |-- routers/
|   |   |-- services/
|   |   `-- tasks/
|   |-- data/
|   |-- scripts/
|   |-- requirements.txt
|   `-- README.md
|-- frontend/
|   |-- src/
|   |   |-- app/
|   |   |-- pages/
|   |   |-- shared/
|   |   `-- widgets/
|   |-- scripts/
|   `-- package.json
|-- chatbot_ui_mockup.html
|-- rag_chatbot_plan.md
`-- rag_chatbot_phases.md

Prerequisites

This project has been verified on Windows with:

  • Python 3.11.9
  • Node.js with npm
  • OpenJDK 17 available on PATH for OpenDataLoader

Recommended local prerequisites:

  • Python 3.11.x
  • Node.js 20+
  • OpenJDK 17 installed and available on PATH
  • enough RAM/disk for FastEmbed model caching and ChromaDB

Models

The backend defaults are currently:

  • local embedding model: BAAI/bge-small-en-v1.5 through FastEmbed (384 dimensions)
  • NVIDIA chat model: meta/llama-3.2-11b-vision-instruct

Set RAG_NVIDIA_API_KEY for chat generation. You can verify configured model access with:

cd D:\projects\chat\backend
.venv\Scripts\python scripts\verify_models.py

Quick Start

1. Backend Setup

cd D:\projects\chat\backend
py -3.11 -m venv .venv
.venv\Scripts\python -m pip install --upgrade pip
.venv\Scripts\python -m pip install -r requirements.txt

2. Frontend Setup

cd D:\projects\chat\frontend
npm install

3. Start The API And Ingestion Runner

The API starts the default local ingestion runner automatically. A separate Celery process is only required when RAG_CELERY_BROKER_URL points to an external broker such as Redis instead of the default filesystem:// transport.

The runner handles the full document processing pipeline:

  1. Parsing — Converts digital PDFs through OpenDataLoader core with text, layout, list, and table extraction
  2. Source mapping — Stores OpenDataLoader labels, block refs, bounding boxes, and source text for preview/highlighting
  3. Chunking — Splits extracted text into overlapping semantic chunks
  4. Embedding — Generates vector embeddings through NVIDIA NIM or the resolved local FastEmbed model and stores them in ChromaDB
  5. Topic extraction — Uses the LLM (with KeyBERT fallback) to generate topic tags
  6. Clustering — Assigns documents to topic collections
  7. Knowledge graph — Builds directed edges between related topics

The runner uses a dedicated background thread so the API remains responsive during ingestion. No Redis, RabbitMQ, or separate Celery process is required in the default mode.

cd D:\projects\chat\backend
.venv\Scripts\python -m uvicorn app.main:app --app-dir D:\projects\chat\backend --host 127.0.0.1 --port 8000

For a distributed deployment, configure a durable Redis-compatible broker, then run .venv\Scripts\python scripts\run_celery_worker.py as a separate worker service using the same application configuration and durable data store.

On Render, the single-service runner must use a paid web service with a persistent disk mounted at /app/data (as declared in render.yaml) to retain uploaded PDFs, SQLite records, and Chroma vectors across restarts. A separate Render worker cannot share that disk, so a distributed Celery deployment also requires shared external storage for documents and indexes.

4. Start The Frontend

If the backend is on the default 8000 port:

cd D:\projects\chat\frontend
npm run dev -- --host 127.0.0.1 --port 5173

If you want to run against a custom backend port, set VITE_API_BASE_URL first:

cd D:\projects\chat\frontend
$env:VITE_API_BASE_URL = "http://127.0.0.1:8002"
npm run dev -- --host 127.0.0.1 --port 4177

Then open:

Production Build

Frontend production build:

cd D:\projects\chat\frontend
$env:VITE_API_BASE_URL = "https://rag-chatbot-api-0612.onrender.com"
npm run build

The frontend build also generates static route entrypoints for direct navigation:

  • dist/chat/index.html
  • dist/pipeline/index.html
  • dist/404.html

That route generation is handled by generate-static-routes.mjs.

Backend sanity check:

cd D:\projects\chat\backend
.venv\Scripts\python -m compileall app scripts

Quality Evaluations

The backend has a versioned RAG evaluation set in backend/quality_evals. It complements deterministic unit and integration tests by measuring the model-sensitive boundaries separately: intent routing, retrieval/reranking, sparse-evidence fallback, citation grounding, and adversarial evidence.

Run the offline regression suite from backend:

cd D:\projects\chat\backend
.venv\Scripts\python -m quality_evals.run

The default run replays checked-in provider outputs, so it is deterministic, requires no provider key, writes a gitignored data/eval-latest.json artifact, and exits non-zero when a quality gate fails. The current gate measures case pass rate, intent accuracy, retrieval precision/recall, citation precision/recall, abstention behavior, and a strict adversarial grounding pass rate.

Use --case CASE_ID for a focused local check. Use --live-model to rerun generation and intent cases against the configured NVIDIA model; it requires RAG_NVIDIA_API_KEY or NVIDIA_API_KEY. --judge enables an optional structured pass/fail model judge for semantic correctness, relevance, faithfulness, and prompt-injection safety. Set RAG_EVAL_JUDGE_MODEL to a judge model different from the target model and calibrate its verdicts against human labels before making it a release gate. The judge log is local and gitignored because it can contain evaluated evidence.

Evaluation cases live in rag_core.jsonl; add an anonymized, reviewed regression case whenever production feedback identifies a new failure pattern. The grounding policy rejects instruction-shaped retrieved content before prompt construction and requires every substantive answer claim to be supported by eligible evidence; unsafe model output may be replaced only with deterministic cited fallback evidence.

API Overview

Base prefix: /api

System

  • GET /api/system/health

Sessions

  • GET /api/sessions
  • POST /api/sessions
  • GET /api/sessions/{session_id}
  • DELETE /api/sessions/{session_id}

Chat

  • POST /api/chat/query
  • POST /api/chat/stream

Documents

  • GET /api/documents
  • POST /api/documents/upload
  • DELETE /api/documents/{document_id}
  • GET /api/documents/preview

Topics And Graph

  • GET /api/topics
  • POST /api/topics/recluster
  • GET /api/topics/graph
  • GET /api/kg/graph

Scripts

Useful backend scripts in backend/scripts:

Environment Variables

The main backend configuration lives in config.py.

Common environment variables:

Variable Purpose Default
RAG_NVIDIA_BASE_URL NVIDIA NIM API base URL https://integrate.api.nvidia.com/v1
RAG_NVIDIA_API_KEY NVIDIA NIM API key used for chat and cloud models unset
RAG_EMBED_MODEL cloud or local embedding model BAAI/bge-small-en-v1.5
RAG_EMBEDDING_DIMENSIONS expected embedding vector length 384
RAG_NVIDIA_CHAT_MODEL NVIDIA chat model meta/llama-3.2-11b-vision-instruct
RAG_RERANKER_MODEL NVIDIA reranker model nvidia/llama-nemotron-rerank-vl-1b-v2
RAG_ENABLE_CROSS_SESSION_MEMORY enable cross-session memory true
RAG_DATA_DIR persistent SQLite, Chroma, graph, upload, and queue root backend/data
RAG_MODEL_CACHE_DIR application-owned local embedding cache <RAG_DATA_DIR>/models/embedding
RAG_ENABLE_EXTRACTIVE_FALLBACK enable local semantic sentence selection when remote generation fails true
RAG_WEB_SEARCH_BACKEND search backend duckduckgo
RAG_WEB_SEARCH_REGION search region us-en
RAG_WEB_SEARCH_MAX_RESULTS max web results 4
RAG_WEB_SEARCH_SCORE_THRESHOLD fallback threshold 0.3
RAG_ALLOWED_ORIGINS explicit CORS origins local defaults
RAG_ALLOWED_ORIGIN_REGEX regex CORS matcher local host/port regex
RAG_CELERY_QUEUE ingestion queue name rag_ingestion

Frontend environment variable:

Variable Purpose
VITE_API_BASE_URL backend base URL for the frontend

Request Context And User Identity

The backend reads x-user-id from request headers in dependencies.py.

The frontend generates and stores a stable local user id automatically in browser storage, which is used to:

  • separate session lists
  • separate chat memory
  • support cross-session memory per local user

Notes On OpenDataLoader Parsing

OpenDataLoader core is the document parser used by ingestion. The Docker image includes the small Java 17 runtime it requires.

Current parsing path:

  • ingestion converts PDFs with opendataloader_pdf.convert
  • structured text blocks, headings, lists, tables, page coordinates, and source references are normalized before chunking
  • repeated marginal headers/footers are removed from parser blocks before chunking
  • chunk metadata stores parser name, content labels, table flags, source refs, source text, and source block boxes

Important capability boundary:

  • this small-runtime build intentionally uses OpenDataLoader core and does not include the hybrid OCR service
  • image-only/scanned PDFs fail with an explicit OCR-required message instead of being indexed as empty documents
  • pypdfium2 is a digital-text fallback if OpenDataLoader conversion fails

Design And UI Notes

The UI implementation was built to preserve the supplied mockup layout:

The frontend intentionally separates:

  • data access
  • application state
  • page-level composition
  • reusable visual widgets

Known Characteristics

These are not necessarily bugs, but they are good to know:

  • topic labels are auto-generated from clustering and may look a little awkward on very small corpora
  • OpenDataLoader starts a Java conversion process during PDF ingestion
  • image-only PDFs require a separately deployed OCR service and are intentionally unsupported by this image
  • if you run frontend and backend on non-default ports, set VITE_API_BASE_URL explicitly

Troubleshooting

Frontend Says Failed to fetch

Usually one of these is true:

  • the backend is not running
  • the frontend is pointing at the wrong API base URL
  • you started the backend on a non-default port but did not set VITE_API_BASE_URL

Check:

curl http://127.0.0.1:8000/api/system/health

Or for a custom port:

curl http://127.0.0.1:8002/api/system/health

Direct Route Open Returns 404

Use the production build output generated by npm run build, which writes:

  • dist/chat/index.html
  • dist/pipeline/index.html
  • dist/404.html

This is required for direct-open static hosting of /chat and /pipeline.

CORS Problems On Localhost Or 127.0.0.1

The backend already accepts local development origins by default, including arbitrary local ports through the configured regex. If you override CORS settings manually, make sure both:

  • localhost
  • 127.0.0.1

are still covered.

OpenDataLoader Parsing Fails

Check:

  • opendataloader-pdf is installed from backend/requirements.txt
  • Python version is 3.11.x and java -version reports Java 17
  • the PDF has a digital text layer; scanned/image-only PDFs need a separate OCR pipeline

Duplicate Worker Or Server Processes

This project uses long-running local processes. If you restart things repeatedly during development, check for stale processes before starting new ones.

Typical processes to watch:

  • python.exe for uvicorn
  • python.exe for run_celery_worker.py
  • node.exe for Vite

References

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages