Skip to content

NewCoder3294/paperstack

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

156 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PaperStack

A local-first research-paper platform. Turn any topic into a living library of arXiv research — discovered daily, read in full, and questioned by a model that runs entirely on your machine. No accounts. No cloud. No telemetry.

PaperStack expands a plain-language topic into an arXiv query, builds a collection that a local agent keeps current, lets you read papers in a built-in PDF viewer, and answers questions grounded in your own library with a language model running locally via Ollama. The only network call it ever makes is to arXiv itself.


Features

  • Topic collections + discovery agent — name a topic in plain language; PaperStack turns it into an arXiv query and a local agent ingests new matches and flags superseded papers.
  • Local, grounded Q&A — ask questions over a paper, a collection, or your whole library; answers stream from a local LLM and cite the exact passages they used (hybrid FTS + vector retrieval, RRF).
  • PDF reading view — read papers in-app (pdf.js) with trackpad pinch-zoom and highlight-to-ask (select text → ask about it).
  • Research subagents — one-shot specialized assistants over your library: Summarize, Key Findings, Methods & Limitations, Critique (on a paper) and Compare, Literature Review, Find Gaps (on a collection), streamed with citations.
  • Saved lists — hand-curated, color-coded reading lists, separate from the auto-discovered topics.
  • Curation — review suggested replacements, browse the archive, and see what the agent changed.
  • Fully local — SQLite (+ sqlite-vec) on disk, models via Ollama. No auth, no accounts, no cloud.

Architecture

Next.js (App Router, TS)  ──/api/* (Next rewrites, same-origin)──▶  FastAPI (Python)
  /        landing page                                              SQLite + sqlite-vec (one file)
  /app     dashboard (collections · reader · chat)                   Ollama  (embeddings + chat, local)
  • Frontend (frontend/): Next.js 16, shadcn/ui, TanStack Query, Zod. Talks to the backend same-origin via Next rewrites (/api/* → the FastAPI server), so there is no CORS to configure.
  • Backend (backend/): FastAPI + SQLite/sqlite-vec. Retrieval, the discovery agent, the RAG chat, and the research subagents all live here.
  • Models (local, via Ollama): an embedding model (nomic-embed-text) and a chat model (default qwen2.5:7b).

Prerequisites

  • Python 3.12+ (3.13 recommended; needs SQLite loadable-extension support for sqlite-vec)
  • Node 20+ and pnpm
  • Ollama running locally, with two models pulled:
    ollama pull nomic-embed-text   # embeddings (~270 MB)
    ollama pull qwen2.5:7b         # chat (~4.7 GB) — or a smaller one (see Configuration)

Quick start

Open two terminals.

1) Backend (defaults to port 8000):

cd backend
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
uvicorn --factory paperstack.main:build_app --port 8000

2) Frontend (defaults to port 3000):

cd frontend
pnpm install
pnpm dev

Then open http://localhost:3000 (landing) and http://localhost:3000/app (the dashboard). Create a collection (e.g. "retrieval augmented generation"), click Ingest to pull papers from arXiv, then read and ask.

Configuration

The backend reads these environment variables (all optional):

Variable Default Purpose
PAPERSTACK_DB paperstack.db SQLite database path
OLLAMA_URL http://localhost:11434 Ollama endpoint
PAPERSTACK_EMBED_MODEL nomic-embed-text embedding model
PAPERSTACK_CHAT_MODEL qwen2.5:7b chat model (e.g. qwen2.5:3b, llama3.2:3b for lighter machines)
PAPERSTACK_CHAT_NUM_CTX 8192 chat context window
PAPERSTACK_SOURCES_DIR paperstack_sources on-disk PDF cache

The frontend proxies /api/* to http://127.0.0.1:8000 by default; override with PAPERSTACK_API_ORIGIN (e.g. if the backend runs on a different port):

PAPERSTACK_API_ORIGIN=http://127.0.0.1:8001 PORT=3001 pnpm dev

The discovery agent

A standalone CLI ingests/curates collections (intended to run on a schedule):

cd backend
python -m paperstack.agent            # all topic collections
python -m paperstack.agent --collection 1

A launchd template for scheduling lives in deploy/ (not auto-installed).

Testing

cd backend  && .venv/bin/pytest          # backend (offline; models/network are stubbed)
cd frontend && pnpm test                 # frontend (Vitest + RTL; fetch/SSE mocked)
cd frontend && pnpm exec tsc --noEmit     # type check

The full suite is deterministic and offline — no live network, model, or clock.

Project layout

backend/    FastAPI app, SQLite/sqlite-vec, retrieval, agent, RAG chat, subagents (pyproject.toml)
frontend/   Next.js app — / (landing) and /app (dashboard)
deploy/     launchd template for scheduling the discovery agent
docs/       design specs and implementation plans

License

MIT.

The 3D bust used on the landing page (“Bust” by Eric Wilson) is licensed CC BY 3.0 and rendered live with three.js.

About

A local-first research-paper platform: arXiv discovery, local RAG Q&A with citations, a PDF reader, and research subagents — all running on your own machine. No accounts, no cloud.

Topics

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages