Skip to content

About

Document-grounded equity research copilot with SEC filing ingestion, cited RAG, memos, comparisons, and evals

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

AI Equity Research Copilot

Document-grounded equity research assistant for public US companies. The app is designed around company profiles, uploaded filings/transcripts, retrieval over parsed document chunks, cited Q&A, company comparisons, and structured research memos.

Demo: cited answer from SEC filings

Gemma 3 4B — recorded 2026-09-29, real model output, no fallback. 71.5-second app-only edit with synthetic narration (OpenAI Marin). The closing advice refusal is an application guardrail before inference. Video provenance.

Live demo — keyless deterministic API. Watch the video above for the real-model run.

Results

Measured on 29 September 2026: 35 synthetic finance QA cases, one pass per configuration, Gemma 3 4B (gemma3:4b, Q4_K_M) through Ollama 0.34.4 on an Apple M5 with 32 GiB memory.

Metric Gemma configuration Deterministic baseline
Graded answer accuracy 41.4% 40.0%
Citation precision 96.4% (27/28) 100.0% (24/24)
Citation recall 43.4% 43.4%
Graded refusal correctness 100% (7/7) 85.7% (6/7)
Unsupported claims 1 (1/35 answers) 0
Validation fallback / model attempts 23.5% (4/17) N/A
Median HTTP latency, all cases 54.8 ms 16.1 ms
Median HTTP latency, model attempts 7,861.7 ms N/A

13/35 Gemma-configuration cases produced accepted model answers; four fell back and 18 bypassed inference through deterministic comparisons or evidence/refusal gates.

This small synthetic development set tracks changes between iterations; it does not establish accuracy on real SEC filings. Rubric annotations are not blinded or independently validated; all failures, including refusals of answerable questions, remain included, with full per-case notes in the raw results.

Reproduce collection: python scripts/run_live_eval.py --output evals/results/new-run

For a deterministic baseline without Ollama, add --providers local. --providers ollama collects only the model configuration; the default both retains the existing Ollama-then-local order. Hardware metadata is optional on macOS, Linux and Windows. See the collection and scoring runbook for setup, partial-run diagnostics and response-bound review requirements.

Raw results, per-case scoring notes, commit/source hashes, earlier runs and reproduction details.

This is research assistance software, not investment advice. It must not issue buy, sell, hold, or price-target recommendations unless the user supplies explicit valuation assumptions.

Why This Matters

  • Starts from company filings and uploaded source documents instead of market commentary.
  • Makes retrieval visible through chunk-level citations, source metadata, and fallback behavior when evidence is weak.
  • Includes a finance QA eval set so answer quality, citation precision, unsupported-question refusal, and hallucination risk can be tested instead of hand-waved.

Quick Proof

  • Corpus builder can discover a ticker through SEC EDGAR, fetch recent 10-K, 10-Q, 8-K, and DEF 14A filings, convert them into chunks, and add them to the RAG corpus.
  • Chat synthesis retrieves evidence first, then uses local Gemma/Ollama only when available; invalid or uncited model output falls back to deterministic cited synthesis.
  • The eval suite has 35 finance QA cases covering factual extraction, multi-document synthesis, company comparison, and unsupported questions.
  • Synthetic fixtures include companies, documents, chunks, chat responses, memo responses, and comparison responses for deterministic review.

See docs/walkthrough.md for an end-to-end walkthrough: a finance question, the retrieved filing evidence, and the cited answer it produces.

Current Scope

  • Backend API: FastAPI service under backend/.
  • Frontend: Vite React TypeScript app under frontend/.
  • Storage: a local JSON repository holds metadata and research history; source documents live on the filesystem. See the implemented architecture.
  • Evals: curated finance QA cases under evals/.
  • Fixtures: deterministic sample companies, documents, chunks, and API payloads under tests/fixtures/.

API Contract Summary

The backend exposes these endpoints:

  • GET /companies
  • GET /companies/search?q=AAPL
  • POST /companies
  • POST /companies/discover
  • GET /companies/{company_id}
  • POST /companies/{company_id}/documents
  • GET /documents/{document_id}
  • GET /documents/{document_id}/chunks
  • DELETE /documents/{document_id}
  • POST /research/chat
  • POST /research/memo
  • POST /research/compare

See docs/api-contract.md for request and response shapes aligned with the project spec.

SEC Company Discovery

The app can move beyond the seeded local company list and build a filings-first company corpus:

  1. Search by ticker or company name in the sidebar.
  2. Local matches filter immediately.
  3. Press SEC to query SEC EDGAR's public company ticker mapping.
  4. Choose Build corpus to create the company, fetch SEC filing metadata, download filings, convert them to text, chunk them, embed them, and add them to the RAG corpus.

Default corpus import:

  • latest 10-K
  • latest 4 10-Q filings
  • latest 6 8-K filings
  • latest DEF 14A proxy statement where available

Backend endpoints:

curl "http://localhost:8001/companies/search?q=AAPL"

curl -X POST "http://localhost:8001/companies/discover" \
  -H "Content-Type: application/json" \
  -d '{"query":"AAPL","build_corpus":true}'

SEC access uses official public EDGAR data endpoints and requires a declared user agent. Set this before running the API if you want your own contact string in requests:

export AIERC_SEC_USER_AGENT="Your Name [email protected]"

Uploaded PDFs/text files are still supported for transcripts, investor decks, and custom notes.

Local LLM Synthesis

The chat endpoint always retrieves evidence first. It then tries to synthesize the answer with a local Ollama model when available. The recommended default is a small 3-5B class Gemma model:

ollama pull gemma3:4b
export AIERC_LLM_PROVIDER=auto
export AIERC_OLLAMA_MODEL=gemma3:4b
export AIERC_OLLAMA_TIMEOUT_SECONDS=180

Provider modes:

  • AIERC_LLM_PROVIDER=auto: use Ollama/Gemma if available, otherwise deterministic cited synthesis.
  • AIERC_LLM_PROVIDER=ollama: require Ollama/Gemma and return a clear low-confidence response if unavailable.
  • AIERC_LLM_PROVIDER=local: deterministic cited synthesis only.

The Ollama prompt uses retrieved filing text (up to five chunks, 6,000 characters per chunk and 24,000 characters total), independently of the shorter retrieval-debug previews. It uses schema-constrained JSON generation with allowed source indices, a 512-token output cap, and inline references such as [1] in the answer and every key point. Invalid schemas, missing references, and inconsistent or out-of-range references fall back to deterministic cited synthesis, including in ollama mode. A required Ollama server that cannot respond still produces an explicit unavailable response. The UI displays the returned model and provider; the coverage panel exposes the chat retrieval limit (default 8).

Deterministic answers retain the supporting chunk for each selected sentence. Their inline references, source excerpts, persisted citations, and cited counts come from the same selections. Model citation validation checks source membership and numbering; it cannot independently prove that every generated claim follows from that source. Memos label generic bull/bear prompts as analyst scenarios rather than filing conclusions.

Local Setup

For frontend development outside Docker, use Node.js 22.12+ (22.x) or 24+ and install dependencies with npm ci from frontend/.

  1. Copy the environment template:
cp .env.example .env
  1. Fill in provider credentials as needed. Local tests and fixture-based eval dry runs should not require live LLM credentials.

Storage Mode

The default storage mode is intentionally simple and local-first:

  • Metadata, conversations, chunks, embeddings, and citations are stored in AIERC_DATA_DIR/storage/state/store.json.
  • Uploaded or SEC-downloaded source files are stored under AIERC_DATA_DIR/storage/raw/<company_id>/.
  • AIERC_DATA_DIR defaults to data/; set it to a disposable directory for tests, demos, or CI.

Upload guardrails are configurable with:

  • AIERC_MAX_UPLOAD_MB (or legacy MAX_UPLOAD_MB), default 10.
  • AIERC_ALLOWED_UPLOAD_EXTENSIONS, default .txt,.md,.text,.pdf.
  • AIERC_ALLOWED_UPLOAD_CONTENT_TYPES, default text/plain,text/markdown,application/pdf,application/octet-stream.
  • AIERC_CORS_ORIGINS (or legacy CORS_ORIGINS), default http://localhost:3000,http://127.0.0.1:3000.
  1. Run the full stack:
docker compose up --build

The default command starts only the API and web app. Compose maps web port 3000 to Vite's fixed container port 5173 and sets the API seed directory to /app/data/sample_documents. On a fresh workspace the bundled companies must have ready documents. Verify after startup:

curl --fail http://localhost:3000/
curl --fail http://localhost:8000/health

The health response should show nonzero documents and chunks. If using a Mac-hosted Ollama from the API container, set AIERC_OLLAMA_BASE_URL=http://host.docker.internal:11434 in .env; 127.0.0.1 inside a container refers to that container.

Expected local URLs:

  • Web: http://localhost:3000
  • API: http://localhost:8000
  • API docs: http://localhost:8000/docs

Eval Dataset

The MVP eval set lives in evals/finance_qa_v1.jsonl. It includes 35 cases:

  • 12 single-document factual questions
  • 10 multi-document synthesis questions
  • 6 company comparison questions
  • 7 unsupported or not-enough-information questions

Each case includes id, company_ids, question, required_source_documents, expected_answer_points, citation rules or acceptable chunk IDs, must_not_include, and difficulty.

Supporting eval docs:

Recommended MVP pass criteria:

  • Answer accuracy: at least 80%
  • Factual answers with citations: at least 90%
  • Citation precision: at least 80%
  • Severe hallucinations: 0 in smoke tests
  • Median indexed-document Q&A latency: under 8 seconds

Fixture Data

Fixtures and sample data are synthetic and intentionally small:

Use these fixtures for backend API tests, retrieval scoring smoke tests, and frontend contract mocks.

The sample document corpus covers NVDA, MSFT, AAPL, JPM, XOM, and TSLA with synthetic 10-K and earnings transcript excerpts. The canonical eval set uses NVDA, AAPL, JPM, XOM, and TSLA fixture IDs; MSFT remains useful for local seed/demo data.

API and browser preview modes

The supported backend is ai_equity_research_copilot_backend.main; run it from backend/ with uvicorn ai_equity_research_copilot_backend.main:create_app --factory. backend.main is only a compatibility re-export, with no separate state or endpoints.

The frontend uses the real API by default. Failed requests display the backend error and retain rejected upload input for retry. They never create fake documents, replace company data, or synthesize demo answers. VITE_PUBLIC_DEMO=true selects the read-only live seeded API presentation used for deployment.

For an intentionally offline, synthetic browser preview, run VITE_BROWSER_DEMO=true npm run dev from frontend/. This adapter is selected before any requests; it sends no API calls and disables uploads and SEC imports. Its prepared examples are labeled and kept separate from live workspace data. This flag is distinct from VITE_PUBLIC_DEMO.

Retrieval loads one request-scoped corpus snapshot for chat, memo, or comparison, filtering unrelated rows before validation and preparing chunk term sets once. The seven memo queries reuse that snapshot. A later request reloads storage, so newly ingested documents are visible without cache invalidation.

Question scope and source sufficiency

Chat, memo retrieval and comparisons require lexical evidence; hashed-vector similarity alone cannot make a chunk relevant. Query-term weights and normalized chunk terms are prepared once per request and reused across memo sections. Comparisons reserve retrieval space for different companies/documents before redundant chunks.

Question text can constrain annual/quarterly filings, earnings calls/releases, fiscal years, and quarter forms such as Q2 FY2025 or fiscal 2025 Q2. These constraints intersect explicit API filters. An unavailable requested period does not silently fall back to another period. A release without quarter metadata can qualify only when its text explicitly identifies the requested quarter and the year agrees with document metadata; deterministic answers then require matching passage-level period evidence.

Without a source type or date, research uses each company's latest available annual filing when one is available. Quarterly disclosure questions prefer quarterly filings. An explicit request for the latest information uses the latest available fiscal period across source types. Other undated source families use their own latest available fiscal period, including the annual side of an annual-versus-quarter comparison whose question names only the quarterly year. These choices are disclosed in answer limitations and never replace explicit source/year filters. Supply explicit filters when a different baseline is intended; the parser is a bounded English heuristic, not a general temporal reasoner.

Period ordering uses period-end dates when all candidate sources provide them; otherwise it uses fiscal years and known quarters, treating a full annual period as ending in Q4. Sources that cannot be ordered from the available metadata are retained with an explicit ambiguity disclosure. Quarter names such as “first quarter of fiscal 2031” use the same scope as Q1 FY2031. Conflicting short fiscal years and windows containing an excluded period cannot establish a single requested period. These passage checks apply to deterministic extraction: Ollama still receives full retrieved chunks and may interpret a different period within them, so model citation membership is not a period-correctness guarantee.

Answer selection requires matching content terms in an actual passage, never just a high chunk score. Narrow requests such as price targets or internal targets also require corresponding evidence wording. Unsupported requests return low confidence and no citations before any model call. Filings-only buy/sell/hold requests and requests explicitly beyond the document coverage are refused. Supported analyst-note price targets can still be quoted; this is not a blanket topic ban.

These are conservative relevance checks, not semantic entailment or completeness guarantees. Deterministic output remains extractive. Retrieval favors query terms occurring together in usable source passages. Compact numeric tables retain available year headings and adjacent unit captions; missing column labels are not guessed. Comparisons label each source period and retain separate citations even when wording repeats. Flattened or complex tables remain limited; the system does not compute cross-period changes or establish exact forecasts from related text. Mocked provider tests check integration; they do not measure live model quality.

Useful Commands

Run the deterministic demo smoke and write a reviewable artifact:

python scripts/demo_smoke.py
cat demo_artifacts/demo_smoke_output.json

Run the eval metadata smoke used by CI:

python scripts/eval_smoke.py

Validate JSON fixtures:

python -m json.tool tests/fixtures/companies.json >/dev/null
python -m json.tool tests/fixtures/documents.json >/dev/null
python -m json.tool tests/fixtures/chunks/retrieval_chunks.json >/dev/null

Validate eval JSONL:

python - <<'PY'
import json
from pathlib import Path
path = Path("evals/finance_qa_v1.jsonl")
for line_no, line in enumerate(path.read_text().splitlines(), 1):
    if line.strip():
        json.loads(line)
print(f"valid jsonl: {path}")
PY

Validate eval support artifacts:

python - <<'PY'
import json
from pathlib import Path

for path in [
    Path("evals/scoring_rubric.json"),
    Path("data/sample_documents/manifest.json"),
    Path("data/evals/finance_qa_examples.json"),
]:
    json.loads(path.read_text())

print("valid eval support artifacts")
PY

Run backend tests:

cd backend
pytest

Run frontend checks:

cd frontend
npm run build
npm test

Implementation Notes

  • Keep raw uploaded files outside the public web directory.
  • Store citation metadata at chunk level: company, document, page range, section title, excerpt, and score.
  • All factual answer claims must map back to cited retrieved chunks.
  • Refuse unsupported questions instead of answering from model memory.
  • Log request metadata, retrieval IDs, model name, latency, token counts, and estimated cost, but not API keys or raw secrets.

License

Original project code is licensed under the MIT License, copyright 2026 Johan Vaz. Third-party dependencies retain their own licenses and notices.

About

Document-grounded equity research copilot with SEC filing ingestion, cited RAG, memos, comparisons, and evals

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages