Document-grounded equity research assistant for public US companies. The app is designed around company profiles, uploaded filings/transcripts, retrieval over parsed document chunks, cited Q&A, company comparisons, and structured research memos.
Gemma 3 4B — recorded 2026-09-29, real model output, no fallback. 71.5-second app-only edit with synthetic narration (OpenAI Marin). The closing advice refusal is an application guardrail before inference. Video provenance.
Live demo — keyless deterministic API. Watch the video above for the real-model run.
Measured on 29 September 2026: 35 synthetic finance QA cases, one pass per configuration, Gemma 3 4B (gemma3:4b, Q4_K_M) through Ollama 0.34.4 on an Apple M5 with 32 GiB memory.
| Metric | Gemma configuration | Deterministic baseline |
|---|---|---|
| Graded answer accuracy | 41.4% | 40.0% |
| Citation precision | 96.4% (27/28) | 100.0% (24/24) |
| Citation recall | 43.4% | 43.4% |
| Graded refusal correctness | 100% (7/7) | 85.7% (6/7) |
| Unsupported claims | 1 (1/35 answers) | 0 |
| Validation fallback / model attempts | 23.5% (4/17) | N/A |
| Median HTTP latency, all cases | 54.8 ms | 16.1 ms |
| Median HTTP latency, model attempts | 7,861.7 ms | N/A |
13/35 Gemma-configuration cases produced accepted model answers; four fell back and 18 bypassed inference through deterministic comparisons or evidence/refusal gates.
This small synthetic development set tracks changes between iterations; it does not establish accuracy on real SEC filings. Rubric annotations are not blinded or independently validated; all failures, including refusals of answerable questions, remain included, with full per-case notes in the raw results.
Reproduce collection: python scripts/run_live_eval.py --output evals/results/new-run
For a deterministic baseline without Ollama, add --providers local. --providers ollama collects only the model configuration; the default both retains the existing Ollama-then-local order. Hardware metadata is optional on macOS, Linux and Windows. See the collection and scoring runbook for setup, partial-run diagnostics and response-bound review requirements.
Raw results, per-case scoring notes, commit/source hashes, earlier runs and reproduction details.
This is research assistance software, not investment advice. It must not issue buy, sell, hold, or price-target recommendations unless the user supplies explicit valuation assumptions.
- Starts from company filings and uploaded source documents instead of market commentary.
- Makes retrieval visible through chunk-level citations, source metadata, and fallback behavior when evidence is weak.
- Includes a finance QA eval set so answer quality, citation precision, unsupported-question refusal, and hallucination risk can be tested instead of hand-waved.
- Corpus builder can discover a ticker through SEC EDGAR, fetch recent
10-K,10-Q,8-K, andDEF 14Afilings, convert them into chunks, and add them to the RAG corpus. - Chat synthesis retrieves evidence first, then uses local Gemma/Ollama only when available; invalid or uncited model output falls back to deterministic cited synthesis.
- The eval suite has 35 finance QA cases covering factual extraction, multi-document synthesis, company comparison, and unsupported questions.
- Synthetic fixtures include companies, documents, chunks, chat responses, memo responses, and comparison responses for deterministic review.
See docs/walkthrough.md for an end-to-end walkthrough: a finance question, the retrieved filing evidence, and the cited answer it produces.
- Backend API: FastAPI service under
backend/. - Frontend: Vite React TypeScript app under
frontend/. - Storage: a local JSON repository holds metadata and research history; source documents live on the filesystem. See the implemented architecture.
- Evals: curated finance QA cases under
evals/. - Fixtures: deterministic sample companies, documents, chunks, and API payloads under
tests/fixtures/.
The backend exposes these endpoints:
GET /companiesGET /companies/search?q=AAPLPOST /companiesPOST /companies/discoverGET /companies/{company_id}POST /companies/{company_id}/documentsGET /documents/{document_id}GET /documents/{document_id}/chunksDELETE /documents/{document_id}POST /research/chatPOST /research/memoPOST /research/compare
See docs/api-contract.md for request and response shapes aligned with the project spec.
The app can move beyond the seeded local company list and build a filings-first company corpus:
- Search by ticker or company name in the sidebar.
- Local matches filter immediately.
- Press
SECto query SEC EDGAR's public company ticker mapping. - Choose
Build corpusto create the company, fetch SEC filing metadata, download filings, convert them to text, chunk them, embed them, and add them to the RAG corpus.
Default corpus import:
- latest
10-K - latest 4
10-Qfilings - latest 6
8-Kfilings - latest
DEF 14Aproxy statement where available
Backend endpoints:
curl "http://localhost:8001/companies/search?q=AAPL"
curl -X POST "http://localhost:8001/companies/discover" \
-H "Content-Type: application/json" \
-d '{"query":"AAPL","build_corpus":true}'SEC access uses official public EDGAR data endpoints and requires a declared user agent. Set this before running the API if you want your own contact string in requests:
export AIERC_SEC_USER_AGENT="Your Name [email protected]"Uploaded PDFs/text files are still supported for transcripts, investor decks, and custom notes.
The chat endpoint always retrieves evidence first. It then tries to synthesize the answer with a local Ollama model when available. The recommended default is a small 3-5B class Gemma model:
ollama pull gemma3:4b
export AIERC_LLM_PROVIDER=auto
export AIERC_OLLAMA_MODEL=gemma3:4b
export AIERC_OLLAMA_TIMEOUT_SECONDS=180Provider modes:
AIERC_LLM_PROVIDER=auto: use Ollama/Gemma if available, otherwise deterministic cited synthesis.AIERC_LLM_PROVIDER=ollama: require Ollama/Gemma and return a clear low-confidence response if unavailable.AIERC_LLM_PROVIDER=local: deterministic cited synthesis only.
The Ollama prompt uses retrieved filing text (up to five chunks, 6,000 characters per chunk and 24,000 characters total), independently of the shorter retrieval-debug previews. It uses schema-constrained JSON generation with allowed source indices, a 512-token output cap, and inline references such as [1] in the answer and every key point. Invalid schemas, missing references, and inconsistent or out-of-range references fall back to deterministic cited synthesis, including in ollama mode. A required Ollama server that cannot respond still produces an explicit unavailable response. The UI displays the returned model and provider; the coverage panel exposes the chat retrieval limit (default 8).
Deterministic answers retain the supporting chunk for each selected sentence. Their inline references, source excerpts, persisted citations, and cited counts come from the same selections. Model citation validation checks source membership and numbering; it cannot independently prove that every generated claim follows from that source. Memos label generic bull/bear prompts as analyst scenarios rather than filing conclusions.
For frontend development outside Docker, use Node.js 22.12+ (22.x) or 24+ and install dependencies with npm ci from frontend/.
- Copy the environment template:
cp .env.example .env- Fill in provider credentials as needed. Local tests and fixture-based eval dry runs should not require live LLM credentials.
The default storage mode is intentionally simple and local-first:
- Metadata, conversations, chunks, embeddings, and citations are stored in
AIERC_DATA_DIR/storage/state/store.json. - Uploaded or SEC-downloaded source files are stored under
AIERC_DATA_DIR/storage/raw/<company_id>/. AIERC_DATA_DIRdefaults todata/; set it to a disposable directory for tests, demos, or CI.
Upload guardrails are configurable with:
AIERC_MAX_UPLOAD_MB(or legacyMAX_UPLOAD_MB), default10.AIERC_ALLOWED_UPLOAD_EXTENSIONS, default.txt,.md,.text,.pdf.AIERC_ALLOWED_UPLOAD_CONTENT_TYPES, defaulttext/plain,text/markdown,application/pdf,application/octet-stream.AIERC_CORS_ORIGINS(or legacyCORS_ORIGINS), defaulthttp://localhost:3000,http://127.0.0.1:3000.
- Run the full stack:
docker compose up --buildThe default command starts only the API and web app. Compose maps web port 3000 to Vite's fixed container port 5173 and sets the API seed directory to /app/data/sample_documents. On a fresh workspace the bundled companies must have ready documents. Verify after startup:
curl --fail http://localhost:3000/
curl --fail http://localhost:8000/healthThe health response should show nonzero documents and chunks. If using a Mac-hosted Ollama from the API container, set AIERC_OLLAMA_BASE_URL=http://host.docker.internal:11434 in .env; 127.0.0.1 inside a container refers to that container.
Expected local URLs:
- Web:
http://localhost:3000 - API:
http://localhost:8000 - API docs:
http://localhost:8000/docs
The MVP eval set lives in evals/finance_qa_v1.jsonl. It includes 35 cases:
- 12 single-document factual questions
- 10 multi-document synthesis questions
- 6 company comparison questions
- 7 unsupported or not-enough-information questions
Each case includes id, company_ids, question, required_source_documents, expected_answer_points, citation rules or acceptable chunk IDs, must_not_include, and difficulty.
Supporting eval docs:
Recommended MVP pass criteria:
- Answer accuracy: at least 80%
- Factual answers with citations: at least 90%
- Citation precision: at least 80%
- Severe hallucinations: 0 in smoke tests
- Median indexed-document Q&A latency: under 8 seconds
Fixtures and sample data are synthetic and intentionally small:
- tests/fixtures/companies.json
- tests/fixtures/documents.json
- tests/fixtures/chunks/retrieval_chunks.json
- tests/fixtures/api/chat_request.json
- tests/fixtures/api/chat_response.json
- tests/fixtures/api/memo_response.json
- tests/fixtures/api/compare_response.json
- data/sample_documents/manifest.json
Use these fixtures for backend API tests, retrieval scoring smoke tests, and frontend contract mocks.
The sample document corpus covers NVDA, MSFT, AAPL, JPM, XOM, and TSLA with synthetic 10-K and earnings transcript excerpts. The canonical eval set uses NVDA, AAPL, JPM, XOM, and TSLA fixture IDs; MSFT remains useful for local seed/demo data.
The supported backend is ai_equity_research_copilot_backend.main; run it from backend/ with uvicorn ai_equity_research_copilot_backend.main:create_app --factory. backend.main is only a compatibility re-export, with no separate state or endpoints.
The frontend uses the real API by default. Failed requests display the backend error and retain rejected upload input for retry. They never create fake documents, replace company data, or synthesize demo answers. VITE_PUBLIC_DEMO=true selects the read-only live seeded API presentation used for deployment.
For an intentionally offline, synthetic browser preview, run VITE_BROWSER_DEMO=true npm run dev from frontend/. This adapter is selected before any requests; it sends no API calls and disables uploads and SEC imports. Its prepared examples are labeled and kept separate from live workspace data. This flag is distinct from VITE_PUBLIC_DEMO.
Retrieval loads one request-scoped corpus snapshot for chat, memo, or comparison, filtering unrelated rows before validation and preparing chunk term sets once. The seven memo queries reuse that snapshot. A later request reloads storage, so newly ingested documents are visible without cache invalidation.
Chat, memo retrieval and comparisons require lexical evidence; hashed-vector similarity alone cannot make a chunk relevant. Query-term weights and normalized chunk terms are prepared once per request and reused across memo sections. Comparisons reserve retrieval space for different companies/documents before redundant chunks.
Question text can constrain annual/quarterly filings, earnings calls/releases, fiscal years, and quarter forms such as Q2 FY2025 or fiscal 2025 Q2. These constraints intersect explicit API filters. An unavailable requested period does not silently fall back to another period. A release without quarter metadata can qualify only when its text explicitly identifies the requested quarter and the year agrees with document metadata; deterministic answers then require matching passage-level period evidence.
Without a source type or date, research uses each company's latest available annual filing when one is available. Quarterly disclosure questions prefer quarterly filings. An explicit request for the latest information uses the latest available fiscal period across source types. Other undated source families use their own latest available fiscal period, including the annual side of an annual-versus-quarter comparison whose question names only the quarterly year. These choices are disclosed in answer limitations and never replace explicit source/year filters. Supply explicit filters when a different baseline is intended; the parser is a bounded English heuristic, not a general temporal reasoner.
Period ordering uses period-end dates when all candidate sources provide them; otherwise it uses fiscal years and known quarters, treating a full annual period as ending in Q4. Sources that cannot be ordered from the available metadata are retained with an explicit ambiguity disclosure. Quarter names such as “first quarter of fiscal 2031” use the same scope as Q1 FY2031. Conflicting short fiscal years and windows containing an excluded period cannot establish a single requested period. These passage checks apply to deterministic extraction: Ollama still receives full retrieved chunks and may interpret a different period within them, so model citation membership is not a period-correctness guarantee.
Answer selection requires matching content terms in an actual passage, never just a high chunk score. Narrow requests such as price targets or internal targets also require corresponding evidence wording. Unsupported requests return low confidence and no citations before any model call. Filings-only buy/sell/hold requests and requests explicitly beyond the document coverage are refused. Supported analyst-note price targets can still be quoted; this is not a blanket topic ban.
These are conservative relevance checks, not semantic entailment or completeness guarantees. Deterministic output remains extractive. Retrieval favors query terms occurring together in usable source passages. Compact numeric tables retain available year headings and adjacent unit captions; missing column labels are not guessed. Comparisons label each source period and retain separate citations even when wording repeats. Flattened or complex tables remain limited; the system does not compute cross-period changes or establish exact forecasts from related text. Mocked provider tests check integration; they do not measure live model quality.
Run the deterministic demo smoke and write a reviewable artifact:
python scripts/demo_smoke.py
cat demo_artifacts/demo_smoke_output.jsonRun the eval metadata smoke used by CI:
python scripts/eval_smoke.pyValidate JSON fixtures:
python -m json.tool tests/fixtures/companies.json >/dev/null
python -m json.tool tests/fixtures/documents.json >/dev/null
python -m json.tool tests/fixtures/chunks/retrieval_chunks.json >/dev/nullValidate eval JSONL:
python - <<'PY'
import json
from pathlib import Path
path = Path("evals/finance_qa_v1.jsonl")
for line_no, line in enumerate(path.read_text().splitlines(), 1):
if line.strip():
json.loads(line)
print(f"valid jsonl: {path}")
PYValidate eval support artifacts:
python - <<'PY'
import json
from pathlib import Path
for path in [
Path("evals/scoring_rubric.json"),
Path("data/sample_documents/manifest.json"),
Path("data/evals/finance_qa_examples.json"),
]:
json.loads(path.read_text())
print("valid eval support artifacts")
PYRun backend tests:
cd backend
pytestRun frontend checks:
cd frontend
npm run build
npm test- Keep raw uploaded files outside the public web directory.
- Store citation metadata at chunk level: company, document, page range, section title, excerpt, and score.
- All factual answer claims must map back to cited retrieved chunks.
- Refuse unsupported questions instead of answering from model memory.
- Log request metadata, retrieval IDs, model name, latency, token counts, and estimated cost, but not API keys or raw secrets.
Original project code is licensed under the MIT License, copyright 2026 Johan Vaz. Third-party dependencies retain their own licenses and notices.
