Skip to content

About

An AI-powered medical consultation pipeline using CrewAI with a parallel independent analysis approach

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

15 Commits

Folders and files

Repository files navigation

Medical Consultant

An AI-powered medical consultation pipeline using CrewAI with a parallel independent analysis approach. Two (optionally three) different LLM providers independently analyse the same raw medical record, and their findings are compared, synthesised, and verified for safety.

Instead of a sequential "therapist → specialist consilium → synthesis" flow (which suffered from the "broken telephone" problem — each downstream agent only saw the previous agent's summary), this pipeline runs independent analyses in parallel, then compares them to find agreements, disagreements, and unique insights.

Quick Start

1. Prerequisites

  • Python ≥ 3.12
  • uv (or pip)

2. Install

git clone <repo-url> medical_consultant
cd medical_consultant
uv sync

3. API keys

Copy the example file and fill in your keys:

cp .env.example .env
# Edit .env with your actual API keys

The file looks like this:

# .env
DEEPSEEK_API_KEY=sk-your-deepseek-key-here
MISTRAL_API_KEY=your-mistral-key-here

Alternatively, export them directly in your shell:

export DEEPSEEK_API_KEY="sk-your-deepseek-key-here"
export MISTRAL_API_KEY="your-mistral-key-here"

Optional — Gemini as third-party independent auditor. For true cross-family verification (completely different architecture from DeepSeek and Mistral), set:

GOOGLE_API_KEY=your-google-api-key

When GOOGLE_API_KEY is set, Gemini Flash handles synthesis and Gemini Pro handles verification — providing an independent audit trail that eliminates self-confirmation bias.

4. Google Sheets setup (optional)

The pipeline can load a patient's medical record directly from Google Sheets.

Easiest — OAuth Desktop auto-discovery

If you have a client_secret.json saved at ~/.config/medical_consultant/client_secret.json, the loader picks it up automatically. No env vars needed — the first run opens a browser for "Sign in with Google", and subsequent runs are silent.

You get this file from Google Cloud Console → Credentials → Create Credentials → OAuth client ID → Desktop app → download JSON.

Alternative — gcloud CLI

If your spreadsheet is in the same Google account that your gcloud CLI is logged into, you're already set. The pipeline calls gcloud auth print-access-token on each run. Just make sure you've run gcloud auth login at least once.

No env vars, no JSON files, no client secrets.

Alternative — explicit OAuth path

GOOGLE_OAUTH_CREDENTIALS=~/.config/medical_consultant/client_secret.json

Alternative — KDE Wallet

On KDE Plasma, you can store the client secret JSON in KWallet under my-apps/med_consultant_google. The loader reads it automatically via D-Bus (uses the jeepney library).

Alternative — Service account (headless/server)

GOOGLE_SERVICE_ACCOUNT_KEY=/path/to/service-account-key.json

5. Run

Option A — pass medical record as text:

python -m medical_consultant.main \
  "I've been feeling extremely tired for 3 months, gaining weight despite no diet change, and my skin is very dry. I'm 42F."

Option B — load from Google Sheets:

# By spreadsheet name
python -m medical_consultant.main --sheet "Медична картка пацієнта"

# By spreadsheet ID (from the URL)
python -m medical_consultant.main --sheet-id "1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgVE2upms"

# Bypass 24h cache and re-fetch
python -m medical_consultant.main --sheet "Медична картка" --no-cache

Option C — pipe from stdin:

echo "Frequent headaches, blurred vision, and tingling in my hands. 35M." \
  | python -m medical_consultant.main

Option D — preview mode (dry-run with cheap local model):

# Uses local Ollama gemma4 (free, no API key)
uv run python -m medical_consultant.preview

# Override model or test complaint
PREFERRED_MODEL=deepseek/deepseek-v4-flash uv run python -m medical_consultant.preview
COMPLAINT="headache dizziness" uv run python -m medical_consultant.preview

Preview mode runs the full pipeline with stub prompts — exercises the live dashboard, token tracking, and health checks without spending time or money on real analysis.


Architecture

Why parallel independent analysis?

The old sequential architecture had a fundamental flaw: downstream agents only saw summaries, not raw data. This created a "broken telephone" — each agent couldn't find anything the previous one missed.

The current architecture gives every analyst access to the same raw medical record, making them true independent experts whose opinions are compared.

flowchart LR
    A[Raw Medical Record] --> B[Phase 1: Preprocessor\nStructures raw data]
    B --> C[Phase 2a: DeepSeek\nIndependent analysis]
    B --> D[Phase 2b: Mistral\nIndependent analysis]
    C --> E[Phase 3: Synthesizer\nCompares A vs B]
    D --> E
    E --> F[Phase 4: Verifier\nSafety cross-check]
    F --> G[Final Verified Report]
Loading

When Gemini is enabled (GOOGLE_API_KEY set), the Synthesizer uses Gemini Flash and the Verifier uses Gemini Pro — providing a completely different architecture from either analyst, eliminating self-confirmation bias.

Pipeline steps

Phase Model Agent Description
1. Preprocessing DeepSeek Flash data_preprocessor Extracts structured data from raw record: medications, lab results, diagnoses, timeline. Builds a chronological context map linking each lab value to concurrent treatments.
2a. DeepSeek analysis DeepSeek Flash analyst_deepseek Full independent clinical analysis of the raw data. Differential diagnosis, physiological causal chains, chronological lab analysis.
2b. Mistral analysis Mistral Large analyst_mistral Full independent analysis of the same raw data. A genuine "second opinion" from a different model with different reasoning style.
3. Synthesis Gemini Flash / DeepSeek Flash synthesizer Compares both analyses — finds agreements (high confidence), disagreements (flagged for review), and unique findings (one saw, other missed).
4. Verification Gemini Pro / DeepSeek Flash verifier Adversarial safety cross-check: detects duplicate/contradictory recommendations, wrong vitamin forms (folic acid vs methylfolate), dangerous drug interactions, and incorrect lab interpretations. Actively hunts for at least 2 issues per report.

Model assignment

Using different LLM providers gives a genuine "second opinion" dynamic — each model brings its own reasoning style and training distribution, reducing blind spots.

Role Provider Default Model
Preprocessor DeepSeek deepseek/deepseek-v4-flash
Analyst A DeepSeek deepseek/deepseek-v4-flash
Analyst B Mistral mistral/mistral-large-latest
Synthesizer Gemini / DeepSeek gemini/gemini-2.5-flash / deepseek/deepseek-v4-flash
Verifier Gemini / DeepSeek gemini/gemini-2.5-pro / deepseek/deepseek-v4-flash

The Verifier defaults to a stronger model than the analysts to avoid a Flash model rubber-stamping its own peer's hallucinations. Set VERIFIER_MODEL=deepseek/deepseek-v4-pro to upgrade without Gemini.

Live dashboard

During a pipeline run you see a live ANSI dashboard showing real-time phase progress with elapsed timers:

  ⚙️  PIPELINE STATUS
  ────────────────────────────────────────
  ✓   Preprocessing — structuring raw data   ↑1,234 → ↓567 · 1.2s
  ✓   Independent analysis — DeepSeek        ↑5,678 → ↓2,345 · 8.4s
  ◉   Independent analysis — Mistral         (12.3s)
     Synthesis — comparing both analyses
     Safety verification

Each phase transitions from waiting → running (with live timer) → done (with token stats). When stdout is a pipe (CI, log capture), it falls back to simple one-line status updates with no ANSI magic.

After the pipeline completes, a token summary table is printed with health warnings:

  ────────────────────────────────────────────────────────────
  📋 PIPELINE SUMMARY
  ────────────────────────────────────────────────────────────
  Phase                                    Sent  Received   Cached      Total
  ────────────────────────── ────────── ────────── ──────── ──────────
  Preprocessing                             1,234       567        0      1,801
  DeepSeek analysis                         1,234     5,678      100      6,912
  Mistral analysis                          2,456     7,890      200     10,346
  Synthesis                                   789       345        0      1,134
  Safety verification                         456       234       50        690
  ────────────────────────── ────────── ────────── ──────── ──────────
  TOTAL                                     6,169    14,714      350     20,883

Health checks

After each phase, the pipeline runs automatic diagnostics:

Check What it catches
Output truncation Completion tokens hit max_tokens limit — report is incomplete
Low efficiency Chars/token < 0.8 — output may be repetitive or garbage
Short output Large input but tiny output — model barely analysed
Incomplete ending Output doesn't end with a sentence terminator — likely cut mid-sentence

Warnings appear inline after each phase and are aggregated in the final summary.

Output files

Every run produces two persistent outputs:

logs/run_2026-07-21_14-32-05.log    ← Operational log (LiteLLM token stats per call)
results/run_2026-07-21_14-32-05/    ← Full pipeline output directory
├── 00-summary.md                   ← Metadata, input, token table, warnings
├── 01-final-report.md              ← Phase 4: verified report (MAIN result)
├── 02-consilium.md                 ← Phase 3: synthesis
├── 03-analysis-deepseek.md         ← Phase 2a: independent analysis
├── 04-analysis-mistral.md          ← Phase 2b: independent analysis
└── 05-preprocessing.md             ← Phase 1: structured data

Pipeline statistics (JSONL)

Use --stats-dir to save structured run statistics for historical analysis:

python -m medical_consultant.main --stats-dir stats/ "my symptoms..."
# or via env var:
MEDICAL_STATS_DIR=stats/ python -m medical_consultant.main "my symptoms..."

Each run appends one JSON line to stats/pipeline_stats.jsonl with token counts per phase, timing, cost estimates, and health warnings. The format is designed for analysis with Pandas, DuckDB, or plain jq:

cat stats/pipeline_stats.jsonl | jq '.cost_usd, .elapsed_total_s'

Configuration

All configuration is via environment variables (from .env file or exported manually):

Variable Default Description
DEEPSEEK_API_KEY — DeepSeek API key (set in .env or exported directly)
`M
ISTRAL_API_KEY` — Mistral API key (set in .env or exported directly)
GOOGLE_API_KEY — Google Gemini API key — enables Gemini as third-party verifier/synthesizer
THERAPIST_MODEL deepseek/deepseek-v4-flash LiteLLM model string for DeepSeek-based agents (preprocessor, analyst)
ENDOCRINOLOGIST_MODEL openai/mistral-large-latest LiteLLM model string for Mistral-based analyst
VERIFIER_MODEL deepseek/deepseek-v4-flash LiteLLM model string for the verifier — upgrade to deepseek/deepseek-v4-pro for stronger auditing
GEMINI_SYNTHESIZER_MODEL gemini/gemini-2.5-flash Model for synthesis when Gemini is enabled
GEMINI_VERIFIER_MODEL gemini/gemini-2.5-pro Model for verification when Gemini is enabled
DEEPSEEK_API_BASE "" Custom API base for DeepSeek-proxied endpoints
MISTRAL_API_BASE https://api.mistral.ai/v1 Custom API base for Mistral-proxied endpoints
GOOGLE_OAUTH_CREDENTIALS "" Path to OAuth client secret JSON (Google Sheets)
GOOGLE_OAUTH_TOKEN ~/.config/medical_consultant/google_token.json Path to saved OAuth token
GOOGLE_SERVICE_ACCOUNT_KEY "" Path to service account JSON key (alternative auth)
SHEETS_TO_EXCLUDE Довідник,Службова Worksheets to skip when exporting (comma-separated)
MEDICAL_STATS_DIR — Directory for JSONL pipeline statistics (same as --stats-dir CLI flag)

Using different models

Set env vars before running. Any LiteLLM-supported provider works:

env \
  THERAPIST_MODEL=openai/gpt-4o \
  ENDOCRINOLOGIST_MODEL=anthropic/claude-sonnet-4-20250514 \
  python -m medical_consultant.main "my symptoms..."

For Gemini integration:

env \
  GOOGLE_API_KEY=your-key \
  GEMINI_VERIFIER_MODEL=gemini/gemini-2.5-pro \
  python -m medical_consultant.main "my symptoms..."

Project Structure

medical_consultant/
├── .env.example                    # Environment variable template
├── .gitignore                      # Ignored files (logs, results, .env, etc.)
├── pyproject.toml                  # Project metadata & dependencies
├── README.md                       # This file
├── docs/                           # Documentation (currently empty)
├── results/                        # Per-run output directories (gitignored)
├── logs/                           # Per-run operational logs (gitignored)
├── tests/                          # Test suite (96 unit + smoke + preflight tests)
│   ├── conftest.py                 # Shared fixtures
│   ├── test_agents.py              # Agent definitions & LLM setup
│   ├── test_config.py              # Env var loading
│   ├── test_crew.py                # Crew structure
│   ├── test_main.py                # Pipeline orchestrator & CLI
│   ├── test_tasks.py               # Task definitions
│   ├── test_sheets_integration.py  # Google Sheets loader (marked: sheets)
│   ├── test_smoke.py               # Real API connectivity (marked: smoke)
│   └── test_preflight.py           # Full pipeline with real model (marked: preflight)
└── src/
    └── medical_consultant/
        ├── __init__.py             # Public API: run_consultation, load_sheet
        ├── py.typed                # PEP 561 marker
        ├── config.py               # Env var loading & model config
        ├── agents.py               # Agent definitions (LLM, roles) — 5 agents + 2 optional Gemini LLMs
        ├── tasks.py                # Task definitions (5 tasks across 4 phases)
        ├── crew.py                 # Crew factory functions (one per phase)
        ├── main.py                 # Pipeline orchestrator + CLI entrypoint
        ├── sheets_loader.py        # Google Sheets → markdown export (gcloud/OAuth/KWallet/SA)
        ├── monitoring.py           # Live dashboard, token tracking, LiteLLM callbacks, log capture
        ├── preview.py              # Dry-run mode with stub prompts and cheap local model
        ├── stats_writer.py         # JSONL pipeline statistics storage
        └── translations/
            └── ua.json             # Ukrainian CrewAI prompt translations

Extending the Pipeline

Add a new analyst provider

  1. config.py — add a new env var for the model/key.
  2. agents.py — create a new LLM instance and an Agent for it.
  3. tasks.py — create a new analysis task (identical structure to the existing ones).
  4. crew.py — create a factory function for the new analyst crew.
  5. main.py — add a new phase call in run_consultation().

Example:

# agents.py
claude_llm = _build_llm(
    model="anthropic/claude-sonnet-4-20250514",
    api_key=CLAUDE_API_KEY,
    api_base=None,
)
analyst_claude = Agent(
    role="Лікар-аналітик (комплексний аналіз)",
    goal="...",
    backstory="...",
    llm=claude_llm,
    verbose=True,
)

Add a new phase

  1. agents.py — define a new agent for the phase.
  2. tasks.py — define the task with required template variables.
  3. crew.py — create a factory function.
  4. main.py — call it in run_consultation() and pass outputs as inputs to the next phase.

Loading from Google Sheets (architecture)

The pipeline can ingest a multi-sheet Google Spreadsheet and convert it into a single structured Markdown document for the AI agents.

flowchart LR
    A[Google Sheets] -->|gcloud / OAuth / KWallet / SA| B[sheets_loader.py]
    B --> C[Markdown tables]
    C --> D[AI Pipeline]
Loading

Authentication methods (tried in order):

  1. OAuth Desktop auto-discovery — picks client_secret.json from ~/.config/medical_consultant/ automatically
  2. gcloud CLI (easiest) — calls gcloud auth print-access-token using your currently active account
  3. KDE Wallet — reads from my-apps/med_consultant_google via D-Bus (Plasma desktop)
  4. OAuth Desktop explicit — GOOGLE_OAUTH_CREDENTIALS env var
  5. Service Account — JSON key for headless/server deployments

Service sheets (e.g. "Довідник") are automatically excluded. Results are cached locally in /tmp/medical_consultant_cache/ for 24 hours to avoid repeated API calls.


Testing

Unit tests (fast, no API calls)

Run these always — they mock all LLM calls:

uv run pytest

These run by default. They cover config loading, agent roles, task wiring, crew structure, pipeline input/output wiring, CLI parsing, and health checks. 96 tests covering all pipeline phases.

Smoke tests (real API calls)

These make actual requests to DeepSeek and Mistral to verify connectivity and the full pipeline end-to-end. They are excluded from the default run.

# All smoke tests
uv run pytest -m smoke

# Just one provider's connectivity
uv run pytest -m smoke -k deepseek
uv run pytest -m smoke -k mistral

Preflight tests (real pipeline, cheap model)

Runs the full 5-phase pipeline once with stub prompts and a local/cheap model (defaults to Ollama gemma4). Validates the live dashboard, token tracking, log files, and health checks end-to-end.

# Watch the live dashboard (recommended)
uv run pytest -m preflight -v -s

# Headless — just pass/fail
uv run pytest -m preflight

# Override model
PREFERRED_MODEL=deepseek/deepseek-v4-flash uv run pytest -m preflight -s

Sheets tests (Google API)

Requires Google authentication and a test spreadsheet:

uv run pytest -m sheets

Disclaimer

This tool is an AI prototype for educational and experimental purposes. It is not a real medical device. Always consult a qualified healthcare professional for medical advice, diagnosis, or treatment.

About

An AI-powered medical consultation pipeline using CrewAI with a parallel independent analysis approach

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages