Skip to content

Session logging — track model usage, latency, and costs #9

@canoo

Description

@canoo

Add session logging to track which model handled each task, response time, and estimated cost.

Scope:

  • Log each MCP tool invocation with model, prompt length, response length, latency
  • Store logs locally in ~/.config/nexus/logs/
  • JSON format for easy parsing
  • TUI screen to browse recent sessions

Why: Without observability, we cannot improve routing or measure value.


Data Persistence Requirement (added from Opus audit)

Session logging must use SQLite (not flat JSON files) from the start. The schema needs to serve both:

Design the schema before writing code. Key tables:

  • sessions — id, start_time, end_time, cli_tool, persona
  • tasks — id, session_id, model, routing (cloud/local), tokens_in, tokens_out, latency_ms, cost_usd, quality_rating
  • routing_decisions — id, task_id, reason, alternatives_considered

Retrofitting a schema later is painful. Get this right in v0.2.0.


Proxy-Ready Schema (added)

The SQLite schema must support proxy-intercepted requests (v0.3.0), not just MCP tool calls. Design for this now:

CREATE TABLE sessions (
    id TEXT PRIMARY KEY,
    start_time TEXT NOT NULL,
    end_time TEXT,
    cli_tool TEXT,           -- 'claude-code', 'gemini-cli', 'kiro-cli', 'proxy', 'mcp'
    persona TEXT,
    nexus_mode TEXT           -- 'hybrid', 'cloud', 'personas-only'
);

CREATE TABLE tasks (
    id TEXT PRIMARY KEY,
    session_id TEXT NOT NULL REFERENCES sessions(id),
    timestamp TEXT NOT NULL,
    source TEXT NOT NULL,     -- 'mcp-tool', 'proxy-intercept', 'manual'
    model TEXT NOT NULL,
    routing TEXT NOT NULL,    -- 'cloud', 'local'
    routing_reason TEXT,      -- 'keyword:commit_msg', 'token_count<500', 'user_override'
    tokens_in INTEGER,
    tokens_out INTEGER,
    latency_ms INTEGER,
    cost_usd REAL,
    cloud_cost_equivalent REAL,  -- what this would have cost on cloud (for savings calc)
    quality_rating INTEGER   -- null, 1 (helpful), 0 (not helpful)
);

CREATE TABLE routing_decisions (
    id TEXT PRIMARY KEY,
    task_id TEXT NOT NULL REFERENCES tasks(id),
    reason TEXT NOT NULL,
    alternatives_considered TEXT,  -- JSON array of {model, reason_rejected}
    classifier_version TEXT       -- 'rules-v1', 'ml-v1' (future)
);

The source field distinguishes MCP tool calls (v0.2.0) from proxy intercepts (v0.3.0). The classifier_version field future-proofs for when the rules engine is eventually replaced by a trained classifier.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    Status

    Todo

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions