Skip to content

feat(llm): implement ClassifierMode::Judge for cascade routing #1723

@bug-ops

Description

@bug-ops

Context

Cascade routing (PR #1721) introduced ClassifierMode::Judge as a config option but the implementation falls through to heuristic mode. The architecture is designed but not wired.

Problem

Users who configure classifier_mode = "judge"" get heuristic behavior with no warning. The judge mode was intended to use summary_model` for LLM-based quality scoring.

Fix

Implement LLM-based quality scoring in CascadeRouter:

  1. Call summary_model with a lightweight prompt asking it to score the response
  2. Parse score from response
  3. Fall back to heuristic on LLM error (already designed)
  4. Add config validation to warn when judge mode is set but summary_model is missing

Priority

Low — heuristic mode is functional for v1; judge mode is a quality improvement.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestllmzeph-llm crate (Ollama, Claude)

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions