-
Notifications
You must be signed in to change notification settings - Fork 2
feat(llm): implement ClassifierMode::Judge for cascade routing #1723
Copy link
Copy link
Closed
Labels
enhancementNew feature or requestNew feature or requestllmzeph-llm crate (Ollama, Claude)zeph-llm crate (Ollama, Claude)
Description
Context
Cascade routing (PR #1721) introduced ClassifierMode::Judge as a config option but the implementation falls through to heuristic mode. The architecture is designed but not wired.
Problem
Users who configure classifier_mode = "judge"" get heuristic behavior with no warning. The judge mode was intended to use summary_model` for LLM-based quality scoring.
Fix
Implement LLM-based quality scoring in CascadeRouter:
- Call
summary_modelwith a lightweight prompt asking it to score the response - Parse score from response
- Fall back to heuristic on LLM error (already designed)
- Add config validation to warn when judge mode is set but
summary_modelis missing
Priority
Low — heuristic mode is functional for v1; judge mode is a quality improvement.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestllmzeph-llm crate (Ollama, Claude)zeph-llm crate (Ollama, Claude)