-
Notifications
You must be signed in to change notification settings - Fork 2
observability(router): add tracing instrumentation to cascade router #1825
Copy link
Copy link
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Problem
crates/zeph-llm/src/router/cascade.rs has zero tracing calls. Every routing decision, quality score, escalation, and judge invocation is completely invisible in logs.
When debugging cascade routing behavior (wrong model selected, unexpected escalation, judge not firing), there is no way to observe what happened without adding debug prints to the source.
Expected behavior
At minimum, add tracing::debug! or tracing::info! calls for:
- Model selection: which provider was selected for this turn, and why (quality threshold check result)
- Judge scoring:
classifier_mode=judgefires a separate LLM call to score response quality — this should log the score, the threshold, and the pass/fail decision - Escalation: when quality score < threshold, log which provider is being escalated to and remaining escalation budget
- Best-seen update: when
best_seenis updated, log the new score and provider - Exhaustion: when all escalations are exhausted, log that
best_seenresponse is being returned
Impact
Without these logs, the cascade router is a black box. The judge LLM call (which costs tokens and latency) cannot be confirmed to fire, and escalation chain behavior cannot be diagnosed from RUST_LOG=debug output.
Suggested log fields
tracing::debug!(
provider = %selected_provider,
quality_score = score,
threshold = self.config.quality_threshold,
classifier_mode = ?self.config.classifier_mode,
escalations_remaining = remaining,
"cascade: quality score evaluated"
);Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request