Skip to content

fix(llm): cascade best-seen can capture empty string response #1754

@bug-ops

Description

@bug-ops

Context

Both cascade_chat and cascade_chat_stream track best_seen: Option<(String, f64)> with score 0.0 as default. If a provider returns an empty string, its score is 0.0 and it can become best_seen, which is then returned on all-fail conditions.

Problem

An empty string returned from a provider as best_seen creates worse UX than explicit error propagation — the caller receives a silent empty response instead of an actionable error.

Fix

Guard the best_seen update: only store response if !text.is_empty(), or use a minimum score threshold (e.g. score > 0.0).

Applies to both cascade_chat and cascade_chat_stream.

Labels

enhancement, llm

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestllmzeph-llm crate (Ollama, Claude)

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions