Skip to content

refactor(zeph-llm): remove dead ModelOrchestrator in favour of RouterProvider #2540

@bug-ops

Description

@bug-ops

Summary

ModelOrchestrator and its supporting module (orchestrator/) are dead code. The routing path that was supposed to activate them (LlmRoutingStrategy::Task) was never implemented and explicitly falls back to a single provider. All real routing goes through RouterProvider (EMA / Thompson / Cascade / Bandit) and TriageRouter, which are strictly more capable.

This is a pre-v1.0.0 cleanup: no backward compatibility concerns, no deprecation shims needed.

Evidence

LlmRoutingStrategy::Task is a no-op stub

crates/zeph-core/src/bootstrap/provider.rs:

LlmRoutingStrategy::Task => {
    // Task-based routing is not yet implemented; fall back to single provider.
    tracing::warn!(
        "routing = \"task\" is not yet implemented; \
         ...
    );
    build_single_provider_from_pool(pool, config)
}

AnyProvider::Orchestrator is never constructed

Searching the entire workspace for AnyProvider::Orchestrator(Box::new or any construction site yields zero results. The variant exists only as match arms — all of which are unreachable at runtime.

Dead code scope

File Lines Status
crates/zeph-llm/src/orchestrator/mod.rs 1 755 dead — ModelOrchestrator struct + impl
crates/zeph-llm/src/orchestrator/classifier.rs 477 dead — TaskType enum + heuristic classify()
crates/zeph-llm/src/orchestrator/router.rs 455 dead — SubProvider enum (internal provider envelope)
AnyProvider::Orchestrator branches ~30 unreachable — any.rs, health.rs, bootstrap/mod.rs, guardrail.rs
LlmRoutingStrategy::Task variant + match arm ~10 unreachable — config.rs, provider.rs

Total dead code: ~2 700 lines.

Why RouterProvider supersedes ModelOrchestrator

Dimension ModelOrchestrator RouterProvider
Selection method Static keyword heuristics on last message (contains("code"), contains("translate"), …) Statistical online learning: EMA latency tracking, Thompson Sampling (Beta conjugate), LinUCB contextual bandit, cascade quality escalation
Failure handling TTL blacklist (hardcoded 300 s) Per-strategy: Thompson updates Beta posteriors; Bandit penalises arm reward; Cascade tracks quality history window
Cost awareness None Bandit: cost_weight × cost_fraction in reward signal; Cascade: cost-tier escalation
Memory signals None MAR (Memory-Augmented Routing): set_memory_confidence() adjusts bandit feature vector
State persistence None Thompson: ~/.zeph/router/thompson.json; Bandit: ~/.zeph/router/bandit.json; Reputation tracker
Complexity pre-triage None TriageRouter: full LLM inference → Simple/Medium/Complex/Expert tier mapping

ModelOrchestrator provides a strict subset of RouterProvider's capabilities. There is no scenario where activating LlmRoutingStrategy::Task would be preferable.

What to remove

  1. Delete crates/zeph-llm/src/orchestrator/ (entire directory: mod.rs, classifier.rs, router.rs)
  2. Remove pub mod orchestrator; from crates/zeph-llm/src/lib.rs
  3. Remove AnyProvider::Orchestrator(Box<ModelOrchestrator>) variant from crates/zeph-llm/src/any.rs and all match arms that handle it
  4. Remove LlmRoutingStrategy::Task variant from config + provider.rs match arm
  5. Clean up unreachable branches in:
    • crates/zeph-core/src/bootstrap/mod.rs (context-window detection for Orchestrator)
    • crates/zeph-core/src/bootstrap/health.rs (Ollama warmup inside Orchestrator)
    • crates/zeph-sanitizer/src/guardrail.rs (MEDIUM-01 rejection check)

Acceptance criteria

  • crates/zeph-llm/src/orchestrator/ directory deleted
  • AnyProvider::Orchestrator variant removed; all match expressions remain exhaustive
  • LlmRoutingStrategy::Task variant removed from config enum and bootstrap match
  • cargo clippy --all-targets --all-features --workspace -- -D warnings passes with zero warnings
  • cargo nextest run --workspace --all-features --lib --bins passes (test count must not decrease — Orchestrator unit tests are also dead weight and should be deleted)
  • CHANGELOG.md updated under [Unreleased]

Metadata

Metadata

Assignees

Labels

P2High value, medium complexityllmzeph-llm crate (Ollama, Claude)refactorCode refactoring without functional changes

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions