Keep client-only nodes out of model election by michaelneale · Pull Request #1074 · Mesh-LLM/mesh-llm

michaelneale · 2026-07-25T08:39:27Z

Client-only nodes now remain proxy-only after joining a mesh, even when active demand includes an unserved model. This prevents zero-capacity clients from electing themselves, downloading a large model, and delaying management API startup.

Validation:

focused host-runtime regression test passed
host-runtime Clippy passed with warnings denied on the source branch
formatting and diff checks passed on this focused branch

Summary by CodeRabbit

Bug Fixes
- Client-mode nodes no longer receive automatic model assignments during host election or model selection.
- Improved role-aware handling of model selection.
Tests
- Added a new async test verifying model assignment is derived from node role (returns no assignment for client role, assigns one for worker role).

coderabbitai · 2026-07-25T08:39:47Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d89c5212-f0d4-41f1-9916-6884576770e7

📥 Commits

Reviewing files that changed from the base of the PR and between dc2f87e and 806e8f5.

📒 Files selected for processing (2)

crates/mesh-llm-host-runtime/src/runtime/auto_join.rs
crates/mesh-llm-host-runtime/src/runtime/tests/auto_join.rs

🚧 Files skipped from review as they are similar to previous changes (1)

crates/mesh-llm-host-runtime/src/runtime/auto_join.rs

📝 Walkthrough

Walkthrough

Client nodes now bypass model assignment during automatic model selection. A role-aware helper preserves existing assignment behavior for non-client roles, and an async test verifies client and worker outcomes.

Changes

Client model assignment

Layer / File(s)	Summary
Role-aware assignment helper `crates/mesh-llm-host-runtime/src/runtime/auto_join.rs`	Adds a helper that returns `None` for client nodes and delegates to the existing model assignment function otherwise.
Automatic selection integration and test `crates/mesh-llm-host-runtime/src/runtime/auto_join.rs`, `crates/mesh-llm-host-runtime/src/runtime/tests/auto_join.rs`	Updates automatic model selection to use the role-aware helper and verifies client and worker assignment behavior.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related issues

bug(client): client-only node can elect itself to serve and download a model #1072 — The change skips model assignment for client nodes during automatic model selection.

Suggested reviewers: i386, ndizazzo

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: preventing client-only nodes from participating in model election.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch fix/client-only-model-election

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

🧹 Nitpick comments (1)

crates/mesh-llm-host-runtime/src/runtime/tests/auto_join.rs (1)
60-72: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Make the regression test validate role-derived behavior.

The test creates a NodeRole::Client node but passes true explicitly, so it would pass even if the role were ignored. Exercise the production selection path, or otherwise verify the role-to-is_client wiring; add a non-client case to confirm delegation still works.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/mesh-llm-host-runtime/src/runtime/tests/auto_join.rs` around lines 60
- 72, The regression test currently passes is_client=true explicitly, so it does
not verify NodeRole::Client wiring. Update
client_role_never_receives_model_assignment to exercise the production selection
path or derive is_client from the node role, and add a non-client test case
confirming model delegation still succeeds.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/mesh-llm-host-runtime/src/runtime/tests/auto_join.rs`:
- Around line 60-72: The regression test currently passes is_client=true
explicitly, so it does not verify NodeRole::Client wiring. Update
client_role_never_receives_model_assignment to exercise the production selection
path or derive is_client from the node role, and add a non-client test case
confirming model delegation still succeeds.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 50bfe559-1a60-40dc-92fe-d1f46007da4a

📥 Commits

Reviewing files that changed from the base of the PR and between f430bff and dc2f87e.

📒 Files selected for processing (2)

crates/mesh-llm-host-runtime/src/runtime/auto_join.rs
crates/mesh-llm-host-runtime/src/runtime/tests/auto_join.rs

i386

Reviewed current head 806e8f5. No blocking findings.

The role gate now runs before demand-based assignment, the existing ctx.is_client fallback guard still keeps clients out of the auto-pack path, and the regression test verifies both client rejection and normal worker delegation. The green client-auto boot job also reaches passive_mode / Client ready without electing or downloading a model.

Non-blocking follow-up: #1072 also documents that parse_size_str("36 layers") returns 0. This PR makes that unable to elect a client, but unknown catalog sizes can still be treated as fitting worker capacity. Please keep that fail-closed size parsing fix tracked separately if it is intentionally outside this focused PR.

Uses MeshLLM built-in `mesh` collective intelligence / Mixture of Agents when Buzz Auto sees two or more distinct physical models. With zero or one model, Auto remains ordinary `auto`. This is an alternative way to improve tool responses, accuracy, and resistance to hallucination when a high-latency distributed mesh contains diverse models. Models and members may come and go: collective routing enables only after stable capacity, drops on confirmed contraction, and can recover later. Mesh-specific failures retry once through ordinary Auto. This update also pins MeshLLM to a v0.73.1-compatible backport of [MeshLLM #1074](Mesh-LLM/mesh-llm#1074), so client-only Buzz nodes cannot enter model election or download a remote provider model. Buzz preserves the selected local sharing model and switches an existing client to sharing across a controlled app restart, retaining one runtime and one `:9337` / `:3131` pair per machine. Validation: - Full local `just ci` passes on the cleaned branch. - MeshLLM host-runtime suite: 1,568 passed, 0 failed; strict Clippy passes. - Buzz desktop Tauri suite with `mesh-llm`: 1,721 passed, 0 failed; strict feature Clippy passes. - Playwright covers client-to-share using the saved local model and no destructive stop. - Packaged two-machine testing proved single-model routing, dual-model collective routing, tool-markup fallback, and runtime reuse. - Packaged client-only recheck routed a real Mini Buzz turn through M5 while Mini stayed `is_client=true`, `is_host=false`, hosted no models, and created no Gemma cache. Builds on the recovery work merged in #2823; this PR does not duplicate it. --------- Signed-off-by: Michael Neale <[email protected]> Signed-off-by: Tyler Longwell <[email protected]> Co-authored-by: Michael Neale <[email protected]> Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Co-authored-by: Tyler Longwell <[email protected]>

* origin/main: Fix Metal small-batch matmul parity for GLM verification (#1078) Handle K-only transposed KV page import and export (#1084) Refresh llama.cpp upstream patch queue (#1085) chore: improve embedded native-runtime compatibility guidance (#1043) fix(console-ui): chat transcript snapping during live status updates (#1083) ci: bump Linux CUDA slim container to gha-convention base runner image fix: record activation cache prefix identities (#1041) fix: read-only model download caches (#1042) ci: disable sccache for Windows ROCm native runtime build (#1087) ci: fix v0.74 release GPU builds (sccache disk-only + force_hosted_runners) (#1086) Make release sccache failures non-fatal (#1079) Keep client-only nodes out of model election (#1074) # Conflicts: # crates/mesh-llm-host-runtime/src/runtime/auto_join.rs # crates/mesh-llm-host-runtime/src/runtime/tests/auto_join.rs # third_party/llama.cpp/patches/0004-Add-lanes-external-media-and-chat-grammar-support.patch

Uses MeshLLM built-in `mesh` collective intelligence / Mixture of Agents when Buzz Auto sees two or more distinct physical models. With zero or one model, Auto remains ordinary `auto`. This is an alternative way to improve tool responses, accuracy, and resistance to hallucination when a high-latency distributed mesh contains diverse models. Models and members may come and go: collective routing enables only after stable capacity, drops on confirmed contraction, and can recover later. Mesh-specific failures retry once through ordinary Auto. This update also pins MeshLLM to a v0.73.1-compatible backport of [MeshLLM block#1074](Mesh-LLM/mesh-llm#1074), so client-only Buzz nodes cannot enter model election or download a remote provider model. Buzz preserves the selected local sharing model and switches an existing client to sharing across a controlled app restart, retaining one runtime and one `:9337` / `:3131` pair per machine. Validation: - Full local `just ci` passes on the cleaned branch. - MeshLLM host-runtime suite: 1,568 passed, 0 failed; strict Clippy passes. - Buzz desktop Tauri suite with `mesh-llm`: 1,721 passed, 0 failed; strict feature Clippy passes. - Playwright covers client-to-share using the saved local model and no destructive stop. - Packaged two-machine testing proved single-model routing, dual-model collective routing, tool-markup fallback, and runtime reuse. - Packaged client-only recheck routed a real Mini Buzz turn through M5 while Mini stayed `is_client=true`, `is_host=false`, hosted no models, and created no Gemma cache. Builds on the recovery work merged in block#2823; this PR does not duplicate it. --------- Signed-off-by: Michael Neale <[email protected]> Signed-off-by: Tyler Longwell <[email protected]> Co-authored-by: Michael Neale <[email protected]> Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Co-authored-by: Tyler Longwell <[email protected]>

runtime: keep client-only nodes out of model election

dc2f87e

github-actions Bot requested a review from i386 July 25, 2026 08:39

github-actions Bot assigned michaelneale Jul 25, 2026

coderabbitai Bot reviewed Jul 25, 2026

View reviewed changes

test(runtime): exercise role-aware model assignment

806e8f5

i386 approved these changes Jul 25, 2026

View reviewed changes

i386 merged commit cdd793b into main Jul 25, 2026
26 checks passed

i386 deleted the fix/client-only-model-election branch July 25, 2026 11:55

i386 mentioned this pull request Jul 25, 2026

Add Inkling Q2 staged runtime and certify text split serving #1017

Closed

micspiral mentioned this pull request Jul 26, 2026

feat(desktop): use collective mesh routing for Auto block/buzz#2825

Merged

This was referenced Aug 7, 2026

feat: make plugin-only Serve mode nodes discoverable/reachable by other mesh peers #1192

Merged

fix(client): prevent auto model assignment #1208

Merged

i386 mentioned this pull request Aug 8, 2026

bug(client): client-only node can elect itself to serve and download a model #1072

Closed

Navigation Menu

Repository navigation

Keep client-only nodes out of model election - #1074

Keep client-only nodes out of model election#1074
i386 merged 2 commits into
mainfrom
fix/client-only-model-election

michaelneale commented Jul 25, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jul 25, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Uh oh!

i386 left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

michaelneale commented Jul 25, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jul 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

i386 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

michaelneale commented Jul 25, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 25, 2026 •

edited

Loading