Skip to content

fix: final salvage wave — backend/gateway cluster (#95164 #95173 #95328 #95601 #95237) - #95980

Merged
teknium1 merged 6 commits into
mainfrom
salvage/final-wave-backend
Aug 27, 2026
Merged

teknium1 merged 6 commits into
mainfrom
salvage/final-wave-backend

Conversation

@teknium1

Copy link
Copy Markdown
Collaborator

Final salvage wave — backend/gateway cluster (tracker #94724)

Re-verified and cherry-picked five backend PRs onto current main with original authorship preserved. One PR from the assigned cluster (#91043) was found already merged to main (2552579) and is not included.

Salvaged PRs

PR Author Fix Verdict
#95164 @rainbowgore tui_gateway adopts live compression.* / model.context_length config at turn start (closes #95151); keeps #95571's lean-tail default coherent for Desktop/TUI users changing tail_mode live Salvaged (one both-added conflict vs the merged #91043 guard block, kept both)
#95173 @kvnloo Compaction tail-anchor + auto-focus scans exclude display_kind-typed synthetic wakes (#92703). Both scan sites re-verified as still unguarded on post-lean-tail main Salvaged (clean pick)
#95328 @gokay-ai fleet_restart_pending marker: interrupted hermes update catches up the fleet restart on the next run (#95294 parts 1+2; part 3 left for maintainer). Rebase-checked against 27385e5 / 306a096 / adb29d8 Salvaged (clean pick; see local-env test note)
#95601 @fangliquanflq Profile/tools/status control routes moved off the default executor to run_in_threadpool — fixes #95559 "Runtime not ready" starvation Salvaged (clean pick)
#95237 @Finn763 POOL_KEEPALIVE_FRESH_MS 90s → 4min: pool LRU eviction tolerates missed keepalive pings / WSL2 IPC stalls (#95189; author de-scoped from #95189 fix set) Salvaged (clean pick)
#91043 @jackulau — Already merged to main earlier today (2552579); skipped

Verification (A/B per pick)

Credits

All commits carry original authorship: @rainbowgore (rainbowgits), @kvnloo (lesseradmin), @gokay-ai (via Cursor Agent co-author), @fangliquanflq, @Finn763. Contributor email mapping added for [email protected] → kvnloo. Rebase-merge to preserve per-commit attribution.

Infographic

final-wave-backend

teknium1 and others added 6 commits August 26, 2026 18:39
…nd auto-focus (#92703)

Kanban/background completion wakes persist as role=user rows typed with
display_kind="internal_notification" (the synthetic-wake path in run.py).
The model-payload builder already strips display_kind before the request
and is_user_originated_turn already ignores it, but two compaction scans
still treated those rows as real user turns:

- _is_actionable_user_turn (tail anchor) only checked role/content, so a
  notification became the protected 'last user turn' the compressor keeps.
- _derive_auto_focus_topic only skipped synthetic compression turns, so
  operational notices leaked into the compact focus hint.

Both now exclude display_kind-typed rows, mirroring the existing
is_user_originated_turn exclusion. No schema change; cache- and
role-alternation-safe.

Behavior-contract tests feed 1,000 operational notifications around one
human turn and assert they never anchor the tail, become the auto-focus
source, or count as actionable user turns.

Fixes #92703
An interrupted hermes update after git pull advanced HEAD never
restarted running gateways, and the next update said "Already up to
date" and skipped the fleet. Persist a HERMES_HOME fleet_restart_pending
marker after HEAD moves, clear it only when restart completes (or
nothing was running), and catch up on the next hermes update even when
git is current — also when latest.json records a stale runtime SHA.

Co-authored-by: GokayAI <[email protected]>
…alls (#95189)

The renderer pings each pool backend every 60s (`hermes:backend:touch` →
`touchPoolBackend` → updates `lastActiveAt`). The LRU eviction cap used a
keepalive-fresh window of 90s — only 1.5× the ping cadence — to decide
whether a backend was "plausibly still alive". One missed or delayed ping
pushed a live backend past the threshold and the cap-driven eviction killed
the active profile's backend mid-session, restarting the gateway and
re-minting runtime ids. On WSL2, where the renderer→Electron IPC roundtrips
through 9p, brief 9p hiccups commonly stretch a ping to seconds of observed
silence, producing the ~80–90s exit / ~2 min cycle reported in #95189
(122 gateway starts on 2026-08-26 alone, driving renderer OOM via reconnect
churn at ~5GB/day).

Widen POOL_KEEPALIVE_FRESH_MS to 4 minutes (3× ping cadence + IPC stall
headroom, still bounded well below POOL_IDLE_MS=10min). Backends with one or
even two missed pings are now spared; truly idle backends (multiple lapses,
minutes idle) remain eligible for eviction by the cap and the idle reaper.
The constant is also overridable via HERMES_DESKTOP_POOL_KEEPALIVE_FRESH_MS
to make this tunable without a rebuild.
@github-actions

github-actions Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

૮ >ﻌ< ა ci review

ran on 81e4a97 — fix(desktop): widen pool keepalive-fresh window to absorb WS

⚠️ Warnings

OSV vulnerability scan · View job

6 known vulnerabilities found in pinned dependencies.

How to fix:

Review the findings in the Security tab. Update the affected dependencies if a patched version is available.


debug info

CI timings

CI timings · View report · View job

Wall time 3m35s vs 3m55s (-8.5%). 8 job(s) slower, 4 faster, 1 unchanged.

  • Python tests / Run tests: -26.0s
  • OS-specific tests / Windows-only tests: -21.0s
  • Check contributors / check-attribution: -6.0s
  • JS & TS checks / JS & TS checks: +3.0s
  • Detect affected areas: +3.0s

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard comp/tui Terminal UI (ui-tui/ + tui_gateway/) comp/desktop Electron desktop app (apps/desktop/*) comp/dashboard Web dashboard / control panel UI (dashboard/, landing) area/install-update Installer, updater, packaging, wheels, doctor sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Aug 27, 2026
@teknium1
teknium1 merged commit 00988f3 into main Aug 27, 2026
39 checks passed
@teknium1
teknium1 deleted the salvage/final-wave-backend branch August 27, 2026 04:38
teknium1 added a commit that referenced this pull request Aug 27, 2026
…model key

Independent review finding on the merged #95980:
_apply_live_compression_config only acted on PRESENT keys, so removing
tail_mode / model.context_length / target_ratio / model_thresholds /
proactive_prune_* / protect_last_n / min_tail_user_messages / threshold /
idle_compact_after_seconds from config.yaml left stale values active in
live sessions forever (probe-verified: all six stale after applying
empty mappings).

Absence now restores the normalized default — or the model-derived
value — through the SAME derivation the construction path uses:

- ContextCompressor ctor defaults read off its real __init__ signature
  (no hardcoded copies to drift)
- compression.threshold removal re-derives via agent_init's
  _resolve_compression_threshold (Codex gpt-5.4/5.5 + spark autoraise
  included)
- model.context_length removal drops the config override and forces
  re-inference through the deferred get_model_context_length resolution,
  which also re-applies the small-context threshold floor
- model_thresholds removal clears stale per-model overrides from the
  live threshold; tail_mode falls back to the ctor's 'lean' (the old
  present-key path normalized invalid values to 'legacy', diverging
  from the compressor's own fallback)

Also fixes proactive_prune_min_reclaim_tokens's present-but-null default
(was 0; the real default is 4096).

Refs #94724
and7777 pushed a commit to and7777/hermes-agent that referenced this pull request Aug 27, 2026
…model key

Independent review finding on the merged NousResearch#95980:
_apply_live_compression_config only acted on PRESENT keys, so removing
tail_mode / model.context_length / target_ratio / model_thresholds /
proactive_prune_* / protect_last_n / min_tail_user_messages / threshold /
idle_compact_after_seconds from config.yaml left stale values active in
live sessions forever (probe-verified: all six stale after applying
empty mappings).

Absence now restores the normalized default — or the model-derived
value — through the SAME derivation the construction path uses:

- ContextCompressor ctor defaults read off its real __init__ signature
  (no hardcoded copies to drift)
- compression.threshold removal re-derives via agent_init's
  _resolve_compression_threshold (Codex gpt-5.4/5.5 + spark autoraise
  included)
- model.context_length removal drops the config override and forces
  re-inference through the deferred get_model_context_length resolution,
  which also re-applies the small-context threshold floor
- model_thresholds removal clears stale per-model overrides from the
  live threshold; tail_mode falls back to the ctor's 'lean' (the old
  present-key path normalized invalid values to 'legacy', diverging
  from the compressor's own fallback)

Also fixes proactive_prune_min_reclaim_tokens's present-but-null default
(was 0; the real default is 4096).

Refs NousResearch#94724
melon-xf added a commit to melon-xf/hermes-agent that referenced this pull request Sep 3, 2026
…model key

Independent review finding on the merged NousResearch#95980:
_apply_live_compression_config only acted on PRESENT keys, so removing
tail_mode / model.context_length / target_ratio / model_thresholds /
proactive_prune_* / protect_last_n / min_tail_user_messages / threshold /
idle_compact_after_seconds from config.yaml left stale values active in
live sessions forever (probe-verified: all six stale after applying
empty mappings).

Absence now restores the normalized default — or the model-derived
value — through the SAME derivation the construction path uses:

- ContextCompressor ctor defaults read off its real __init__ signature
  (no hardcoded copies to drift)
- compression.threshold removal re-derives via agent_init's
  _resolve_compression_threshold (Codex gpt-5.4/5.5 + spark autoraise
  included)
- model.context_length removal drops the config override and forces
  re-inference through the deferred get_model_context_length resolution,
  which also re-applies the small-context threshold floor
- model_thresholds removal clears stale per-model overrides from the
  live threshold; tail_mode falls back to the ctor's 'lean' (the old
  present-key path normalized invalid values to 'legacy', diverging
  from the compressor's own fallback)

Also fixes proactive_prune_min_reclaim_tokens's present-but-null default
(was 0; the real default is 4096).

Refs NousResearch#94724
zapabob pushed a commit to zapabob/hermes-agent-windows that referenced this pull request Sep 5, 2026
…model key

Independent review finding on the merged NousResearch#95980:
_apply_live_compression_config only acted on PRESENT keys, so removing
tail_mode / model.context_length / target_ratio / model_thresholds /
proactive_prune_* / protect_last_n / min_tail_user_messages / threshold /
idle_compact_after_seconds from config.yaml left stale values active in
live sessions forever (probe-verified: all six stale after applying
empty mappings).

Absence now restores the normalized default — or the model-derived
value — through the SAME derivation the construction path uses:

- ContextCompressor ctor defaults read off its real __init__ signature
  (no hardcoded copies to drift)
- compression.threshold removal re-derives via agent_init's
  _resolve_compression_threshold (Codex gpt-5.4/5.5 + spark autoraise
  included)
- model.context_length removal drops the config override and forces
  re-inference through the deferred get_model_context_length resolution,
  which also re-applies the small-context threshold floor
- model_thresholds removal clears stale per-model overrides from the
  live threshold; tail_mode falls back to the ctor's 'lean' (the old
  present-key path normalized invalid values to 'legacy', diverging
  from the compressor's own fallback)

Also fixes proactive_prune_min_reclaim_tokens's present-but-null default
(was 0; the real default is 4096).

Refs NousResearch#94724
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/install-update Installer, updater, packaging, wheels, doctor comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard comp/dashboard Web dashboard / control panel UI (dashboard/, landing) comp/desktop Electron desktop app (apps/desktop/*) comp/tui Terminal UI (ui-tui/ + tui_gateway/) P2 Medium — degraded but workaround exists sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

7 participants