Skip to content

perf: issue #108 backend cleanup — alias-collision warn, SSE idle backoff, public-URL cache (items 3+4+5)#113

Merged
sweetcornna merged 4 commits into
mainfrom
chore/issue-108-backend-cleanup
Jul 4, 2026
Merged

perf: issue #108 backend cleanup — alias-collision warn, SSE idle backoff, public-URL cache (items 3+4+5)#113
sweetcornna merged 4 commits into
mainfrom
chore/issue-108-backend-cleanup

Conversation

@sweetcornna

Copy link
Copy Markdown
Owner

Summary

The backend-only bundle from issue #108's deferral list (items 3, 4, 5 — verified all still live at HEAD by adversarial-checked triage). Items 1 (live-registry process-wide feed, architectural) and 2 (in-progress user bubble, ui/) deliberately remain open.

Orchestrated build: Opus triage (13 claims, 0 refuted) → Opus implementation in an isolated worktree → independently re-validated before push.

What shipped

  • Item 3 — tool-alias collision policy (WARN, never reject): detect_alias_collisions + warn_alias_collisions in tool_aliases.py (structlog key tool_aliases.collision, process-lifetime dedup). Wired at both security gates — skill allowed-tools and subagent allowlist escalation — as a pure side-effect: allow/deny outcomes unchanged (verified in review), first-registered-wins documented. Policy rationale: no live collision exists today (latent), and a hard reject could break working dotted allowlists at security choke points.
  • Item 4 — /admin/subagents overview SSE idle backoff: pure _next_poll_interval(changed, current) helper — 2.0s base, ×1.5 on unchanged ticks up to 10.0s cap, instant reset to 2.0s on any change (row change OR stale-id reap), so liveness on change is preserved while N idle chat tabs no longer each drive a 2s full list_all()+merge scan. Keepalive switched to wall-clock accounting.
  • Item 5 — public-URL disk-read caching: _resolve_public_base_url now caches on a (mtime_ns, size) signature of the py-config drop + learned-origin file. Env CORLINMAN_PUBLIC_URL short-circuit unchanged; mtime change/appear/disappear invalidates; stat errors force re-read. Eliminates up to 2 disk reads per media-bearing tool result on env-unset deploys.

Testing

Per-item TDD: collision detector + no-false-positive-on-shipped-set + gates-warn-without-outcome-change (caplog); interval helper growth/cap/reset; cache hit/mtime-invalidate/env-wins. 84 passed across the touched suites; agent+server mypy + ruff clean — independently re-verified.

Refs #108 (items 3, 4, 5 — do not close; items 1, 2 remain).

Backend-only cleanup bundle for GitHub issue #108 (items 1+2 are
architectural/frontend and out of scope). TDD throughout; behaviour is
preserved at every call site — these are cost/observability improvements,
not policy changes.

Item 3 — tool_aliases collision policy (WARN, never reject):
canonicalize_tool_name folds dotted→wire lossily, so two distinct
spellings can collide with no detection. Add detect_alias_collisions()
(canonical → the >=2 distinct source spellings) + warn_alias_collisions()
to corlinman_agent.tool_aliases, and call it where the two security gates
build their allow-sets: the skill allowed-tools gate
(agent_servicer._skill_allowed_tools_block) and the subagent allowlist
escalation gate (subagent/runner._filter_tools_for_child). Each distinct
collision logs ONE structured warning (log key tool_aliases.collision,
deduped per gate for the process lifetime) — a hard reject could silently
break a working dotted allowlist on a security path, so first-registered
still wins and allow/deny outcomes are unchanged.

Item 4 — /admin/subagents overview SSE idle backoff: the live-overview
stream ran store.list_all()+merge per client on a fixed 2s tick forever.
Extract the pure _next_poll_interval(changed, current) helper (base 2.0s,
×1.5 backoff, 10.0s cap) and drive the loop with it: an unchanged tick
backs the interval off toward the cap; any change snaps straight back to
2s so liveness is never traded away. Keepalive now rides wall-clock
elapsed time so idle proxies keep the same ~10s byte rhythm.

Item 5 — public-URL disk-read caching: _resolve_public_base_url runs once
per media-bearing tool result and, with CORLINMAN_PUBLIC_URL unset,
re-read BOTH backing files every call. Add a module-level cache keyed on a
cheap stat() signature (mtime_ns, size) of the py-config drop and the
learned public_origin file: a hit skips the two reads; a mtime/size change
or either file appearing/disappearing invalidates; a transient stat()
error yields a never-equal signature so we re-read. The env short-circuit
is unchanged and never touches the cache.

Tests (all TDD, red→green):
- test_tool_aliases.py: detector finds {a.b, a_b}, dedupes identical
  spellings, no false positive on the shipped tool set; warn logs one
  structured event, deduped; runner gate warns without changing outcome.
- test_fix_LANE_A_servicer.py: skill gate logs the collision warning
  without changing allow/deny.
- test_subagents.py: _next_poll_interval grows to the cap when unchanged,
  resets to base on change.
- test_public_url_cache.py: cache hit avoids re-read (counters), mtime
  bump invalidates, appear/disappear invalidate, env still wins and
  short-circuits before any disk read.
@github-actions github-actions Bot added codex:needs-review A Codex review is needed for the current PR head. codex:review-requested A Codex review was requested or should be running automatically. status: 🔁 re-review loop A fresh Codex review was requested after the latest change or comment. labels Jul 3, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cce089aad5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +461 to +465
await asyncio.sleep(_poll)
ticks += 1
if ticks * _poll >= _SUBAGENT_SSE_HEARTBEAT_SECONDS:
since_keepalive += _poll
if since_keepalive >= _SUBAGENT_SSE_HEARTBEAT_SECONDS:
yield b": keepalive\n\n"
ticks = 0
since_keepalive = 0.0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep heartbeats on the 10s cadence during backoff

When this SSE stream is idle, _poll grows 2 → 3 → 4.5 → 6.75 before it reaches the 10s cap. Because the heartbeat check only runs after sleeping the entire current interval and then resets since_keepalive to zero, the first idle keepalive after the initial snapshot is delayed to about 16.25s instead of _SUBAGENT_SSE_HEARTBEAT_SECONDS (10s). In deployments where proxies or clients rely on the documented 10s SSE heartbeat, an otherwise idle /admin/subagents/events/live connection can be closed; clamp the next sleep to the remaining heartbeat time or emit a keepalive before taking a long backoff sleep.

Useful? React with 👍 / 👎.

@github-actions github-actions Bot added codex:reviewed Codex posted a review result or thumbs-up after the latest request. status: 👀 ready for maintainer look No automation blocker is known; a maintainer should inspect the current result. and removed codex:review-requested A Codex review was requested or should be running automatically. codex:needs-review A Codex review is needed for the current PR head. status: 🔁 re-review loop A fresh Codex review was requested after the latest change or comment. labels Jul 3, 2026
… (Codex #113)

The idle backoff let since_keepalive accumulate a full backed-off
interval before the heartbeat check ran — the first idle keepalive after
a growth sequence (2→3→4.5→6.75) landed at ~16.25s, past the documented
~10s SSE rhythm proxies rely on. The poll sleep is now split into
heartbeat-bounded slices via a pure _sleep_slices helper: keepalives fire
exactly at rhythm crossings mid-sleep while the scan cadence still
honours the full poll interval (idle cost unchanged).

5 new unit tests (no-crossing, boundary cut, multi-crossing, entry-at-
rhythm, cadence-preservation sweep). 23 passed; ruff+mypy clean.
@github-actions github-actions Bot added codex:needs-rerun A new push or stale result means @codex review should be requested again. codex:review-requested A Codex review was requested or should be running automatically. status: 🔁 re-review loop A fresh Codex review was requested after the latest change or comment. and removed codex:reviewed Codex posted a review result or thumbs-up after the latest request. status: 👀 ready for maintainer look No automation blocker is known; a maintainer should inspect the current result. labels Jul 3, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 91cd580543

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +503 to +504
await asyncio.sleep(slice_s)
since_keepalive += slice_s

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Measure SSE heartbeat from real elapsed time

Even with the new slicer, this keepalive clock is advanced only by the requested sleep duration; the fresh evidence is that time spent in the subsequent await _combined() scan, JSON serialization, or blocked yields is not counted. In the exact high-load/slow-store case this backoff is meant to help, an unchanged scan that takes a few seconds lets the first idle : keepalive arrive well after the documented 10s cadence, so strict proxies can still drop /admin/subagents/events/live; track a time.monotonic() last-byte timestamp instead of accumulating scheduled sleeps.

Useful? React with 👍 / 👎.

Comment on lines +500 to +501
for slice_s, emit_keepalive in _sleep_slices(
_poll, since_keepalive, _SUBAGENT_SSE_HEARTBEAT_SECONDS

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve 2s detection for the first change after idle

For /admin/subagents/events/live, once an unchanged panel has backed _poll off toward 10s, a new subagent or activity update that occurs just after this sleep starts cannot be observed until the whole backed-off interval finishes and _combined() runs. The reset to the 2s cadence happens only after that late scan, so the first change after an idle period regresses from the previous ~2s visibility to as much as ~10s; use an event wakeup or keep the scan sleep bounded by the liveness target.

Useful? React with 👍 / 👎.

@github-actions github-actions Bot added codex:reviewed Codex posted a review result or thumbs-up after the latest request. status: 👀 ready for maintainer look No automation blocker is known; a maintainer should inspect the current result. and removed codex:review-requested A Codex review was requested or should be running automatically. codex:needs-rerun A new push or stale result means @codex review should be requested again. status: 🔁 re-review loop A fresh Codex review was requested after the latest change or comment. labels Jul 3, 2026
…dex #113 round 2)

Address 2 confirmed Codex #113 round-2 findings on the subagents overview
SSE loop (stream_subagents_overview), replacing the _sleep_slices machinery
with one cohesive three-clock redesign:

(a) 3520699516 — the keepalive clock accumulated only REQUESTED sleep, so
    time in _combined()/serialization/blocked yields went uncounted and the
    first idle keepalive could land past the 10s rhythm under load. The loop
    now measures real elapsed time: last_byte = time.monotonic() is updated
    after every yield (data frames AND keepalives) and a keepalive fires
    whenever now - last_byte >= HEARTBEAT.

(b) 3520699518 — after backing off toward 10s, the first change following an
    idle period was not observed until the whole backed-off sleep elapsed
    (detection regressed ~2s → up to 10s). A new cheap O(1) probe,
    _overview_change_signature(store, registry) = (registry.revision,
    store _persist_path st_mtime_ns), is sampled at the base cadence; a diff
    from the last full scan's signature forces an immediate scan (+ backoff
    reset via _next_poll_interval), so a change surfaces within ~2s again.

LiveSubagentRegistry gains a monotonically increasing `revision` int bumped
on every genuine row create/update/remove (_apply_spawned/_apply_child_event/
_apply_completed/_prune_terminal); the expensive list_all()+merge full scan
now runs only on the backed-off next_full_scan deadline or a detected change.

Tests: revision increments per mutation + is stable on no-op observes;
_overview_change_signature tracks store mtime + registry revision and
tolerates a missing/absent path (None component). _next_poll_interval backoff
tests kept; the 5 _sleep_slices tests deleted with the helper.
@github-actions github-actions Bot added codex:needs-rerun A new push or stale result means @codex review should be requested again. codex:review-requested A Codex review was requested or should be running automatically. status: 🔁 re-review loop A fresh Codex review was requested after the latest change or comment. and removed codex:reviewed Codex posted a review result or thumbs-up after the latest request. status: 👀 ready for maintainer look No automation blocker is known; a maintainer should inspect the current result. labels Jul 4, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b495e128a4

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +539 to +542
if now >= next_full_scan or sig != last_scan_sig:
seen_ids: set[str] = set()
changed = False
for row in await _combined():

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Send keepalive before long overview scans

When the overview stream is idle and next_full_scan wins just before the 10s heartbeat deadline, the loop enters await _combined() before checking the heartbeat again; if store.list_all()/merge/serialization takes more than the remaining heartbeat budget, the connection can still go silent for >10s and strict proxies may close /admin/subagents/events/live. Fresh evidence after the patch is that the new monotonic clock is checked only before this full scan, so the scan itself can still overrun the documented heartbeat cadence; emit a keepalive before starting a scan when the heartbeat is nearly due or split the scan so heartbeats can interleave.

Useful? React with 👍 / 👎.

@github-actions github-actions Bot removed the codex:review-requested A Codex review was requested or should be running automatically. label Jul 4, 2026
@github-actions github-actions Bot added codex:reviewed Codex posted a review result or thumbs-up after the latest request. status: 👀 ready for maintainer look No automation blocker is known; a maintainer should inspect the current result. and removed codex:needs-rerun A new push or stale result means @codex review should be requested again. status: 🔁 re-review loop A fresh Codex review was requested after the latest change or comment. labels Jul 4, 2026
The monotonic-clock loop checks the heartbeat only before a full scan, so
a _combined()+merge that outruns the remaining heartbeat budget could
still silence /admin/subagents/events/live past the 10s rhythm and a
strict proxy would drop it. Now, once silence passes halfway to the
deadline, a keepalive is emitted BEFORE the scan (last_byte resets), so
post-scan silence is bounded by the scan duration alone. Decision
extracted as the pure _prescan_keepalive_due helper + 2 unit tests. 23
passed; ruff+mypy clean.
@github-actions github-actions Bot added codex:needs-rerun A new push or stale result means @codex review should be requested again. codex:review-requested A Codex review was requested or should be running automatically. status: 🔁 re-review loop A fresh Codex review was requested after the latest change or comment. and removed codex:reviewed Codex posted a review result or thumbs-up after the latest request. status: 👀 ready for maintainer look No automation blocker is known; a maintainer should inspect the current result. labels Jul 4, 2026
@sweetcornna
sweetcornna merged commit 3daf7c0 into main Jul 4, 2026
10 checks passed
@github-actions github-actions Bot added status: ✅ merge-ready Evidence and review are clear; normal merge gates may proceed. and removed codex:review-requested A Codex review was requested or should be running automatically. codex:needs-rerun A new push or stale result means @codex review should be requested again. status: 🔁 re-review loop A fresh Codex review was requested after the latest change or comment. labels Jul 4, 2026
sweetcornna added a commit that referenced this pull request Jul 4, 2026
…parse/subagent allowlist)

Each is a DIFFERENT code path than the earlier same-class fixes:

1. shutdown() now reaps each running task's whole group via the shared
   _reap — a daemonized child that outlived the wrapper would otherwise
   keep the pipe open and block the awaited pump indefinitely.
2. The shell_task_output dispatcher's own int(offset) parse catches
   OverflowError too — a non-finite JSON offset (1e10000 → inf) raises
   there BEFORE registry.read()'s seek, so the round-3 seek-side catch
   didn't cover it; now degrades to offset 0 instead of builtin_tool_failed.
3. The subagent runner's _filter_tools_for_child applies the same
   run_shell⇒shell_task_output/kill implication the skill gate got in r4
   (only for task tools the parent actually holds, so the child never
   exceeds its ceiling) — a subagent narrowed to run_shell can now poll
   and terminate its bg tasks.

Also merges origin/main (both #113's alias-collision-warn and this PR's
run_shell skill implication live in _skill_allowed_tools_block — kept
both). 4 new tests. 220 passed; ruff+mypy clean.
sweetcornna added a commit that referenced this pull request Jul 4, 2026
…ell + #108 backend)

Closes claude-code parity Wave 4, merged via #111 (Dim 2), #112 (Dim 4),
#113 (#108 items 3/4/5).

- Dim 2: summary-LLM compaction cooldown + failure/low-savings breaker.
- Dim 4: run_shell run_in_background + shell_task_output/shell_task_kill,
  with session-ownership isolation, 64KiB paged has_more reads, unified
  process-group _reap across all 6 terminal paths (incl. daemonized
  grandchildren + atexit), lifecycle watchdog, evicted-log cleanup, a
  task-control permission model that tracks the start-grant (allow/log/ask
  across scoped/wildcard-deny/default-deny/strict/plan) confined by the
  ownership gate, non-bool/overflow never-raise hardening, and subagent
  safety (bg refused + controls stripped unless explicitly granted).
- #108 items 3/4/5: alias-collision warn, adaptive three-clock overview SSE
  with pre-scan keepalive, public-URL mtime cache.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: ✅ merge-ready Evidence and review are clear; normal merge gates may proceed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant