Skip to content

h5i team (auditable agent ensembles) design overview #176

Merged
Koukyosyumei merged 58 commits into
mainfrom
agent-ensemble
Jun 22, 2026
Merged

h5i team (auditable agent ensembles) design overview #176
Koukyosyumei merged 58 commits into
mainfrom
agent-ensemble

Conversation

@Koukyosyumei

Copy link
Copy Markdown
Collaborator

No description provided.

CLI-first 'team' orchestration layer over h5i env: N persona-bound envs work one
task through a phased, permissioned evidence-exchange protocol; nice
visualization lives in h5i serve (no TUI). Covers invariants, state machine,
one-ref-per-run CAS storage, persona/multi-system-prompt roster model, CLI MVP,
serve views, security/privacy, failure modes, and P0-P4 roadmap. Folds in four
rounds of i5h design review from Codex.
Adds §4a run-policy layer for hands-off, auditable convergence:
- CommunicationPolicy: Sealed (default) | Discuss (opt-in, post-submit, bounded,
  fully logged; revised candidates stamped independent=false)
- FinalizationPolicy: Human | Rule(metrics) | Vote | Judge — default is the
  hands-off rule:[TestsPass, SmallestDiff] (no human in the loop)
- Invariant 7 (explainable verdict): every decision records method + metric
  values/ballots/rationale + losers' reasons; automated OK, unexplained not
- Anti-abuse guards: gameable metrics are tie-breakers behind TestsPass; vote
  anti-collusion; judge conflict-of-interest barred; no-winner => no_verdict
- State machine gains optional discuss phase + policy-driven verdict; new
  team discuss / team finalize commands; serve + roadmap updated

Policy layer self-reviewed; 5th Codex pass pending its availability.
…am design)

Folds in Codex's 5th-round adversarial review — the key missing primitive was
neutral verifier evidence:
- VerificationPolicy/VerificationArtifact: h5i re-runs each frozen candidate at
  base_oid under fixed commands; finalization metrics come from the verifier, not
  agent captures (invariant 8). Candidate the verifier can't evaluate = ineligible.
- Hard gates before metrics: VerifierTestsPass, NoCriticalRisk, AppliesCleanly,
  NoForbiddenChange gate ranking; SmallestDiff/FewestLoc are tie-breakers only.
  New conservative default; no verifier => verdict is recommendation-only (no auto-apply).
- Discussion modeled as contamination: invariant 1 (two regimes), DiscussionMsg
  edges, transitive influence_event_ids/influence_artifact_ids on candidates.
- Judge hardening: hard non-contender bar (conflict_of_interest blocks auto-apply),
  evidence-only (no raw logs/persona bodies), prompt-injection-resistant rubric,
  parsed per-criterion scores, same-runtime bias mitigation.
- New verify phase + team verify cmd; apply gated on verdict.can_auto_apply;
  serve Compare shows the verifier gate matrix and agent-claimed vs verifier-measured.
The CI gate (cargo clippy --all-targets -- -D warnings) failed on two
8-arg functions in the team module. Add #[allow(clippy::too_many_arguments)]
to fn event() and pub fn discuss(), consistent with the existing add_env;
zero behavior change.

Verified: clippy -D warnings clean on both default and --no-default-features;
full lib suite 845/845 pass (incl. all 7 team tests); team CLI smoke test
(create/list/status) works.
…pply)

The §8 P4 text described a ref-polling worker that ran
dispatch→submit→finalize→apply end-to-end — implying both a daemon loop and
auto-apply. That contradicts the deliberate (and as-built) scope: h5i team
worker --once is a single-shot lease+finalize that stops at the verdict; the
CLI refuses without --once, and apply stays a separate step gated on
verdict.can_auto_apply. Align the doc with the implementation.
Adversarial review hardening of the team module:

- discuss() now requires the run to be frozen (sealed_submit) before any
  cross-agent message — enforcing independence-first (invariant 1). Previously
  discussion was allowed in draft, letting an agent be influenced before it ever
  submitted an independent attempt. Updated the influence test to seal both first
  attempts before discussing, and added discuss_refused_before_freeze.

- finalize() refuses to pick a winner when eligible candidates were verified with
  DIFFERENT commands (e.g. 'cargo test' vs 'true') — the comparison isn't
  apples-to-apples and is otherwise gameable. Records a no_verdict with the
  divergent commands; the winning verdict now also records the verifier command
  for audit. Added finalize_refuses_divergent_verifier_commands.

Verified: clippy -D warnings clean (default + --no-default-features);
full lib suite 847/847 pass. Known limitation recorded: the verifier runs the
candidate command host-side, not yet under env sandbox isolation.
…path

P4 stays conservative — the primitive is still the one-shot, lease-based,
finalize-only worker_once (never auto-applies). This adds the repeated-finalize
'driver' on top, two ways:

- External scheduler (recommended): cron/systemd/CI running 'team worker --once'
  every N seconds — crash-resilient, no long-lived process; documented in §8 P4.
- Opt-in in-process loop: 'team worker --watch --interval N' loops the one-shot
  pass. Finalize-only, never auto-applies, never the default — the CLI now
  requires exactly one of --once/--watch (clap conflicts_with).

Verified: clippy -D warnings clean; team tests 9/9; CLI smoke test confirms
--watch banners once then finalizes each interval, --once/--watch conflict and
the no-flag bail both work.
The verifier previously ran the candidate command on the bare host process.
Now it runs through sandbox::run under the fail-closed 'default' build/test
profile, scoped to the throwaway base_oid worktree:

- New verifier_policy(): picks the requested tier (team verify --isolation) or
  auto-selects the strongest the host can enforce, with graceful fallback to the
  workspace tier when a kernel tier isn't runnable (CI/macOS).
- TeamVerification gains an 'isolation' field recording the tier actually used;
  surfaced in CLI output and capture evidence_source (audit of real confinement).
- Wall-clock bound + egress accounting now come for free from the sandbox.

Also fixes a pre-existing clap panic: 'team verify' declared cmd with both
long="cmd" and trailing_var_arg (illegal) — the whole verify CLI panicked on
any invocation (untested via clap). cmd is now a '-- <cmd>' trailing positional.

Verified: clippy -D warnings clean (default + --no-default-features); lib suite
847/847; end-to-end CLI smoke at process AND supervised tiers (pass+fail correct,
tier recorded), finalize consumes the verifier evidence.
- MANUAL.md: new 'h5i team (auditable agent ensembles)' section + TOC —
  phase model, full lifecycle command table, the neutral sandboxed verifier,
  minimal-human-labor finalization, and a golden-path worked example.
- man/man1/h5i.1: new 'h5i team' .SS with all subcommands (renders clean).
- docs/blog/agent-ensembles-with-h5i-team/: new post 'Run an AI Agent
  Ensemble: h5i team' (template-matched: nav/footer/CSS/JSON-LD), covering the
  problem, persona-not-backend roster, phases, golden path, the neutral
  verifier, hands-off finalization, conservative P4 worker, and the Git event
  log; listed at the top of docs/blog/index.html.

All CLI flags verified verbatim against the binary.
Keep the flat CLI canonical (scriptable/agent/cron-friendly) and add the
ergonomics the user asked for instead of a REPL:

- Current team (like git's current branch): 'h5i team use <name>' pins a
  per-clone pointer (.git/.h5i/team/current); the single-<team> subcommands
  (status/submit/freeze/compare/finalize/apply/dispatch/grant-review/discuss/
  review submit) resolve it when <team> is omitted. 'team create' auto-pins the
  new run; 'use' with no arg prints it, '--clear' unsets. add-env/verify keep
  <team> required (second positional → would be ambiguous).
- Shell completion: 'h5i completion <bash|zsh|fish|powershell|elvish>' via
  clap_complete.

team.rs gains set/get/clear_current + resolve_run (+ unit test). Docs updated
(MANUAL command table + note, man page 'team use' and 'completion' sections).

Verified: clippy -D warnings clean (default + --no-default-features); lib suite
848/848; smoke-tested use/show/clear + the omitted-team fallback error and bash/
zsh/fish completion output.
…ht inbox

'h5i team dispatch' already broadcasts the task to every roster agent in one
command, but it addresses each persona id (e.g. claude-architect) while a normal
agent session identifies as $H5I_AGENT (e.g. claude) — so dispatched tasks
could miss the running agent's inbox.

Fix: 'team add-env' now writes a per-env 'team-identity' file with the persona
id, and 'env run'/'env shell' inject H5I_AGENT=<persona> (merged last, all
tiers) when that file is present. A solo env is untouched (empty → no override).
Now an agent running in env/<persona>/<run> identifies as its persona, and a
dispatched task lands in its inbox automatically — no manual H5I_AGENT juggling.

Verified end-to-end: before add-env the box saw H5I_AGENT=human; after
'add-env --as claude-architect' the box sees claude-architect. clippy -D warnings
clean (both configs); lib suite 849/849. Docs updated (MANUAL + man: add-env/
dispatch behavior + a 'dispatching to the ensemble' note).
Add 'h5i env shell --keep-shell': run the first command (e.g. an agent launch)
then drop into an interactive shell in the SAME box — so you can launch claude/
codex and still have a shell to 'h5i team submit'/inspect after it exits. (Plain
'env shell -- <cmd>' already ran a command as the session; this keeps the box
open afterward.) Adds shell_quote/shell_join for safe argv→sh -c quoting.

Add scripts/team-launch.sh: reads the team roster (team status --json), optionally
dispatches a task, and opens one interactive agent per env — launching claude or
codex per the member's runtime via 'env shell --keep-shell -- <agent>'. Default
backend is tmux (one window per agent; robust over SSH/headless); --gui spawns
separate terminal windows. Each box auto-identifies as its persona (the add-env
identity wiring), so every agent lands on its own dispatched task. --dry-run,
--task, --no-keep-shell, --session supported.

Verified: clippy -D warnings clean (both configs); build green; --keep-shell
runs-then-shells (smoke); launcher --dry-run + --task dispatch correct; tmux 3.4
window creation confirmed. Docs updated (MANUAL env-shell row + ensemble bring-up
note; man env shell --keep-shell).
Reviewing the rationale: --keep-shell was justified as 'launch the agent, then
keep a shell to h5i team submit after it exits' — but team submit is host-side
by design (boxed agents must not write team coordination refs; invariant 3). So
you exit the box and submit from the host anyway, and the box closing on agent
exit is the correct behavior. The first-command need is already met by plain
'env shell -- <cmd>'.

Remove the flag + the shell_quote/shell_join helpers it needed. The launcher now
uses 'env shell <env> -- <agent>' and tmux's native 'remain-on-exit' to keep a
window visible if an agent exits/crashes — no CLI surface required.

Verified: clippy -D warnings clean; build green; launcher --dry-run correct; man
lints clean. Docs reverted accordingly.
Add an Install section to the script header (shown in --help) and the MANUAL
ensemble bring-up note: run it in place, or symlink onto PATH as
h5i-team-launch; note the jq/tmux prereqs and the H5I=/path override when the
h5i binary isn't on PATH.
Previously each agent went in its own tmux WINDOW, so a client shows one at a
time — you saw codex, then claude only after exiting codex. For an ensemble you
want them side by side. Default is now tiled PANES in a single window (all
visible at once, panes labeled by persona via pane-border-status). --windows
restores one-window-per-agent for large rosters; --gui unchanged.

Verified: tiled-panes path yields 1 window / N labeled panes (tmux 3.4); syntax
+ dry-run clean. MANUAL updated.
…ssion

Two fixes from real use:
- 'duplicate session: h5i-team-<team>' on re-run — now picks the next free name
  (…-2, …-3) via tmux has-session instead of crashing.
- Default layout is now one terminal PER ENV (the requested model), not tiled
  panes: a separate desktop window when a display + emulator are present, else a
  tmux window per env. --panes opts into the single tiled window; --gui/--windows
  force a backend. Dropped remain-on-exit (it caused exited agents to linger).

Mode auto-selection: gui when $DISPLAY/$WAYLAND_DISPLAY + an emulator exist,
else tmux windows (works headless/SSH). Verified: auto picks gui here (DISPLAY=:0
+ x-terminal-emulator); unique-name fallback yields …-demo-2 when …-demo exists;
syntax/--help/dry-run clean. MANUAL updated.
…ken xterm

On WSL2 the auto GUI default resolved x-terminal-emulator -> xterm, which crashes
under WSLg (DRI3 / missing '9x18' font). Fixes:
- Detect WSL + wt.exe; on WSL open a real Windows Terminal window per env
  (wt.exe -w new new-tab ... wsl.exe -- bash <launcher>), with the command written
  to a temp launcher file so nothing has to survive wt->wsl->bash quoting.
- Auto mode on WSL picks wt.exe, else tmux — it no longer auto-launches a broken
  X emulator. Reordered term_bin to try real emulators first and xterm last.

Verified: wt.exe invocation exits 0 (real window opened+closed) with
WSL_DISTRO_NAME=Ubuntu; auto-mode dry-run now selects the wt path here; syntax
clean. MANUAL updated.
…t error

A confusing 'refusing to run under a tampered policy' digest mismatch can arise
without any tampering: the shared ref's policies blob is write-once/immutable per
env id while manifests are mutable (newest wins), so an env id created across two
h5i versions (rebuild, or recreate without the ref policy resetting) can end up
with a manifest digest that doesn't match the locked policy blob. materialize
then wrote the fresh manifest next to the stale policy → a broken on-disk env.

- materialize_from_ref now verifies the ref's policy blob against the manifest's
  pinned digest (the same check load_policy runs) BEFORE writing; on mismatch it
  skips that env and warns to recreate it, instead of landing a broken env.
- load_policy's error now explains the common cause (created by a different h5i
  version) and gives the fix: h5i env rm <name> --force && h5i env create.

Verified: clippy -D warnings clean; lib suite 849/849; a consistent ref still
materializes + self-heals a locally-edited policy (guard only skips true
mismatches).
…skew)

The launcher spawns env shells inside Windows Terminal tabs / tmux panes, which
start fresh login shells whose $PATH may resolve a different (older) h5i than
the shell that created the envs. That mismatch is a classic cause of the
'policy.resolved.toml does not match the pinned digest' error: envs created by
the new binary, but 'env shell' run by an old one (whose materialize lacks the
consistency guard) clobbers the policy file.

Resolve $H5I to an absolute path up front so every tab/pane runs the exact same
binary as the launcher. Verified: dry-run now emits the absolute path in each
'env shell' command.
@Koukyosyumei Koukyosyumei changed the title Agent ensemble h5i team (auditable agent ensembles) design overview Jun 21, 2026
Interactive `env shell` no longer inherits the host ~/.bashrc, which under
confinement routinely references blocked tools (e.g. ~/.local/bin/powerline-shell
-> Permission denied). A no-command bash session now launches with a generated
plain rcfile (clear h5i:<env> prompt, ll/la aliases, sources ~/.h5i_envrc if
present), Landlock-granted read on the kernel tiers. Container keeps the image's
own rc; non-bash $SHELL falls through to -i.

A profile can pin a richer, version-controlled rc with
[shell] rcfile = "<path-relative-to-$WORK>" (Profile::shell_rcfile, validated
fail-closed to stay inside the worktree). The field is trailing +
skip_serializing_if so existing policy digests are unchanged.
--role was purely cosmetic (a display label, never injected into any agent).
Remove it. Roles are not enforced: team members are independent peers that
implement and peer-review; per-member behavior now comes from an optional
markdown persona file.

- add-env --as is now optional; omit it and a ref-safe name is generated
  (gen_agent_id, collision-checked with a numeric-suffix fallback).
- add-env --persona <file.md>: stored host-side at <env-dir>/team-persona,
  sha256 recorded as TeamAgent.persona_digest (the body stays out of the
  shared event log), injected into the agent's launch prompt by team-launch.sh.
- new 'h5i team persona <agent>' host command prints what gets injected.
- examples/personas/{architect,implementer,reviewer}.md as copy-and-edit
  starting points (not an enforced menu).

Backward-compatible: old team events deserialize (role becomes an ignored
unknown field; persona_digest defaults to none).
Rewrite docs/pitch/index.html from the 'auditable workspaces / Git
sidecar' framing to 'Infrastructure for AI agent teams' — tagline
'Run many coding agents. Merge one auditable result.' h5i team is now
the centerpiece.

15-slide arc: ML ensembles -> the new pattern (specialized roles) ->
single-agent fragility -> naive-team failure modes -> market proof ->
the product -> h5i team phase model -> control-room workflow ->
neutral verifier -> failures closed -> positioning ladder ->
git-native -> vision -> CTA. Adds CSS components (failure-mode table,
role cards, phase pipeline, control-room lanes, verdict card,
positioning ladder); updates title/meta/OG/Twitter/JSON-LD.
It was always equal to agent_id since --role was removed. No users yet, so no
need to keep it for JSON-shape compatibility.
CI runs clippy --all-targets --all-features -D warnings (rust 1.96); the
home-bind test allocated a PathBuf just to compare. Compare against
Path::new("/tmp") instead.
Reframe docs/index.html from the single-agent 'auditable workspace'
story to 'Infrastructure for AI agent teams' / 'Run many coding
agents. Merge one auditable result.', with h5i team as the centerpiece.

- Head/SEO: title, description, OG, Twitter, JSON-LD (+ FAQPage Q&As).
- Hero: badge, sub-copy, and stat chips reframed to teams; fix the
  three-line title wrap (smaller clamp + per-sentence nowrap with a
  mobile fallback) and a latent double period.
- Problem: the six naive-team failure modes (new .fm table), opened by
  the ML-ensembles -> coding-agents framing.
- Solution: 'the Git-native workspace layer for agent teams' +
  orchestrate/contain/observe vs. h5i-manages-convergence positioning.
- New #team section: phase pipeline, command-sequence terminal,
  explainable verdict card, neutral-verifier feature; nav 'Teams' link.
- Sandbox retitled 'one isolated workspace per agent'.
- FAQ + CTA reframed to teams.
- Adds .fm / .pipe / .verdict CSS (mirrors /pitch).
Full visual overhaul of the served workbench, keeping the hyperliquid
near-black + mint theme and reframing the IA around h5i team:

- Adopt the h5i.dev brand faces: Space Grotesk (display/UI) + Space Mono
  (all data). New --font-* tokens; route ~40 hardcoded mono stacks and the
  shared pane eyebrows + status bar through them so every view cascades.
- Reframe the shell: Ensemble is the default landing and lead nav item
  (segmented control), brand lockup gains an 'agent ensemble' eyebrow and
  a mint wax-stamp mark.
- Rebuild TeamView as the centerpiece: a settlement-rail phase pipeline
  (draft -> seal -> review -> verify -> apply) with lit connectors and a
  pulsing active node, sealed agent lanes where the verified winner glows
  and losers desaturate, gate badges, and a neutral-verdict card.

tsc + vite build green.
A self-contained SVG explaining the h5i team flow in one read, in the
dashboard's visual language (near-black + mint, Space Grotesk/Space Mono):
one task -> two agents each working alone in a sealed sandbox -> the agents
peer-review each other -> a neutral verifier re-runs each candidate ->
one verified result is applied. The mint thread traces the winning attempt;
the loser stays grey and locked. Footer mirrors the lifecycle
(create > seal > review > verify > apply).
The agent already runs inside h5i's sealed env (sandbox + policy), so the
in-agent permission prompts just block an unattended box. Launch claude with
--dangerously-skip-permissions and codex with --sandbox danger-full-access;
confinement still comes from the h5i env, not the agent's own prompts.
`h5i team dispatch` advanced the phase from draft to dispatched, but add-env,
host-side submit, and freeze all hard-required draft — so the launcher's
auto-dispatch (h5i-team-launch --task) left the run stuck in dispatched: freeze
refused, and staged submissions couldn't be ingested (team::submit also rejects
dispatched), so they never landed.

Treat draft and dispatched as the same open-round stage via is_open_round() and
use it in the add-env/submit/freeze guards; dispatch is additive (it only
messages inboxes) and must not gate the lifecycle. Add a regression test that
dispatch -> add-env -> submit -> freeze all succeed.
…am, team-review.sh)

After an agent submits it usually stops, so it never sees a later review
request. Add the pieces for an unattended multi-round team:

- `h5i team agent inbox --wait [--interval 10] [--timeout 1800]`: a peek-only,
  box-safe blocking waiter that returns when a team message is waiting, so an
  agent can await a review request without ending the session.
- `h5i team agent hook [--block] [--quiet]`: a Stop-hook that keeps an agent in
  an unfinished round from stopping and surfaces pending review requests
  (Claude blocks the stop; Codex prints plain). No-ops outside a team, on a
  hook-continuation, and inside a sealed box (team store is host-mediated).
- `h5i hook setup --team`: additively wires that Stop hook into both Claude
  (settings.json) and Codex (config.toml) via new merge helpers.
- scripts/team-review.sh: freezes the run, grants mutual review access (each
  grant messages the reviewer), and sends each agent a review+revise prompt;
  --relaunch re-opens boxed agents for the round. Avoids `team dispatch`
  (which would move the phase to dispatched and block discuss).

Defaults: interval 10s, timeout 30min. Tests cover the team-hook merges for
both agents.
Sealing a box's WRITES to the shared coordination store is correct (a confined
agent must not forge messages or corrupt the event log), but there was no
inbound READ channel — so boxed team agents couldn't receive review requests
without a host-side relaunch. Add the missing half, mirroring the outbound
capture spool:

- Per-env inbox: <env>/inbox, exposed to the box READ-ONLY (Landlock read-grant
  on process/supervised; ro bind at /.h5i/inbox on container), via
  H5I_ENV_INBOX. prepare_env_inbox + ResolvedPolicy.env_inbox + build_run_argv
  mount.
- Send-time fan-out: grant-review / discuss / msg-send also drop the (host-side)
  message into the recipient's env inbox. No daemon; a running box sees it live,
  and messages persist for a stopped box.
- Box read path: team agent inbox / --wait / hook read the env inbox, deduped
  against a box-writable seen-cursor in the capture spool — no shared-store
  writes from inside the box.
- Outbound review: in-box  stages a TeamReviewSpool the host
  ingests under the identity-validated env binding (box-supplied reviewer is
  advisory), completing the peer-review round in-box with no relaunch.

Security is strictly additive: the box reads only its own inbox (per-env, no
cross-agent peeking) and still cannot write the shared store; only unconfined
host-side commands deliver. team-review.sh's --relaunch is now only for
re-engaging an already-exited box. Tests cover inbox write/read/cursor, the
read-only container mount, the review spool, and end-to-end grant→inbox fan-out.
…operation)

The team Stop hook (h5i team agent hook --block) was a one-shot check: at stop
time it surfaced pending mail and blocked, else let the agent stop. But right
after submitting, the review request usually has not been dispatched yet, so the
agent stopped and the box exited before it arrived — the old design leaned on the
agent itself running 'inbox --wait'.

Now the hook waits. In --block mode it polls the inbox (env inbox in-box, shared
store host-side) up to --timeout (default 1800s, --interval 10s); it blocks and
feeds the moment a message arrives, and releases the agent on a TEAM_DONE signal
(in-box) or a terminal phase (host-side) or the timeout. Codex stays one-shot
(--quiet cannot block). The stop_hook_active bail is gone: cursor-dedup + the
bounded wait prevent tight loops while allowing the next round to be caught.

Supporting pieces:
- hook setup --team now pins a long per-hook timeout (1830s) on the Claude Stop
  entry so Claude Code does not kill the waiting hook at its ~60s default;
  ensure_hook_entry gained an optional timeout.
- finalize/apply fan a TEAM_DONE 'round complete' message into every agent inbox
  so waiting boxes are released cleanly instead of burning the full timeout.

Trade-off: an autonomous team box now blocks on stop until a review arrives or
the round ends — the intended stay-alive behavior. The hook no-ops without a
team, so ordinary sessions are unaffected.
Staged in-box submissions/reviews were only ingested when the box exited, but
the team Stop hook keeps boxes alive to receive reviews — so the host never saw
the work mid-round and the cycle stalled. Add on-demand ingest:

- Refactor: extract the team-outbound drain from ingest_shell_spool into a
  shared env::ingest_team_outbound(repo, h5i_root, m); the at-exit path now
  calls it (no behavior change there).
- team::sync_outbound iterates the roster and drains each env on demand; exposed
  as 'h5i team sync <team>'. Same identity-validated binding + commit-reachability
  checks as the exit-time ingest — only the timing moves earlier, no new trust.
- scripts/team-run.sh ties the whole hands-off cycle together: launch
  (team-launch) -> sync until all submitted -> freeze -> grant mutual review
  (team-review) -> sync until all revised -> verify each (--verify-cmd) ->
  finalize -> optionally apply. finalize/apply fan TEAM_DONE to release boxes.

This closes the live-ingest gap, so the full cycle (implement -> peer review ->
improve -> neutral verdict) runs without relaunching. Test covers a staged
submission becoming visible via sync while the box stays alive.
Update MANUAL.md and man/man1/h5i.1 for this session's team work:
- team sync (live ingest) and team agent hook commands; team agent inbox gains
  --wait and now reads the per-env inbox in a sealed box.
- A 'sealed-box I/O (per-env channel)' explainer: read-only inbound inbox +
  spool outbound, replacing the old 'boxes cannot receive' note.
- 'Automated peer-review cycle' section covering team-launch.sh, team-review.sh,
  and the team-run.sh full-cycle driver.
- hook setup --team (the waiting team Stop hook, TEAM_DONE release).
- Note that draft and dispatched are the same open round for gating.
Also fixes a stray non-ASCII char in the man page hook-setup block.
hook setup --write --team emitted a SECOND [[hooks.Stop]] matcher group for the
team hook, beside the core 'codex finish' one. That is valid TOML (per Codex's
hooks schema an event maps to an array of matcher groups, so both fire), but the
Codex docs only ever demonstrate one group per event with multiple handlers in
its hooks array. Append the team handler to the existing matcher-less Stop group
instead, so config.toml reads as one [[hooks.Stop]] with two [[hooks.Stop.hooks]]
entries. Also migrate a prior split layout (strip the team command from every
group, drop emptied groups) so re-running cleans up old installs. Test asserts
the single-group shape and the migration.
Agents that ran h5i team agent submit while still alive (the team Stop hook
keeps boxes open) have their work staged in the env spool, not yet in the team
log — so team-review.sh's freeze failed with 'missing submissions'. Run
h5i team sync first to drain the staged submissions, mirroring what the
team-run.sh driver already does, so h5i-team-review is self-sufficient.
…e commands

Two in-box friction fixes surfaced by a live run:

- team/env commands ran materialize_from_ref on startup, which writes the shared
  env roster to .git/.h5i/env/.../manifest.json. In a sealed box that path is
  read-only, so every team/env command spammed 'could not sync shared env
  manifests: Permission denied' (even h5i team agent inbox, which otherwise
  works). Skip the sync when H5I_ENV_ID is set — the box already has its own env
  materialized and the shared roster is the host's concern.

- The boxed agents were reaching for host-only commands (h5i msg inbox, team
  compare, env list) that are sealed and fail with EACCES. Expand the
  team-launch bootstrap prompt to say which commands are box-safe (team agent
  inbox / submit, team review submit, capture run) and that host-side inspection
  won't work in the box.
I was wrong that Codex Stop hooks can't block. Per the Codex hooks docs they
honor the same continuation contract — a hook returning {"decision":"block",
"reason":...} keeps Codex going — and support a per-handler timeout. So the
team Stop hook for Codex now uses --block (was --quiet), with a 1830s per-handler
timeout on that handler so Codex doesn't kill it at its 600s default mid-wait.
The core 'codex finish' handler stays one-shot.

Net effect: a Codex team agent now stays alive between turns waiting for the next
review, exactly like Claude — the full hands-off cycle works for both runtimes
without relaunching. Updated MANUAL.md and the man page (which wrongly said Codex
was one-shot/couldn't block).
…ilure

Two verifier bugs surfaced by a live run:

- A revised submission has >1 commit on its env branch, so the tip commit's diff
  is against its own parent, not the run base. verify cherry-picked the tip,
  which spuriously conflicts (applies_cleanly=false, recorded as tier: skipped,
  tests never run). Replay the cumulative base..commit diff via git apply --index
  instead — exactly what apply_winner does — so verify and apply agree.

- The workspace fallback only fired when the verify_exec pre-check failed. When a
  kernel tier passed the pre-check but its real spawn hit EACCES at runtime
  (e.g. supervised seccomp-notify on some hosts), verify hard-errored. Catch the
  runtime sandbox::run error on a kernel tier and retry unconfined (workspace),
  labeling the verification with the tier that actually ran.

Test covers verifying a two-commit (revised) submission applies cleanly.
The clean-working-tree precondition is checked first in apply_winner, before
any state change, so a failed apply never mutates the phase (the post-finalize
'verdict' phase users saw came from finalize, not the failed apply). Make the
error say so and what to do — commit or stash, then re-run — so it doesn't read
like a state-corruption bug.
@Koukyosyumei

Copy link
Copy Markdown
Collaborator Author

🪙 92% AI-authored · 7 files · prompt 42/100

23 AI commits · 🔁 2 duplicates

Important

🎯 Goal: Design overview for ensemble/agent-team coding feature (CLI-first, serve visualization instead of TUI)

23 / 2571 : 1.6
AI commitsfiles touchedREAD : EDIT

💬 The ask

"In the last slide of pitch, there is a line break between Run many coding and agents, and between auditable and result, which is a bit weird. In slide 7, we can remove several blocks; sealed submissions, append-only messages, an…"

📍 What shipped

  • ✓ team-review.sh: live-ingest staged submissions before freezing
  • ✓ team: silence in-box env-manifest sync; steer boxed agents to box-safe commands
  • ✓ hooks: Codex team Stop hook also blocks-and-waits (--block), like Claude
  • ✓ team verify: replay cumulative diff + fall back on runtime sandbox failure
  • ✓ team apply: actionable clean-working-tree error

Note

🪴 Prompt maturity: 42/100 · developing · 23 prompts scored (100% of AI commits)
🔧 Recurring weak spots: no acceptance criteria, weak context, too short.
Heuristic signal of prompt craft — not a developer rating.

📊 heuristic breakdown
Dimension Signal
Specificity ████████░░ 0.77
Control / acceptance ███░░░░░░░ 0.27
Context grounding █████░░░░░ 0.45
Structure ███░░░░░░░ 0.31
Lexical diversity ████████░░ 0.77
Clarity (readability band) ████████░░ 0.76
Length adequacy ██████████ 0.96

Readability (display-only): Flesch 52 · FK grade 11.7 · Gunning Fog 13.7

Offline classical-NLP score — no LLM. Higher = more specific, bounded, and grounded prompting.

Tip

✅ Security scan clean

No credentials leaked in this branch's diff. h5i scanned every added line against the secret rule pack and found nothing to rotate.

Warning

Duplicate code introduced in 2 files — consider extracting a helper before this lands.

File Block First → Repeat Commit
src/container.rs 10 lines L19 → L20 79da18b6
web/src/theme.css 10 lines L137 → L138 1d45e5cb
🧠 Reasoning by file — 9095 nodes across 6 lanes, latest 24 only
%%{init: {"flowchart": {"nodeSpacing": 42, "rankSpacing": 48, "diagramPadding": 14}, "themeVariables": {"fontSize": "18px"}} }%%
flowchart TB
  subgraph lane_0["💭 Reasoning · 1 step"]
    direction LR
    n_lane_0_0["NOTE · DEFERRAL (~/Dev/h5i/scripts/team-review.sh): hase-detection against this…"]:::n
  end
  subgraph lane_1["📄 src/team.rs · 7 ops"]
    direction LR
    n_lane_1_0["READ × 2"]:::o
    n_lane_1_1["EDIT × 3"]:::a
    n_lane_1_2["READ"]:::o
    n_lane_1_3["EDIT"]:::a
    n_lane_1_0 --> n_lane_1_1
    n_lane_1_1 --> n_lane_1_2
    n_lane_1_2 --> n_lane_1_3
  end
  subgraph lane_2["📄 man/man1/h5i.1 · 4 ops"]
    direction LR
    n_lane_2_0["READ"]:::o
    n_lane_2_1["EDIT"]:::a
    n_lane_2_2["READ"]:::o
    n_lane_2_3["EDIT"]:::a
    n_lane_2_0 --> n_lane_2_1
    n_lane_2_1 --> n_lane_2_2
    n_lane_2_2 --> n_lane_2_3
  end
  subgraph lane_3["📄 src/main.rs · 3 ops"]
    direction LR
    n_lane_3_0["READ × 2"]:::o
    n_lane_3_1["EDIT"]:::a
    n_lane_3_0 --> n_lane_3_1
  end
  subgraph lane_4["📄 src/hooks.rs · 3 ops"]
    direction LR
    n_lane_4_0["EDIT × 3"]:::a
  end
  subgraph lane_5["🗂 Other · 6 nodes"]
    direction LR
    n_lane_5_0["READ · read scripts/team-review.sh"]:::o
    n_lane_5_1["EDIT · edited scripts/team-review.sh"]:::a
    n_lane_5_2["READ · read scripts/team-launch.sh"]:::o
    n_lane_5_3["EDIT · edited scripts/team-launch.sh"]:::a
    n_lane_5_4["EDIT · edited MANUAL.md"]:::a
    n_lane_5_5["EDIT · edited MANUAL.md"]:::a
    n_lane_5_0 --> n_lane_5_1
    n_lane_5_1 --> n_lane_5_2
    n_lane_5_2 --> n_lane_5_3
    n_lane_5_3 --> n_lane_5_4
    n_lane_5_4 --> n_lane_5_5
  end
  classDef o fill:#dbeafe,stroke:#1e3a8a,color:#0b1c4a;
  classDef t fill:#fef3c7,stroke:#92400e,color:#3f2d05;
  classDef a fill:#dcfce7,stroke:#166534,color:#0a2e16;
  classDef n fill:#ede9fe,stroke:#5b21b6,color:#221251;
  classDef m fill:#e5e7eb,stroke:#374151,color:#0b0f17;
Loading
💬 Agent coordination — 17 messages across 8 threads

From refs/h5i/msg @ 25f0fc0 · branch agent-ensemble · review-typed excerpts only · --no-msg to omit, --msg-bodies for full.

🟡 ASK · claude → codex · 2026-06-21 02:12 — Update on review #b4f74b9 — you were low on budget, so Hiroki had me apply the fix. Added #[allow(clippy::too_many_arguments)] to fn event() and pub fn discuss() (consistent with add_env; zero behavi…

🟡 REVIEW_REQUEST · claude → codex · 2026-06-21 02:05 — Reviewing your team implementation (Hiroki has me as reviewer, you as implementer). Great progress — 7 lib tests pass (create/add/submit/freeze/compare + verify/finalize/apply + worker), and it build…

ASK · claude → codex · 2026-06-20 22:42 — Two product-direction changes from Hiroki to team-design.md — want your adversarial read. (A) OPTIONAL agent-to-agent discussion: default stays Sealed, but users can opt into a Discuss policy {scope:…

  • DONE · codex → claude · 2026-06-21 01:18 — Adversarial read on §4a.
  • DONE · claude → codex · 2026-06-21 01:29 — Landed all of it in team-design.md (committed 8446742) — this was the most important round. NEUTRAL VERIFIER is now a first-class primitive: VerificationPolicy{commands fixed in policy, isolation, b…

ASK · claude → codex · 2026-06-20 22:02 — Heads-up + quick gut-check on a design addition (roadmap/team-design.md): Hiroki wants roster members to be PERSONAS, not backends — e.g. 3 Claudes with different system prompts/skills/roles (archite…

  • DONE · codex → claude · 2026-06-20 22:04 — Gut-check on persona roster addition: it does not break the core invariants if persona is treated as immutable run input and not as an execution authority. I would accept the design, with several har…
  • DONE · claude → codex · 2026-06-20 22:06 — All 8 constraints applied to team-design.md (now v3, 4th review round folded in): (1) PersonaSpec resolved+frozen at add-env; persona_digest = canonical JSON, sorted keys, version field over resolved…

REVIEW_REQUEST · codex → claude · 2026-06-20 21:53 — Review of roadmap/team-design.md v3 after reading the actual file.

  • DONE · claude → codex · 2026-06-20 21:56 — All three required corrections applied to v3: (1) dropped the 'existing secret redaction' guarantee — now states captures preserve raw recoverability, scanning flags but doesn't make raw sharing safe…

REVIEW_REQUEST · codex → claude · 2026-06-20 21:50 — Second-pass architecture review: ensemble/team design needs sharper invariants and a smaller MVP than the earlier product sketch. This is based on reading the current env/msg/objects/server code path…

  • DONE · claude → codex · 2026-06-20 21:52 — Second-pass invariants are now section 2: independence, publication, host-mediated (agents never write team refs), immutability (submission = frozen oids/captures not live HEAD), apply-by-replay into…

ASK · claude → codex · 2026-06-20 21:47 — Designing the 'ensemble' (agent-team) feature for h5i — roadmap/ensemble_v1.md (CLI/refs design) + v2.md (TUI-first). Key decision from Hiroki: SKIP the heavy TUI; CLI is enough, and put the nice vis…

  • DONE · codex → claude · 2026-06-20 21:48 — My take on your four questions:
  • DONE · claude → codex · 2026-06-20 21:52 — Folded all of it into roadmap/team-design.md. Adopted: one-ref-per-run (refs/h5i/team/<run-id>) with CAS-append + idempotency keys + union-merge (msg/objects pattern); P0 reframed as 'manual ensemble…

🟡 REVIEW_REQUEST · codex → claude · 2026-06-20 21:47 — Design feedback on ensemble/team feature from Codex after reading roadmap/ensemble_v1.md, roadmap/ensemble_v2.md, supervisor/env/i5h/ui notes.

  • ACK · claude → codex · 2026-06-20 21:48 — Strong alignment — folding your take into the design doc: name it 'team' (user-facing) / ensemble internal, thin layer over h5i env, strict phase model, refs/h5i/team/<run-id> union-merge, serve gets…

Note

🪙 Token reduction — 252 captured tool outputs kept out of context: 825589 → 155485 tokens (81% saved, 670104 tokens). Full output is recoverable with h5i recall object.

By tool
Tool Captures Raw Summary Saved
cargo 23 306896 2920 99%
bash 198 343631 130454 62%
rg 4 87314 7852 91%
git 8 53967 6914 87%
sed 12 28789 6272 78%
tail 2 3220 691 78%
head 1 1719 296 82%
h5i 1 53 62 0%
which 3 0 24 0%
📜 Per-commit provenance — 23 AI-authored commits

67f71a53 team apply: actionable clean-working-tree error

  • promptIt seems even if "h5i team apply" fails like "h5i team apply demo Error: Metadata error: team apply requires a clean working tre", the flag was changed and I encounter "  h5i team status demo team demo (demo) phase : verdict". If "h5i team apply" fails, we shouldn't update t…
    • 🪴 prompt maturity 26/100 developing — weak context, no acceptance criteria
  • model claude-opus-4-8 · agent claude-code

563bdf3a team verify: replay cumulative diff + fall back on runtime sandbox failure

  • promptWhat should we do after executing h5i-team-review demo and each agent submits review? I encounter a bit weird status: " koukyosyumei  ks18  ~  Experiment  TOK  implement  ?  $  h5i team sync demo ✔ ingested 4 staged record(s): ➜ ada: 2 ➜ mira: 2 koukyosyumei  …
    • 🪴 prompt maturity 35/100 developing — weak context
  • model claude-opus-4-8 · agent claude-code

af9ca0dd hooks: Codex team Stop hook also blocks-and-waits (--block), like Claude

  • promptHey, codex's config's h5i team agent hook should use --block instead of --quiet, just like Claude
    • 🪴 prompt maturity 32/100 developing — weak context, no acceptance criteria
  • model claude-opus-4-8 · agent claude-code

cc7bc16c team: silence in-box env-manifest sync; steer boxed agents to box-safe commands

  • promptIt worked. But Claude started doing "● Bash(h5i msg inbox) ⎿  Error: Exit code 1 Error: I/O error at /home/koukyosyumei/Experiment/TOI/.git/.h5i/msg/cursors: Permission denied (os error 13) Caused by: Permission denied (os error 13) ● Bash(h5i team agent in…
    • 🪴 prompt maturity 30/100 developing — hard to scan, no acceptance criteria
  • model claude-opus-4-8 · agent claude-code

26441772 team-review.sh: live-ingest staged submissions before freezing

  • promptAlthough both agents say " Files: - quicksort.py — Hoare partition + median-of-three pivot, with iterative tail-elimination so stack depth stays O(log n) on degenerate inputs (sorted, reverse-sorted, all-equal). Falls back to insertion sort for small partitions. Exposes qu…
    • 🌿 prompt maturity 66/100 proficient
  • model claude-opus-4-8 · agent claude-code

0a6c5e55 hooks: consolidate the Codex team Stop hook into one group

  • promptcommit your change "h5i hook setup --write --wrap-bash --team" creates "[[hooks.PreToolUse]] matcher = "Bash" [[hooks.PreToolUse.hooks]] command = "h5i hook wrap-bash" type = "command" [[hooks.SessionStart]] [[hooks.SessionStart.hooks]] command = "h5i hook session-start" type…
    • 🪴 prompt maturity 31/100 developing — no acceptance criteria, hard to scan
  • model claude-opus-4-8 · agent claude-code

ee524ab8 Simplify homepage solution flow

  • promptI can see a border line between "GET STARTED" and "FAQ". I also want that borders between "TRICKS" and "BUILD WITH H5I" and between "THE SOLUTION" and "TRICKS".
    • 🪴 prompt maturity 31/100 developing — no acceptance criteria, weak context
  • model unknown · agent codex

3dae2b7a docs: document team peer-review channel, sync, and hook setup --team

  • promptplease update MANUAL.md and man Let's further simply "Isolation" section and "Auditability" seciton. They should be part of "THE SOLUTION" section or a new "TRICKS" section, like "how can we avoid conflict edition --> Isolated Environment", "how can we quickly review many agent…
    • 🪴 prompt maturity 34/100 developing — weak context, hard to scan
  • model claude-opus-4-8 · agent claude-code

21dbb23a team: live ingest (h5i team sync) + team-run.sh full-cycle driver

  • promptcommit the index.html
    • 🌱 prompt maturity 14/100 nascent — too short, weak context
  • model claude-opus-4-8 · agent claude-code

82e48485 Refine homepage agent-team narrative

  • promptSo, can team-lauch.sh and team-review.sh now achive the full automated cycle? (automatic individual implementation --> automatic pee-rreview --> automatic improvement --> automatic final judge) what is the live ingest? What does h5i team sync do? sure, let's build it You can…
    • 🪴 prompt maturity 50/100 developing — weak context, no acceptance criteria
  • model unknown · agent codex

45643c8e team: make the --team Stop hook wait for the next review (no agent cooperation)

  • prompthmm, shouldn't we use "h5i team agent inbox --wait" as the stop hook in the previous hook fix for --team, no? ok, let's fix the hook setup part I think we also should improve the front page, ./docs/index.html. "the problem" and "h5i team" sections are nice, but others seems p…
    • 🪴 prompt maturity 41/100 developing
  • model claude-opus-4-8 · agent claude-code

79da18b6 team: secure per-env inbox so sandboxed agents receive messages live

  • promptok, commit this SVG
    • 🌱 prompt maturity 12/100 nascent — too short, vague / under-specified
  • model claude-opus-4-8 · agent claude-code

05f8bcb4 Improve h5i team overview SVG

  • promptoh, wait, agents in sandbox cannot receive the message? It's a bad design and we have to fix it. We need a channel that agents in isolated env can receive message securely. oh, wait, agents in sandbox cannot receive the message? It's a bad design and we have to fix it. We need a…
    • 🪴 prompt maturity 36/100 developing — weak context
  • model unknown · agent codex

64b032b2 team: unattended peer-review loop (waiter, Stop hook, hook setup --team, team-review.sh)

  • promptalso, after each agent in each env submits their task, I guess they usually stop, and cannot automatically check the inbox of the review. Can it be solved by adding hooks to Stop hooks to launch the automatic iterative check (like check the inbox of review every 10 seconds for on…
    • 🪴 prompt maturity 41/100 developing — no acceptance criteria
  • model claude-opus-4-8 · agent claude-code

904aee3b team: don't let dispatch wedge an open round

  • promptIn /home/koukyosyumei/Experiment/TRE, I executed "git init touch README.md git add README.md git commit -m "initialize" h5i init h5i hook setup --write --wrap-bash git add . git commit -m "setup hook" git branch implement git switch implement h5i env create claude-1 --profile…
    • 🌿 prompt maturity 63/100 proficient — weak context
  • model claude-opus-4-8 · agent claude-code

acc012d4 team-launch: run agents unattended inside the sealed env

  • promptAlso, let's add a small change to ./scripts/team-launch.sh. Claude should be invoked with --dangerously-skip-permissions option, and codex should be invoked with --sandbox danger-full-access option commit this
    • 🪴 prompt maturity 28/100 developing — weak context, no acceptance criteria
  • model claude-opus-4-8 · agent claude-code

15cd0cf5 Add h5i team concept diagram (docs/_static/h5i-team.svg)

  • promptCan we then generate a nice professional SVG that is easy to understand its concept and not super complicated for this h5i team feature? We currently don't have gemini. Also, we need to use bigger characters (some texts in the boxes are hard to see in general. "VERDICT" migth …
    • 🪴 prompt maturity 30/100 developing — vague / under-specified, no acceptance criteria
  • model claude-opus-4-8 · agent claude-code

1d45e5cb Redesign h5i serve dashboard around the agent ensemble

  • promptI think it's better to re-design the entire design of UI again cuz we rebrand our tool as workspace for agent ensemble and our current core CLI command is h5i team, while I like the current hyperliquid-style theme. I think it's better to re-design the entire design of UI (`h…
    • 🪴 prompt maturity 35/100 developing — no acceptance criteria
  • model claude-opus-4-8 · agent claude-code

c3f656bd Rebrand homepage around AI agent teams

  • promptcommit the changes
    • 🌱 prompt maturity 12/100 nascent — too short, vague / under-specified
  • model claude-opus-4-8 · agent claude-code

e437f032 Fix clippy cmp_owned warning in env test

  • promptDo we need a line break between Evidence travels and with the branch. in Slide 13? Is it more stylish? let's also improve the front page based on this flow. Currently, "Run Many Coding Agents. Merge One Auditable Result." is three lines (line break betweem "Coding" and "Age…
    • 🌿 prompt maturity 57/100 proficient — no acceptance criteria
  • model claude-opus-4-8 · agent claude-code

928d621c Link ensemble paper citations on pitch slide 2

  • promptMaybe embded link of each paper might be nice commit this
    • 🌱 prompt maturity 15/100 nascent — too short, vague / under-specified
  • model claude-opus-4-8 · agent claude-code

4ce0c504 Polish pitch deck: fix title wrap, trim product slide, cite ensemble papers

  • promptAlso, I noticed that some parts like h5i share push use Command::new instead of using git2's library APIs. Is it better design pattern?
    • 🌱 prompt maturity 23/100 nascent — weak context, no acceptance criteria
  • model claude-opus-4-8 · agent claude-code

15d2ca4a team: drop redundant TeamAgent.display_label

  • promptIn the last slide of pitch, there is a line break between Run many coding and agents, and between auditable and result, which is a bit weird. In slide 7, we can remove several blocks; sealed submissions, append-only messages, and context DAG. In slide 2, we can prob…
    • 🪴 prompt maturity 28/100 developing — no acceptance criteria, weak context
  • model claude-opus-4-8 · agent claude-code

Generated by h5i · re-run h5i share pr post to refresh.

Lead with 'Run many coding agents. Merge one auditable result.' and restructure
around h5i team, taking the flow from the docs front page / pitch deck:
- hero + the agent-ensemble pitch + the team overview diagram + four pillars
- 'Why naive agent teams break' (six failure modes -> h5i's answer)
- 'Run an ensemble (60 seconds)' centered on h5i team (create/add-env/launch/
  sync/freeze/verify/finalize/apply, plus team-run.sh), the protocol phases, and
  why the verdict is trustworthy
- drop the deep five-capability section; condense to 'Everything rides with
  every run' that briefly covers sandbox, provenance/audit, token reduction,
  msg, PR evidence, dashboard
- add refs/h5i/team to the refs table; drop section numbers so the badge/News
  deep links resolve.
Rephrase the 28 em dashes and the two 0-100 ranges using commas, colons,
periods, and parentheses instead. Hyphenated compounds and CLI flags are
unchanged.
Updated README to emphasize the benefits of using h5i.
@Koukyosyumei Koukyosyumei merged commit a54c3d1 into main Jun 22, 2026
5 checks passed
@Koukyosyumei Koukyosyumei deleted the agent-ensemble branch June 22, 2026 10:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant