Repository navigation
PR-7: SDK, harness, runtime and worker coverage sweep beyond Claude and Codex (2026-09-22) - #68
Merged
seathatflowsinourveins merged 2 commits intoSep 22, 2026
Conversation
Full-coverage discovery beyond Claude Code and Codex across six layers (agent SDKs, coding harnesses, durable runtimes, sandbox and worker runtimes, dispatch frameworks, protocol SDKs): 356 repositories found, 38 shortlisted by three-judge majority with the 334 rejections and their reasons retained, an evidence packet and two refutation lenses per shortlisted repository, three selection judges per category on preregistered criteria C1-C6, a completeness critic, and seven coordinator resolutions applied in code. Result: 1 integrate_now (modelcontextprotocol/inspector, executed as a bounded MCP conformance check on the WSL2 host), 6 targeted_candidate, 19 keep_but_compare, 6 not_adopted, 6 refuted. Source review otherwise; nothing here changes a landscape verdict row (PR-5 owns those). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
… gate's failure path, apply the review Resolves the PR #68 review: all 38 shortlisted rows now point at their evidence packet and both refutation votes in per-category artifacts under evidence/artifacts/sdk-runtime-coverage-20260922/ (split so no published file exceeds the guarded Gitleaks 2 MB cap); the codex-acp override is reverted to its category majority and the per-category disposition policy is stated; the container-use override is named as a canonical-record precedence rule; the incumbent list C5 is measured against is recorded; the evidence-class statement names the judges' unretained spot probes; the critic section is labelled pre-resolution with a status list; the newest published release replaces moving-tag latest-release values; host details (user name and groups, credential file locations, token scopes) are redacted. The MCP conformance check's --strict exit-6 path is now observed with a two-mode fixture server (agent-lab tools/mcp-conformance). Counts: integrate_now 1, targeted_candidate 6, keep_but_compare 18, not_adopted 7, refuted 6. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
added a commit
that referenced
this pull request
Sep 22, 2026
…ates (#70) Follow-up to #68: the verified viability probes for three targeted SDK/harness candidates. Adds `evidence/artifacts/sdk-runtime-coverage-20260922/probes-20260922.json` (sanitised executor receipts plus the independent verifier's findings and corrections), a `viability_probes` block in the coverage ledger, a Markdown section and a task-record note. | Repository | Supported by the evidence | Open defects | Effect | | --- | --- | --- | --- | | TypeScript Agent SDK 0.3.280 | `query()` drove the native `claude` binary with `apiKeySource: "none"`; fixture write and a resumed session | `allowedTools` only auto-approves, so every host tool was exposed and host SessionStart hooks ran (no `settingSources`/`tools`); session files and ai-memory hook rows written under the user home | `api_key_required` C1 label does not hold on this host; stays keep_but_compare | | `awslabs/cli-agent-orchestrator` v2.5.1 | isolated install; `CAO_HOME_DIR` relocated CAO's own state (not the worker's); plan-mode Claude worker on the native login | waiting worker reported `idle`; `cao shutdown` hung; worker CLI state written under the user home | stays targeted_candidate | | `untrivial-ai/agent-orchestrator` v0.13.0 | `ao daemon` ran with no Electron process on loopback and stopped cleanly (display independence untested: `DISPLAY` set) | unprompted import of the host Codex login (refresh token included) into its data directory, a Codex account read, host Codex credential files rewritten a second later; parent-death auto-stop failed; probe copy deleted after review | stays targeted_candidate with a credential-import finding | Evidence class: native execution on one WSL2 x86_64 host, pinned versions in a scratch prefix, bounded runner, native Claude login (API-key variables reported unset by the executors, logged only for the Anthropic variables in the TypeScript SDK run); each receipt verified against raw logs (all partially confirmed; the verifier's corrections are recorded). Viability only; no disposition changes; nothing promoted. Review: independent evidence-reviewer pass found the credential import and several summary overclaims; fixed in the second commit. Gate: `validate.py` passed (1,764 hashed files, 140 receipts); `build_ecosystem.py --check` and `validate_catalogs.py` passed; `python3 -m unittest` 1,428 OK (204 skipped); guarded Gitleaks no leaks. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
deleted the
claude/sdk-runtime-coverage-20260922
branch
September 25, 2026 18:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Full-coverage sweep of the SDK, harness, runtime and worker layers beyond Claude Code and Codex, with preregistered selection. Adds
catalogs/sota-convergence/sdk-runtime-coverage-20260922.{json,md}, six per-category evidence artifacts underevidence/artifacts/sdk-runtime-coverage-20260922/(every shortlisted row's evidence packet and both refutation votes) anddocs/tasks/2026-09-22-sdk-runtime-coverage.md; registers the nine files inmanifests/evidence.json. No landscape verdict row, handbook section or SOTA manifest entry is edited (PR-5 and #66 own those); the ledger is the input for the owners ofagent-sdks,workers,isolationandmcp-surfaces.gh api, no model)Dispositions after six coordinator resolutions (all listed in the ledger): integrate_now 1 (
modelcontextprotocol/inspector), targeted_candidate 6 (awslabs/cli-agent-orchestrator,swe-agent/mini-swe-agent,anthropics/claude-code-action,untrivial-ai/agent-orchestrator,trailofbits/coop,snyk/agent-scan), keep_but_compare 18, not_adopted 7, refuted 6. Agreement never promoted anything; every row keeps its named comparison or acceptance command. Dispositions are per (category, repository); the one cross-category override (dagger/container-use) follows a named canonical-record precedence rule.Evidence class: source review, plus unretained spot probes some judges ran (named in the ledger), plus one retained local_integration execution for the integrate_now row: agent-lab
tools/mcp-conformance/check.sh(serena 22, ai-memory 23, socraticode 26 tools,--strictclean;--self-testobserves the failing path with a fixture server whose unportable schema exits 6). No macOS execution for any row; incumbents' capabilities were not re-exercised.Gate:
validate.pypassed (69 components, 1,763 hashed files, 140 receipts);build_ecosystem.py --check,validate_catalogs.py,validate_foundation.py,validate_convergence.py --all-recordedpassed;python3 -m unittest1,428 OK (204 skipped); guarded Gitleaks over both commits: no leaks (2 MB target cap, which now excludes nothing, and decode depth 0). Session identifiers, task handles, local paths, the local user name and groups, credential file locations and token scopes were redacted from worker output.🤖 Generated with Claude Code