Skip to content

PR-8: verified viability probes for three targeted SDK/harness candidates (2026-09-22) - #70

Merged
seathatflowsinourveins merged 3 commits into
mainfrom
claude/sdk-coverage-probes-20260922
Sep 22, 2026
Merged

seathatflowsinourveins merged 3 commits into
mainfrom
claude/sdk-coverage-probes-20260922

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Sep 22, 2026 •

Copy link
Copy Markdown
Collaborator

Follow-up to #68: the verified viability probes for three targeted SDK/harness candidates.

Adds evidence/artifacts/sdk-runtime-coverage-20260922/probes-20260922.json (sanitised executor receipts plus the independent verifier's findings and corrections), a viability_probes block in the coverage ledger, a Markdown section and a task-record note.

Repository Supported by the evidence Open defects Effect
TypeScript Agent SDK 0.3.280 query() drove the native claude binary with apiKeySource: "none"; fixture write and a resumed session allowedTools only auto-approves, so every host tool was exposed and host SessionStart hooks ran (no settingSources/tools); session files and ai-memory hook rows written under the user home api_key_required C1 label does not hold on this host; stays keep_but_compare
awslabs/cli-agent-orchestrator v2.5.1 isolated install; CAO_HOME_DIR relocated CAO's own state (not the worker's); plan-mode Claude worker on the native login waiting worker reported idle; cao shutdown hung; worker CLI state written under the user home stays targeted_candidate
untrivial-ai/agent-orchestrator v0.13.0 ao daemon ran with no Electron process on loopback and stopped cleanly (display independence untested: DISPLAY set) unprompted import of the host Codex login (refresh token included) into its data directory, a Codex account read, host Codex credential files rewritten a second later; parent-death auto-stop failed; probe copy deleted after review stays targeted_candidate with a credential-import finding

Evidence class: native execution on one WSL2 x86_64 host, pinned versions in a scratch prefix, bounded runner, native Claude login (API-key variables reported unset by the executors, logged only for the Anthropic variables in the TypeScript SDK run); each receipt verified against raw logs (all partially confirmed; the verifier's corrections are recorded). Viability only; no disposition changes; nothing promoted.

Review: independent evidence-reviewer pass found the credential import and several summary overclaims; fixed in the second commit.

Gate: validate.py passed (1,764 hashed files, 140 receipts); build_ecosystem.py --check and validate_catalogs.py passed; python3 -m unittest 1,428 OK (204 skipped); guarded Gitleaks no leaks.

🤖 Generated with Claude Code

…andidates

Adds the sanitised receipts and independent verifications of three bounded
probes on the WSL2 host (evidence/artifacts/sdk-runtime-coverage-20260922/
probes-20260922.json), a viability_probes block in the coverage ledger, a
Markdown section and a task-record note. The TypeScript Agent SDK 0.3.280
ran on the native Claude Code login with no API key (fixture write and a
resumed session), so its api_key_required C1 label does not hold on this
host; cli-agent-orchestrator v2.5.1 launched a plan-mode worker on the
native login but misreported its waiting state; the agent-orchestrator
v0.13.0 daemon ran headless on loopback but read the host's Codex account
unprompted. All three receipts were partially confirmed by the verifier and
its corrections are recorded; no disposition changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
… summaries

Resolves the PR #70 review: the agent-orchestrator daemon imported the
host Codex login (refresh token included) into its probe data directory
without being asked, which the verifier had missed; the ledger, Markdown
table and task note now carry it, and the probe copy was deleted. The
API-key wording now says the variables were reported unset by the
executors and logged only for the Anthropic variables in the TypeScript
SDK run. The Markdown rows carry every defect the verifier named, the
headless and isolation wording matches the ledger, the remaining home-dir
leftovers are listed, and session-id prefixes, the subscription tier and
the usage line are redacted from the artifact.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant