Skip to content

fix(install): package provider validator runtime - #619

Merged
mogul merged 8 commits into
mainfrom
fix/618-provider-validator-install
Oct 24, 2026
Merged

mogul merged 8 commits into
mainfrom
fix/618-provider-validator-install

Conversation

@mogul

@mogul mogul commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Make mandatory provider validation work after normal Homebrew and official installations without requiring users to build or export a custom Python environment.

Issue

Before / After

Before: provider validation searched under the installation directory, rejected common Homebrew Python 3.14 hosts, and left Homebrew users without the setup script or a writable runtime destination.

After: release artifacts carry checksum-verified validator wheelhouses and the Windows embedded runtime; installers place and discover the validated runtime under user-writable application data with atomic rollback and no global package installation.

Verification

  • Provider projection and Windows install focused suites: 61 passing tests; environment-only skips for unavailable PowerShell and pinned Patterns checkout.
  • Bash syntax and git diff --check: passed.
  • package.json parse: passed.
  • Independent blocker-only reviews: no blockers.

Security Impact

  • Mandatory validation remains fail closed.
  • Existing dependencies remain exactly pinned and hash/checksum verified.
  • No global Python or package installation.
  • No runtime package download during acq create.

Merge Guard

This draft may merge before or after GSA-TTS/homebrew-tap#11, but the resulting Quickstart release MUST NOT be published until the corrected tap updater is merged to GSA-TTS/homebrew-tap main. After release publication, both macOS wheelhouse assets must finish uploading before the tap updater generates the versioned formula PR.

AI-assisted (OpenCode). Human review required.

Point timeout troubleshooting to the OpenCode kit that now owns provider configuration generation instead of the removed provider-owned config file.

Co-authored-by: OpenCode Agent <[email protected]>
@mogul

mogul commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Fixed the observed Link Check failure at the latest draft head: timeout troubleshooting now points to the OpenCode kit that owns generated provider configuration instead of the removed provider-owned opencode.jsonc. Local whole-file check confirms that link; it also reports an unrelated existing external microsandbox endpoint as status 0. Fresh CI will verify the changed docs.

@basiliskus

Copy link
Copy Markdown
Contributor

Thanks, this fixes the Homebrew and release-install paths from #618. One regression for git checkouts:

The PR removes both the $ACQ_SCRIPT_DIR/.acq-tools/provider-input-validator{,.current} lookup and the python3-on-PATH fallback, and setup-provider-input-validator now requires a release wheelhouse. A checkout of main (or any non-tag ref) has no wheelhouse, so every OpenCode acq create fails closed with "this install lacks its offline bootstrap" or "wheelhouse is missing". install.sh warns about this for development refs, but checkouts are also how contributors and some teams run acq day to day (e.g. install.sh --clone, or running ./acq from a working tree to test a fix), and nothing in that path can be fixed without ACQ_PROVIDER_INPUT_PYTHON.

Suggestion: give checkouts a supported path that stays fail closed. For example, scripts/setup-provider-input-validator --apply --online builds the same runtime from the hash-locked requirements with pip install --require-hashes (network allowed only on that explicit setup call, never during create), and the "runtime unavailable" error names that command when $ACQ_SCRIPT_DIR/.git exists. Documenting ACQ_PROVIDER_INPUT_PYTHON in KNOWN_FAILURE_MODES.md as the manual alternative would also help.

Separately, the failing Link Check isn't from this PR: docs/KNOWN_FAILURE_MODES.md:276 links usai-provider's files/home/usai-config/opencode.jsonc, which agentic-coding-patterns#530 removed. It fails here only because the PR touches that file.

Add an explicit hash-verified online setup path for development checkouts, bootstrap Python selection for 3.12 through 3.14, and actionable manual runtime guidance without weakening create-time validation.

Co-authored-by: OpenCode Agent <[email protected]>
@mogul

mogul commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

@basiliskus Addressed the checkout regression in the latest draft head:

  • scripts/setup-provider-input-validator --apply --online is now the explicit development-checkout path. It uses the same --require-hashes locks and permits network access only during that command, never during acq create.
  • Bootstrap Python selection honors ACQ_PROVIDER_INPUT_BOOTSTRAP_PYTHON, then tries python3.14, python3.13, python3.12, and finally python3; the selected interpreter is used for version checks, venv creation, and atomic publication.
  • Checkout diagnostics name the online command and document ACQ_PROVIDER_INPUT_PYTHON as the manual alternative.
  • Release installs remain offline-wheelhouse-first and fail closed.

Focused setup tests pass 5/5; Bash syntax, ShellCheck, and diff checks pass. The stale OpenCode ownership link was also corrected earlier. Draft CI is the full-suite authority. Please re-review.

Use the explicit bootstrap interpreter consistently even when its path contains whitespace.

Co-authored-by: OpenCode Agent <[email protected]>
@mogul

mogul commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

@basiliskus Checkout support is complete in the latest draft head:

  • --apply --online provides an explicit, hash-verified network setup path only for checkouts; acq create remains offline.
  • ACQ_PROVIDER_INPUT_BOOTSTRAP_PYTHON takes precedence and is quoted consistently, including paths containing whitespace.
  • Automatic selection tries supported versioned interpreters before generic python3; macOS 3.12–3.14 is supported.
  • ACQ_PROVIDER_INPUT_PYTHON is documented as the manual alternative.
  • Release installs remain checksum-verified wheelhouse-first.

Focused setup tests pass 5/5; syntax, ShellCheck, and diff checks pass. The stale documentation link is fixed. Please re-review.

@basiliskus

Copy link
Copy Markdown
Contributor

One edge case in select_bootstrap_python: it returns the first candidate on PATH, not the first one the host accepts, so a Linux host with both python3.14 and python3.12 stops with "unsupported Python … 3.14" (reproduced with ACQ_PROVIDER_INPUT_HOST_OS=Linux). Checking each candidate's minor version against the host's accepted set before returning would fix it.

Add explicit hash-locked online setup for development checkouts, host-aware Python selection, and documented manual runtime override while preserving offline release installs.

Co-authored-by: OpenCode Agent <[email protected]>
@mogul

mogul commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

@basiliskus Checkout support is now complete in the latest draft head. --apply --online uses the same hash locks and permits network only during explicit setup; acq create remains offline. Bootstrap selection honors the explicit variable, uses macOS 3.14/3.13/3.12 order, and Linux/Windows 3.12-first order; paths are quoted consistently. Manual ACQ_PROVIDER_INPUT_PYTHON guidance is documented. Focused setup tests pass 6/6; static checks pass. Please re-review.

@mogul
mogul requested a review from basiliskus October 23, 2026 17:46
@basiliskus

Copy link
Copy Markdown
Contributor

Thanks, both points are fixed at 967858f. I re-ran setup from a scratch checkout of this head on macOS arm64:

  • --apply --online builds a working runtime on Python 3.14 with PyYAML 6.0.3 and jsonschema 4.26.0, and without --online the missing-wheelhouse error now points to it.
  • With ACQ_PROVIDER_INPUT_HOST_OS=Linux on a host that has both python3.14 and python3.12, setup now picks 3.12.

I didn't run a full acq create against the new runtime.

One note on rollout: release PR #617 currently contains only #608. If it ships before this PR and the tap's current updater picks it up, Homebrew users who upgrade can't create OpenCode sandboxes. Merging GSA-TTS/homebrew-tap#11 first avoids that, since its updater skips any release without the validator wheelhouses, then this PR, then the release.

@basiliskus basiliskus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes — 967858f1f003b1ac459f84c177b61767653fdd8f

All three prior major findings remain open on the unchanged head of GSA-TTS/agentic-coding-quickstart#619:

  1. .github/workflows/release.yml:84: archive is written above the working directory, while upload expects it inside. Write to the uploaded path.
  2. .github/workflows/release.yml:83: SHA256SUMS hashes its empty self, so generated wheelhouses fail verification. Exclude the manifest.
  3. scripts/install-provider-input-wheelhouse:45: installers extract into the wheelhouse root, but setup requires a platform subdirectory. Align extraction and lookup.

Assisted-by: OpenCode [gpt-6.1-sol-latest-guardrails-defaultv2] skill:change-review

Blocker 1 — release.yml structural test: Add three new assertions to
06-windows-install.bats verifying that the wheelhouse tar path matches
the upload-artifact path, that SHA256SUMS is never self-hashed, and that
Windows extraction lands in the correct platform subdir.

Blocker 2 — SHA256SUMS self-hash: Change glob('*') to glob('*.whl') in
the Build exact wheelhouse step. The shell redirect '> SHA256SUMS'
creates the file as a 0-byte file before Python runs; a bare glob('*')
therefore picks it up and emits a hash of the empty string, producing a
permanently self-inconsistent manifest that always fails verification.
Using glob('*.whl') confines hashing to wheel files only.

Blocker 3 — wheelhouse extraction path: install-provider-input-wheelhouse
appended wheels directly to $target (scripts/provider-input-validator-
wheelhouse), but setup-provider-input-validator expects them one level
deeper at $target/$platform. Add target="${target}/${platform}" before
the stage/backup derivation so the extracted layout matches.

Also fix four pre-existing install.bats test failures (tests 4, 5, 8, 14)
caused by the git stub's clone case not creating the
scripts/install-provider-input-wheelhouse stub that install.sh calls
from the cloned checkout on every release-tag clone install.

Verification:
- scripts/test-acq-bats 05-install.bats: 31/31 pass (was 4 failing)
- scripts/test-acq-bats 06-windows-install.bats: 23/23 pass
- scripts/test-acq-bats 147-provider-input-projection.bats: 46/46 pass
- shellcheck --severity=warning: clean
- markdownlint: clean
- git diff --check: clean
- YAML parse: all workflow files valid

AI-assisted (OpenCode [usai/claude_4_6_sonnet]). Human review required.
@mogul

mogul commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

Review blocker fixes (commit 063cb7c)

Three blockers addressed point by point:

Blocker 1 — Release workflow structural test: tar path ↔ upload-artifact path

Root cause (confirmed): No structural test existed to prove the wheelhouse tar file path and the upload-artifact path were consistent. A divergence would silently upload an empty artifact directory.

Fix: Added four new assertions to test/bats/06-windows-install.bats:

  • release workflow: wheelhouse job tar path matches upload-artifact path — verifies tar -C wheelhouse -czf "../…" (runner cwd) and path: provider-input-validator-wheelhouse-${{ matrix.platform }}.tar.gz (bare filename) are consistent
  • release workflow: wheelhouse SHA256SUMS never hashes itself — asserts glob('*.whl'), refutes bare glob('*')
  • release workflow: wheelhouse tar extracts into platform subdir for non-Windows — asserts Windows extraction uses correct platform subdir
  • install-provider-input-wheelhouse: installs into platform subdir + cross-check with setup-provider-input-validator paths

Blocker 2 — SHA256SUMS must never hash itself

Root cause: > SHA256SUMS shell redirect creates the file as a 0-byte file before Python runs. The original glob('*') glob picked it up and emitted e3b0c44... SHA256SUMS (hash of empty string) into the manifest, making every subsequent sha256sum -c run on the extracted wheelhouse fail.

Fix: glob('*.whl') in .github/workflows/release.yml line 83. Confines hashing to wheel files only; SHA256SUMS is never a candidate regardless of creation order.

Regression test: release workflow: wheelhouse SHA256SUMS never hashes itself asserts glob('*.whl') and refutes the bare glob('*') pattern.


Blocker 3 — Wheelhouse extraction path / setup-provider-input-validator discovery

Root cause: install-provider-input-wheelhouse extracted wheels directly to $target (e.g. scripts/provider-input-validator-wheelhouse/), but setup-provider-input-validator expects them one level deeper at $target/$platform (e.g. scripts/provider-input-validator-wheelhouse/linux-x86_64/). The Windows release-assets extraction already used the correct platform subdir path; the POSIX installer did not.

Fix: target="${target}/${platform}" in scripts/install-provider-input-wheelhouse line 42, before stage and backup are derived. All subsequent mkdir/mv/verify operations remain correct; mkdir -p on the stage path automatically creates the new parent directory.

Tests: Structural cross-check verifies all four platform slugs (linux-x86_64, linux-aarch64, macos-arm64, macos-x86_64) appear in both setup-provider-input-validator and install-provider-input-wheelhouse, and that the target=${target}/${platform} assignment is present.


Verification transcript

scripts/test-acq-bats test/bats/05-install.bats:              1..31, 31 pass (was 4 failing — pre-existing git stub gap)
scripts/test-acq-bats test/bats/06-windows-install.bats:      1..23, 23 pass  
scripts/test-acq-bats test/bats/147-provider-input-projection.bats: 1..46, 46 pass
shellcheck --severity=warning (all shell files):              clean
markdownlint:                                                 clean
git diff --check:                                             clean
python3 -c 'import yaml; yaml.safe_load(...)' (all workflows): valid

The four pre-existing failures in 05-install.bats (tests 4, 5, 8, 14) were caused by the git stub's clone case not creating the scripts/install-provider-input-wheelhouse stub that install.sh calls after every release-tag clone; they were not introduced by this PR but masked by --no-msb not being used in those tests. Fixed as part of this commit.

AI-assisted (OpenCode [usai/claude_4_6_sonnet]). Human review required.

@basiliskus basiliskus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes — 063cb7c3a15d49f86295c6971666c85c4284f447

Two prior findings are resolved on GSA-TTS/agentic-coding-quickstart#619: wheel-only checksum enumeration and platform-directory installation.

The release archive destination remains a blocker at .github/workflows/release.yml:84. Executing the exact command with GNU tar 1.35 from a job-like working directory writes ../provider-input-validator-wheelhouse-linux-x86_64.tar.gz into the checkout's parent, not the checkout. GNU tar's -C wheelhouse changes the input-file context; it does not relocate the relative archive output. Upload at line 105 expects the bare filename inside the checkout, where no archive exists.

Please remove ../ from the archive output and replace the structural assertion at test/bats/06-windows-install.bats:279 with an execution test that verifies the file exists at the upload path. That assertion currently enshrines the incorrect path assumption. The earlier archive finding is therefore not retracted.

Runtime creation and real release builds remain unverified here; fixture-based execution verifies the checksum/layout fixes, not packaged dependency compatibility.

Assisted-by: OpenCode [gpt-6.1-sol-latest-guardrails-defaultv2] skill:change-review

Create validator wheelhouse archives in the runner working directory and execute the release command in a regression test to prove upload-artifact can read the exact path.

Co-authored-by: OpenCode Agent <[email protected]>
@mogul

mogul commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

@basiliskus Fixed the remaining release archive blocker in the latest draft head. The wheelhouse tar command now writes the bare archive filename in the runner working directory (tar -czf NAME -C wheelhouse .), exactly matching upload-artifact. The previous structural assertion was replaced by an execution regression that runs the command in a job-like directory and asserts the upload path exists. The archive-path, SHA256SUMS, and platform-extraction focused tests pass 3/3; workflow YAML and diff checks pass. Please re-review.

@basiliskus basiliskus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: comment — 01117db5ee0940bec9f200b630c04db07fd21258

All three prior findings on GSA-TTS/agentic-coding-quickstart#619 are resolved:

  • Archive destination: the exact release tar command now creates the archive at the upload path. The new regression executes tar and checks that output exists, rather than relying solely on a structural assertion.
  • Manifest self-inclusion: the exact manifest-generation command excludes SHA256SUMS and produces a manifest that verifies.
  • Platform-directory installation: actual installer/setup execution confirms matching extraction and discovery paths, using fixture wheels and download transport.

No new must-fix findings. Real release-matrix builds, runtime creation and live install channels remain unverified here. The current-head pre-commit and bash-3.2 CI checks are still running; merge should wait for their results.

Assisted-by: OpenCode [gpt-6.1-sol-latest-guardrails-defaultv2] skill:change-review

@basiliskus

Copy link
Copy Markdown
Contributor

I can't approve because it's a draft, but it's approved

@mogul
mogul marked this pull request as ready for review October 23, 2026 21:56
@basiliskus
basiliskus self-requested a review October 23, 2026 22:00
@basiliskus
basiliskus added this pull request to the merge queue Oct 23, 2026
@mogul

mogul commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Merge-queue sequencing check: current queued head 01117db includes the approved checkout/bootstrap and release-artifact fixes. This PR may merge before GSA-TTS/homebrew-tap#11. Hard guard: do not merge/publish the resulting Quickstart release PR until Homebrew updater #11 is merged to tap main; otherwise the published wheelhouse assets cannot be consumed by the generated formula workflow.

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 23, 2026
@mogul
mogul added this pull request to the merge queue Oct 24, 2026
Merged via the queue into main with commit 5192fa1 Oct 24, 2026
7 checks passed
@mogul
mogul deleted the fix/618-provider-validator-install branch October 24, 2026 20:43
mogul pushed a commit that referenced this pull request Nov 5, 2026
🤖 I have created a release *beep* *boop*
---


##
[4.2.0](v4.1.1...v4.2.0)
(2026-11-05)


### Features

* **acq:** project provider inputs read-only
([#608](#608))
([bd11c77](bd11c77))
* **sbx:** trust host state for built-in kit heal; add MSB forged-marker
and legacy-sandbox coverage
([#637](#637))
([ef24965](ef24965))
* **scripts:** add summarize-bats-timing for bats TAP timings
([#652](#652))
([5bf9398](5bf9398))


### Bug Fixes

* **agents:** move the default OpenCode image to devenv-shell 2.1.0
([#668](#668))
([dbd8a76](dbd8a76))
* **common:** canonicalize macOS /var,/tmp,/etc aliases before
O_NOFOLLOW traversal
([#622](#622))
([1135fcc](1135fcc))
* **install:** package provider validator runtime
([#619](#619))
([5192fa1](5192fa1))
* **msb:** distinguish denied probe writes from writable mounts
([#670](#670))
([049931c](049931c))
* **msb:** strip CR from jq output in acq ports (Windows)
([#663](#663))
([7ad6e56](7ad6e56))
* **release:** use supported macOS wheelhouse runners
([#674](#674))
([9dbc9aa](9dbc9aa))


### Performance Improvements

* **acq:** stop the kit-fetch guardian tick from delaying exit
([#664](#664))
([fe686ea](fe686ea))
* **kits:** reuse pinned fetches within one invocation
([#631](#631))
([b433a61](b433a61))
* **msb:** reduce foreground node kit invocation latency
([#616](#616))
([ec73abd](ec73abd))
* **test:** eliminate redundant bats fetches and waits
([#621](#621))
([dda5fe1](dda5fe1))
* **test:** shorten hostport and cli-kits test tail
([#654](#654))
([a3acc7b](a3acc7b))
* **test:** shorten port fixtures and parallelize shellcheck
([#624](#624))
([8eb3ff8](8eb3ff8))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

acq create fails for every OpenCode sandbox after #608 unless an exact-pinned validator Python exists; Homebrew installs can't provide one

2 participants