Skip to content

Commit 4d66150

Browse files
committed
chore: merge main
Signed-off-by: Eric Curtin <[email protected]>
2 parents 8e5c5f3 + 8d0797a commit 4d66150

480 files changed

Lines changed: 192701 additions & 36212 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.agents/skills/build-from-issue/SKILL.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,3 +34,5 @@ Every PR from this issue-backed workflow must have its own existing issue and us
3434
Do not apply acceptance or roadmap decisions on behalf of a maintainer. Do not introduce `agent:*` workflow labels.
3535

3636
For first-party security-sensitive crypto, use `openshell-crypto` and extend its facade/backend contracts when needed. Follow the [crypto architecture](../../../crates/openshell-crypto/CRYPTO.md) for compatibility, failure handling, and explicit coverage exceptions.
37+
38+
When changing crypto dependencies, run `mise run rust:deny:policy`, including its dependency-boundary regression fixtures. Review any `deny.toml` wrapper change against the documented exception scope; an existing allowance does not authorize new security-sensitive operations outside the facade.

‎.agents/skills/fix-security-issue/SKILL.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,3 +18,5 @@ Use this skill after an authorized `review-security-issue` review identifies an
1818
Begin any fix comments with `> **🔧 security-fix-agent**`. Do not change human disposition or introduce `agent:*` workflow labels.
1919

2020
For first-party security-sensitive crypto, use `openshell-crypto` and extend its facade/backend contracts when needed. Follow the [crypto architecture](../../../crates/openshell-crypto/CRYPTO.md) for compatibility, failure handling, and explicit coverage exceptions.
21+
22+
When changing crypto dependencies, run `mise run rust:deny:policy`, including its dependency-boundary regression fixtures. Review any `deny.toml` wrapper change against the documented exception scope; an existing allowance does not authorize new security-sensitive operations outside the facade.

‎.agents/skills/helm-dev-environment/SKILL.md‎

Lines changed: 38 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -269,7 +269,10 @@ trace-context annotation. The same command exposes OTLP/gRPC on
269269
`http://127.0.0.1:4317` and, when deployed, the Kubernetes gateway on
270270
`http://127.0.0.1:8090`. The local `gateway:docker`, `gateway:podman`, and
271271
`gateway:vm` tasks detect the collector listener at startup and enable trace
272-
export only while it is reachable.
272+
export, OCSF log export, and the agent trace relay only while it is reachable.
273+
Spans an agent exports from inside a sandbox arrive through the supervisor's
274+
relay with `openshell.telemetry.source=agent` and share `openshell.sandbox.id`
275+
with that sandbox's supervisor spans.
273276

274277
The Skaffold profile for HA reverse-proxy development is available from
275278
`deploy/helm/openshell/`:
@@ -418,6 +421,30 @@ above) instead of relying only on `OPENSHELL_GATEWAY_ENDPOINT`; some tests call
418421
### Kubernetes E2E Notes
419422

420423
Use `mise run e2e:kubernetes` for the standard Helm-backed Kubernetes suite.
424+
Set `OPENSHELL_E2E_KUBE_TEST=kubernetes_image_update` to scope the Rust suite to
425+
combined image, command, environment, and metadata updates with retained sandbox,
426+
SSH, and PVC identities. The next start verifies the new argv and environment. This target is
427+
included in both existing Agent Sandbox API-version lanes; unlike the Python
428+
Docker suite, it exercises the driver that supports image updates.
429+
430+
Image updates default to disabled outside test fixtures. The e2e wrapper opts
431+
in only for a fresh gateway fixture that it owns; set
432+
`OPENSHELL_E2E_KUBE_IMAGE_UPDATES=0` to prevent that opt-in. Do not propagate
433+
activation to an existing shared gateway or enable it during a mixed-version
434+
rollout. For manual testing, follow the retirement and activation checklist in
435+
`docs/kubernetes/high-availability.mdx` before setting
436+
`gatewayConfig: {"openshell.gateway": {enable_sandbox_updates: true}}`.
437+
The gateway process equivalents are `--enable-sandbox-updates true` and
438+
`OPENSHELL_ENABLE_SANDBOX_UPDATES=true`. Retire every old process and Job,
439+
settle outstanding driver work, prevent old binaries returning, and do not
440+
treat PostgreSQL connection markers or membership TTL expiry as retirement
441+
proof. SQLite testing supports one gateway process only. Image-update clients
442+
must call the separate `SandboxUpdates` service on the existing gateway
443+
listener, not add a method to the baseline `OpenShell` service or inspect new
444+
public `SandboxStatus` image fields. Resume a pending operation with
445+
`SandboxUpdates.UpdateSandbox`, the current resource version, and no selected
446+
fields, following the documented request-ID semantics.
447+
421448
The kube e2e wrapper creates only one port-forward, to `svc/openshell`; it no
422449
longer forwards the unauthenticated health listener or runs a `/readyz` e2e
423450
target. `/readyz` remains covered by server unit/integration tests.
@@ -608,3 +635,13 @@ for dependencies still declared in `Chart.yaml`.
608635
| `deploy/kube/manifests/envoy-gateway-openshell.yaml` | GatewayClass and BackendTrafficPolicy for Envoy Gateway (`mise run helm:gateway:apply`) |
609636
| `tasks/scripts/helm-k3s-local.sh` | k3d cluster create/delete/start/stop/status |
610637
| `tasks/scripts/keycloak-k8s-setup.sh` | Keycloak deploy, realm import, and development TLS trust anchor |
638+
639+
## Sandbox update regression lane
640+
641+
Run `mise run e2e:kubernetes:updates` for the testkit update suite against both
642+
supported Agent Sandbox APIs with the combined Helm chart and its bundled
643+
workspace resources. It verifies stopped-only updates, retained CR/PVC
644+
identity and contents, command/environment changes, and request replay.
645+
`OPENSHELL_E2E_UPDATE_ORIGINAL_IMAGE` and `OPENSHELL_E2E_UPDATE_TARGET_IMAGE` can
646+
select mirrored Alpine 3.21 and 3.22 manifests. Defaults use the public registry
647+
cache; keep immutable digests when overriding them.

‎.agents/skills/launch-openshell-gator/SKILL.md‎

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,22 @@ If this fails, run the local Codex login flow outside the gator launch. If Codex
8282

8383
The launcher passes the non-secret host account ID as literal `CODEX_ACCOUNT_ID` for Codex's local workspace routing. The harness must not use the opaque `CODEX_AUTH_ACCOUNT_ID` credential placeholder for that comparison. Access tokens remain placeholders, and refresh material remains gateway-only.
8484

85+
The image contains task-independent tooling and a prompt template, not the run's PR assignment or test authorization. The launcher supplies those instructions at runtime through `OPENSHELL_AGENT_TASK` in the canonical process command. The supervisor captures the assembled prompt in memory and supplies it on stdin for every cycle. Task text is non-secret and may appear in runtime command metadata; never put credentials in the task. Task-only changes reuse the same tooling build inputs, although the launcher still builds/tags an image per invocation.
86+
87+
For Kubernetes gateways, select image delivery explicitly: `--k3d-cluster NAME`
88+
builds with Docker and imports into a running local cluster with the matching
89+
current kube context; `--publish-image REGISTRY/REPOSITORY:TAG` builds and pushes,
90+
then launches by resolved digest; `--image REGISTRY/REPOSITORY@sha256:DIGEST`
91+
reuses trusted compatible tooling without local image tools. Do not infer
92+
locality from a loopback endpoint. The k3d selection asserts which cluster
93+
serves the gateway; imported images require `if_not_present` or `never` pull
94+
policy. Registry builds use the builder's platform and existing registry auth;
95+
nodes need their own pull access. `--build-only` builds/delivers without
96+
gateway mutations, provider credential reads, or a sandbox launch. Reused
97+
images must match the local manifest's agent ID and payload version and support
98+
the runtime task contract; legacy baked-task images fail startup. See
99+
`scripts/agents/gator/README.md` for examples and option compatibility.
100+
85101
### Step 5: Verify Gateway Is Registered And Alive
86102

87103
Use the target gateway from the operator request or current session context. Do not assume a gateway name. If the operator did not specify one, list registered gateways and ask before launching when the correct target is ambiguous.
@@ -163,7 +179,7 @@ sandbox_name="gtr-pr-${pr_number}-watch"
163179
"Review and monitor PR #${pr_number} through the gator-gate workflow. Scope this invocation only to PR #${pr_number}."
164180
```
165181

166-
The launcher queries the gateway's selected compute driver, builds the gator image in the matching Docker or Podman image store, stages the immutable payload, imports provider profiles, configures provider credentials and refresh, and starts the agent supervisor as the sandbox's canonical main process. The detached main process survives loss of the host CLI connection and reconnects to a restarted gateway. Unless `--keep` is set, the sandbox is marked ephemeral so the gateway deletes it after the canonical main process exits and its terminal result is finalized. `CONTAINER_ENGINE`, when set, must match the gateway driver.
182+
The launcher queries the gateway's selected compute driver and prepares the image through the selected build/import/publish or prebuilt path before importing provider profiles, configuring provider credentials and refresh, and starting the agent supervisor as the sandbox's canonical main process. The detached main process survives loss of the host CLI connection and reconnects to a restarted gateway. Unless `--keep` is set, the sandbox is marked ephemeral so the gateway deletes it after the canonical main process exits and its terminal result is finalized. `CONTAINER_ENGINE`, when set, must match Docker/Podman gateways or Docker for k3d imports; registry builds for Kubernetes may use either local engine.
167183

168184
The launcher streams image-build and provisioning output until the detached workload is ready, then exits. Use `openshell logs <sandbox-name>` or the TUI for runtime output.
169185

‎.agents/skills/release-captain/SKILL.md‎

Lines changed: 1 addition & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -42,21 +42,6 @@ tag, or start a second `Pre-release` run for a tag; it has no concurrency guard.
4242
picks the highest `-pre.N` on `main` whose whole `Pre-release` run passed,
4343
creates the stable tag on that commit, and publishes without requalifying.
4444

45-
### Patch-train protobuf exception
46-
47-
All `v0.1.6-pre.N` candidates have a release-captain-approved protobuf compatibility exception
48-
in `release-qualification.yml`. The check still compiles both protobuf snapshots
49-
and runs Buf; only detected breaking changes are allowed. Compilation and tooling
50-
errors still fail. Other versions retain normal version-based enforcement.
51-
The run summary and qualification JSON record the exception and reason. Review
52-
all protobuf findings for each candidate and include migration guidance in the
53-
release notes; the exception does not accept other compatibility-review findings.
54-
Squash-merge PR #4976 and, after verifying v0.1.6 publication, prepare a cleanup PR
55-
reverting its entire squash-merge commit. Follow "Remove the v0.1.6 exception" in
56-
`CI.md`; remove the checker support and metadata along with the workflow exception.
57-
A new prerelease number
58-
within this patch train retains the exception; other patch or minor trains do not.
59-
6045
## Setup
6146

6247
```shell
@@ -234,7 +219,7 @@ git worktree remove --force "$wt"
234219
| Transient infrastructure failure in a pre-release, including a registry push | After approval, `gh run rerun <run-id> --failed`. `Release` won't select the run until all of it passes. |
235220
| Pre-release run ends in `startup_failure` | The workflow file is invalid at that commit. Fix it in a new `-pre.N` (Monday step 3). |
236221
| Code defect or `Cargo Deny` advisory | Rerunning won't help. Fix it in a new `-pre.N` (Monday step 3). |
237-
| `Release` fails after selection | After approval, rerun only the failed jobs that haven't published. |
222+
| `Release` fails after selection | After approval, rerun only the failed jobs that haven't published. If the run's artifacts expired or its jobs were skipped, dispatch `Release` again with `-f tag=vX.Y.Z`; it publishes only what is missing. |
238223

239224
## Related
240225

‎.agents/skills/release-openshell/SKILL.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,9 @@ passed but publication failed, rerun the failed jobs before promoting the candid
3232
Release creates `vX.Y.Z` at that exact commit, then rebuilds and publishes all
3333
stable artifacts without rerunning qualification. Embedded versions change, so
3434
stable artifacts are not byte-identical to the qualified prerelease. An existing
35-
stable tag may be retried only if it already points to the selected commit.
35+
stable tag may be retried only if it already points to the selected commit. A retry
36+
publishes only missing artifacts; published images, charts, release assets, npm,
37+
and PyPI versions are kept, while the snap is uploaded again.
3638
Stable images are saved as OCI archives during the build and pushed under the
3739
stable version tags during publication. Their SBOMs are verified before updating
3840
`latest`; the qualified prerelease's commit-SHA image tags remain unchanged.

‎.agents/skills/review-github-pr/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -153,7 +153,7 @@ Read through the full diff (and the PR description if available). Produce a summ
153153
suggested change.
154154
- Do not fabricate concerns or claim a behavioral regression without evidence
155155
for both the prior and proposed behavior.
156-
- **Cryptographic boundaries**: For changed first-party security-sensitive crypto, verify use of `openshell-crypto`, backend dispatch, failure propagation without fallback, and protocol/storage compatibility. Missing operations belong in the facade/backend contracts. Check dependency-owned and non-security exceptions against the repository's [crypto architecture](../../../crates/openshell-crypto/CRYPTO.md); protocol/parsing types alone are not bypasses. Report undocumented bypasses or scope expansion under **Potential Concerns**.
156+
- **Cryptographic boundaries**: For changed first-party security-sensitive crypto, verify use of `openshell-crypto`, backend dispatch, failure propagation without fallback, and protocol/storage compatibility. Missing operations belong in the facade/backend contracts. Check dependency-owned and other explicit exceptions against the repository's [crypto architecture](../../../crates/openshell-crypto/CRYPTO.md); protocol/parsing types alone are not bypasses. Check `mise run rust:deny:policy` results and review changes to `deny.toml` wrappers; package-level allowances do not authorize new bypasses within those packages. Report undocumented bypasses or scope expansion under **Potential Concerns**.
157157
- **Agent infrastructure**: When the PR changes behavior, commands, or development workflows, use the `sync-agent-infra` maintenance map to check that related skills were updated. When it adds, removes, or renames skills or crates; changes workflow relationships or skill coverage; modifies issue or PR templates; or changes agent cross-references, apply the full consistency checklist. Report missing companion updates or drift under **Potential Concerns**.
158158

159159
## Step 5: Output

‎.agents/skills/test-release-canary/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,7 @@ Note `install.sh` is pulled from `raw.githubusercontent.com/NVIDIA/OpenShell/${h
9595

9696
## Testing artifacts from a specific SHA
9797

98-
`Release Dev` publishes two chart versions for every dev build (see `.github/actions/release-helm-oci/action.yml:89-102`):
98+
`Release Dev` publishes two chart versions for every dev build (see the push steps in `.github/actions/release-helm-oci/action.yml`):
9999

100100
- `oci://ghcr.io/nvidia/openshell/helm-chart:0.0.0-dev` — floating, overwritten on every main push.
101101
- `oci://ghcr.io/nvidia/openshell/helm-chart:0.0.0-dev.<sha>` — immutable, `appVersion` set to the same SHA so it pulls the matching `gateway`, `sandbox`, and `supervisor` images.

‎.agents/skills/watch-github-actions/SKILL.md‎

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,9 @@ gh run list --json databaseId,status,headBranch,url --jq '.[] | {id: .databaseId
125125

126126
## View Job Logs
127127

128-
For tmachine Kubernetes conformance failures, inspect the `tmachine-diagnostics-*` artifact. The K3s report includes Agent Sandbox controller state, leader leases, events, and current and previous controller logs. The conformance playbook checks reconciliation after guest boot with a temporary scheduling-gated Sandbox probe before running tests. Distinguish a failed controller probe from a failed conformance scenario; gateway connectivity and controller Deployment availability do not prove reconciliation works.
128+
For tmachine Kubernetes conformance failures, inspect the `e2e-results-conformance-sandbox-ubuntu-k3s-*` artifact. Its `diagnostics/` holds the K3s report with Agent Sandbox controller state, leader leases, events, and current and previous controller logs. The k3s installers check reconciliation after guest boot with a temporary scheduling-gated Sandbox probe, so a failed probe fails provisioning before any test runs. Distinguish a failed controller probe from a failed conformance scenario; gateway connectivity and controller Deployment availability do not prove reconciliation works.
129+
130+
For the `e2e-suites` lane (`.github/workflows/e2e-suites.yml`), the last lines of the job log give the runner's result: PASS, FAIL (exit 1) after the end of the nextest log (which names the failed tests), or ERROR (exit 2) with the step that broke. FAIL and ERROR also appear as annotations on the run's summary page. Then inspect the `e2e-results-<family>-<package>-<environment>-<installer>` artifact, which holds `junit.xml`, `nextest.log` and, after a failure, the guest's `diagnostics/`.
129131

130132
`Pre-release` temporarily omits Codex Security from its `Security Scan` job.
131133
`Release` uses the passing prerelease qualification result. Other scanners and
@@ -162,15 +164,20 @@ not substitute the current `main` tip or the event's older PR base SHA. Merge
162164
groups and manual runs use their explicit baseline. Findings are reported by
163165
`Reject new high or critical findings`; distinguish those from scanner failures.
164166

167+
For `Cargo Deny`, the metadata-only commands clear Rust compiler wrappers after
168+
entering mise; they do not require sccache. For OpenSSL backend and dynamic-linkage
169+
checks, use `nix develop .#crypto` from outside the default shell. The release
170+
toolchains in the default shell expose an unprefixed system AWS-LC that conflicts
171+
with OpenSSL. The crypto shell uses native OpenSSL and Cargo's symbol-prefixed
172+
AWS-LC build for dual-backend tests.
173+
165174
For `Protobuf Compatibility`, check the logged train and comparison baseline.
166175
Branch Checks compares the prospective merge tree with its target; release qualification
167176
compares the tagged candidate with the previous stable release. Both use
168177
the shared `check-protobuf-compatibility` action with `nix run .#check-protobuf-compatibility -- <ref>`.
169178
During `0.x`, a minor train permits compatibility findings
170179
as warnings; a patch train or no active train rejects them. Compare the current
171180
train's version with the latest stable release; commit messages are irrelevant.
172-
For the patch-train `v0.1.6-pre.N` exception, inspect the run summary and the
173-
qualification JSON's `exceptions` array; see `release-captain` for disposition.
174181
Compilation, baseline, and tool errors remain fatal. The `protobuf_compatibility` suite participates in
175182
the `release-tag-v1` qualification profile. Pre-release calls the reusable
176183
`release-qualification.yml` workflow after building, in parallel with publication. Release
@@ -190,7 +197,7 @@ qualification; stable publication is blocked, while published pre-release
190197
artifacts remain available. The Pre-release failure notification includes the
191198
run link. Both callers share `release-build.yml` and `release-publish.yml`.
192199
Qualification evidence remains in Actions; publishing a copy to OCI storage is best effort.
193-
Snap Store uploads happen only for stable releases after qualification passes.
200+
Pre-release uploads snaps to `latest/beta`; stable releases upload to `latest/stable`.
194201

195202
For a release notes preview, dispatch `release.yml` with `notes-only=true` and
196203
`tag=<stable-tag>`. The generator resolves the previous reachable stable tag;
@@ -207,8 +214,8 @@ and qualification, and keeps the notes only in the `release-notes` artifact.
207214
This job is advisory: a notes failure emits a warning without failing the
208215
prerelease run or blocking promotion. Stable publication reuses the promoted
209216
prerelease's `release-notes` artifact, retagged, and generates notes only when
210-
that preview is unavailable. The `Select Qualified Prerelease` summary says
211-
which happened.
217+
that preview is unavailable. The Release workflow's `Release Notes` job summary
218+
says which happened, and Publish waits for that job.
212219

213220
For `Codex Compatibility Review`, inspect its job summary and the
214221
`compatibility-review-run-<run-id>-attempt-<attempt>` artifact. Confirm the

‎.dockerignore‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
1-
# Git
1+
# Version control
22
.git
33
.gitignore
4+
.jj
45

56
# Rust build artifacts (use BuildKit cache mounts instead)
6-
target
7+
**/target
78

89
# Python
910
__pycache__
@@ -37,3 +38,6 @@ kubeconfig
3738
*.log
3839
*.tmp
3940
.DS_Store
41+
42+
# Node
43+
**/node_modules

0 commit comments

Comments
 (0)