Skip to content

Commit 795f69b

Browse files
committed
Merge upstream/main into fix2639
Resolve conflicts from the Rust SDK transport refactor (#4817) and the server-macros crate removal (#4818): - grpc_client keeps the SDK's channel construction (TLS discovery, edge tunnel, insecure mode) and re-attaches the CLI's GatewayInterceptor composing auth headers with W3C trace-context forwarding, using the SDK's re-exported raw gRPC client. The CLI never wires an OIDC refresher, so composing the interceptor client-side loses nothing from OpenShellClient::connect. - Drop the stale tls_server_name helper and tests; upstream moved server-name normalization into the SDK transport. - Cargo.lock: remove the deleted openshell-server-macros entry and regenerate for consistency. Signed-off-by: Chris Moore <[email protected]>
2 parents b2633d9 + 39d95b4 commit 795f69b

189 files changed

Lines changed: 7233 additions & 2249 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.agents/skills/helm-dev-environment/SKILL.md‎

Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,27 @@ Override with env vars before running `helm:k3s:create`:
5757
`mcr.microsoft.com/dotnet/aspire-dashboard:latest`)
5858
- `HELM_K3S_COLLECTOR_HEALTH_TIMEOUT` (default: `120` seconds)
5959

60+
### Shared supervisor plumbing
61+
62+
Set `gatewayConfig.openshell.drivers.kubernetes.supervisor_topology`
63+
using a values file (the table key is literally `openshell.drivers.kubernetes`):
64+
65+
```yaml
66+
gatewayConfig:
67+
openshell.drivers.kubernetes:
68+
supervisor_topology: gateway
69+
```
70+
71+
The gateway chart creates a separate, single-replica shared-host placeholder Deployment
72+
in the release namespace, regardless of shared/managed/operator workspace mode.
73+
Use a supervisor image built from this branch: `--mode=shared` is new.
74+
The chart sets `--gateway-endpoint` from the gateway's gRPC endpoint and
75+
`--address` to `0.0.0.0:5501`. Check its rollout and TCP health probes on this
76+
listener (container port `supervisor`, 5501). The outbound gateway connection and
77+
session RPCs are not implemented yet; sandboxes
78+
still use dedicated supervisors. `workspace` is reserved and rejected. Omit the
79+
field to disable it.
80+
6081
### 2. Deploy OpenShell
6182

6283
**Iterative dev** (rebuilds on file changes, recommended during active development):
@@ -81,7 +102,16 @@ The Skaffold flow builds distinct `gateway`, `sandbox`, and `supervisor` images
81102
and deploys the OpenShell Helm chart. The Kubernetes driver creates a
82103
capability-free workload Pod and a directly managed capability-free supervisor
83104
Pod. One namespace-wide NetworkPolicy denies direct egress from every OpenShell
84-
workload Pod. The
105+
workload Pod. New sandbox generations use `runtime-initiated` control: the sandbox
106+
runtime dials its supervisor through `os-supervisor-<id>`. The legacy
107+
`os-boundary-<id>` Service keeps selecting the workload for old gateways during
108+
rolling upgrades. New gateways create both Services and bind the control-mode
109+
annotation to its runtime generation. The driver adds a per-sandbox
110+
`os-control-<id>` NetworkPolicy for that path only. Existing running generations
111+
keep their connection until replaced; stop/start and gateway recovery that
112+
recreates both Pods upgrade legacy generations automatically. Use compatible
113+
gateway, supervisor, and sandbox runtime images together; there is no legacy
114+
rollout flag. The
85115
`pkiInitJob` hook (a pre-install Job that runs `openshell-gateway generate-certs`)
86116
generates gateway and CLI TLS secrets on first install. Supervisor Pods project
87117
only `ca.crt` and authenticate gateway RPCs with sandbox bearer tokens. User
Lines changed: 228 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
---
2+
name: release-captain
3+
description: Run the weekly OpenShell release captain checklist. Monday checks the pre-release train, triages failures and the compatibility review, and drafts the coordination post. Tuesday promotes the qualified pre-release after approval, verifies publication, and starts the next train. Trigger keywords - release captain, weekly release, Tuesday release, cut the release, release checklist, pre-release status, release train, start next train, stable release.
4+
metadata:
5+
internal: true
6+
---
7+
8+
# Release Captain
9+
10+
OpenShell publishes a stable release every Tuesday from a qualified
11+
pre-release ([RFC 0014](../../../rfc/0014-release-stability/README.md)). The
12+
release captain keeps the week's pre-release train green, coordinates the last
13+
merges, cuts the release, and starts the next train.
14+
15+
## How to use this skill
16+
17+
| When | Ask | What happens |
18+
|---|---|---|
19+
| The week before your turn | "I'm release captain next week. Is the train ready?" | Checks that the next version's `-pre.1` exists. If not, ask this week's captain to start the train (Tuesday step 6). Then check each weekday that Auto-Tag's pre-release ran and passed (`gh run list --workflow release-prerelease.yml --limit 3`), and fix failures as they happen (Monday step 3) instead of on Monday. |
20+
| Monday | "Run the release captain Monday checklist." | Monday steps 1–4. |
21+
| A pre-release failed | "`vX.Y.Z-pre.N` failed, help me fix it." | Monday step 3 and Recovery. |
22+
| Tuesday | "Cut the release." | Tuesday steps 1–6, ending with starting the next train. |
23+
24+
## Guardrails
25+
26+
Get explicit approval, with the exact command and target SHA, before pushing a
27+
tag, dispatching, rerunning, or cancelling a workflow, merging or approving a
28+
PR, or posting in Slack or GitHub. Never repoint a stable tag, delete a `-pre.N`
29+
tag, or start a second `Pre-release` run for a tag; it has no concurrency guard.
30+
31+
## How the train works
32+
33+
- A train has one base version, such as `v0.1.4`, fixed by its `-pre.1` tag. A
34+
patch train can't take breaking changes to Stable interfaces.
35+
- `Release Auto-Tag` runs on weekdays at 14:00 Europe/Zurich. If `main` has
36+
commits since the latest `-pre.N`, it tags the next one and dispatches
37+
`Pre-release` (`release-prerelease.yml`, about 45 minutes). It skips when the
38+
train has no `-pre.1` or the base version is already stable.
39+
- `Pre-release` builds, then publishes and qualifies in parallel, and writes a
40+
`release-notes` preview artifact. Pushing a `-pre.N` tag by hand starts it.
41+
- `Release` (`release.yml`) is dispatched by hand with the stable version. It
42+
picks the highest `-pre.N` on `main` whose whole `Pre-release` run passed,
43+
creates the stable tag on that commit, and publishes without requalifying.
44+
45+
## Setup
46+
47+
```shell
48+
export GH_REPO=NVIDIA/OpenShell
49+
git fetch origin main --tags --force
50+
PREV=$(git tag -l 'v*' --sort=-v:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | head -1)
51+
git tag -l 'v*' --sort=-v:refname | grep -E -- '-pre\.[0-9]+$' | head -5
52+
```
53+
54+
Find the `Pre-release` run for a tag. A tag pushed by hand runs on the tag; an
55+
Auto-Tag run is dispatched on `main`:
56+
57+
```shell
58+
TAG=vX.Y.Z-pre.N
59+
gh run list --workflow release-prerelease.yml --branch "$TAG"
60+
gh run list --workflow release-prerelease.yml --event workflow_dispatch --commit "$(git rev-parse "$TAG^{commit}")"
61+
```
62+
63+
Preview the candidate `Release` would promote, read-only:
64+
65+
```shell
66+
VERSION=vX.Y.Z
67+
wt=$(mktemp -d)
68+
git worktree add -q --detach "$wt" origin/main
69+
(cd "$wt" && GITHUB_REPOSITORY=NVIDIA/OpenShell uv run -q --no-project python tasks/scripts/select-release-candidate.py "$VERSION" --branch main | jq .)
70+
git worktree remove --force "$wt"
71+
```
72+
73+
## Monday: pre-flight
74+
75+
1. **Check the candidate.** Run the preview. If the latest `-pre.N` isn't the
76+
candidate, its run failed; read why:
77+
78+
```shell
79+
RUN=<run-id>
80+
gh run view "$RUN" --json jobs --jq '.jobs[] | select(.conclusion == "failure" or .conclusion == "timed_out") | "\(.name)\t\(.url)"'
81+
gh run view "$RUN" --log-failed
82+
```
83+
84+
If the next version has no `-pre.1`, start the train (Tuesday step 6).
85+
86+
2. **Triage the compatibility review** for the candidate's run. It's advisory,
87+
and a green job doesn't mean it found no breaks:
88+
89+
```shell
90+
ATTEMPT=$(gh run view "$RUN" --json attempt --jq .attempt)
91+
dir=$(mktemp -d)
92+
gh run download "$RUN" -n "compatibility-review-run-$RUN-attempt-$ATTEMPT" -D "$dir"
93+
jq '{status, verdict: .review.verdict}' "$dir/report.json"
94+
cat "$dir/report.md"
95+
```
96+
97+
Only `complete` with `no_unaddressed_breaks` is clean. A real break needs a
98+
fix, a revert, a minor train, or a maintainer-approved exception with upgrade
99+
steps in the release notes.
100+
101+
3. **Fix forward.** A failed pre-release doesn't end the train. Find the cause,
102+
and let the failed run finish rather than cancelling it.
103+
104+
A rerun repeats the same commit and workflow, so it only helps with a
105+
transient failure (see Recovery). Anything that needs a code or workflow
106+
fix ships in a new number: that `-pre.N` stays failed, and `Release` skips
107+
it. Merge the fix, then cut the next `-pre.N` from the tip of `main` after
108+
approval:
109+
110+
```shell
111+
gh workflow run release-auto-tag.yml --ref main
112+
```
113+
114+
Tag a specific commit by hand (as in Tuesday step 6) only when the tip of
115+
`main` isn't the commit you want.
116+
117+
4. **Draft the coordination post.** Start a thread in the OpenShell channel of
118+
the CNCF Slack. The release ships from `main` with no release branch, so the
119+
thread is for getting ready PRs reviewed and merged before the cutoff. List
120+
approved PRs:
121+
122+
```shell
123+
gh pr list --base main --state open --search "review:approved" --json number,title,author --jq '.[] | "#\(.number) \(.title) (@\(.author.login))"'
124+
```
125+
126+
The release ships the final pre-release's commit, if it passes. Pick a
127+
cutoff before Tuesday's Auto-Tag run, or a later one after which you tag the
128+
final pre-release by hand. Don't promise that everything merged by the
129+
cutoff ships, or name the next version.
130+
131+
```text
132+
:thread: OpenShell vX.Y.Z release, Tuesday <date>. I'm release captain this week.
133+
134+
Status: vX.Y.Z-pre.N (<commit>) passed qualification. <blockers and owners, or "No blockers.">
135+
136+
Reply here with any ready PRs you'd like in vX.Y.Z. Merge cutoff is <HH:MM UTC> (<local time>) Tuesday. <"Tuesday's automatic pre-release picks up everything merged by then" | "I'll cut the final pre-release after the cutoff">, and vX.Y.Z ships from it if it passes qualification. Anything that misses it goes in the next release.
137+
```
138+
139+
If the pre-release that ships was cut before the posted cutoff, say so in
140+
the thread, with its commit and time.
141+
142+
## Tuesday: cut the release
143+
144+
1. **Select the final candidate.** For a late cutoff, cut the final `-pre.N`
145+
after the cutoff as in Monday step 3. When it finishes, run the preview and
146+
repeat Monday step 2 for its run.
147+
148+
2. **Request approval.** Present the candidate tag, commit, qualification run,
149+
compatibility assessment, commits since `$PREV`, and release notes preview:
150+
151+
```shell
152+
git log --oneline --no-merges "$PREV..<candidate_tag>"
153+
gh run download "$RUN" -n release-notes -D "$dir/notes" && cat "$dir/notes/release-notes.md"
154+
```
155+
156+
The notes come from each PR's "Release Notes" section. To fix them, edit the
157+
PR, then preview with
158+
`gh workflow run release.yml --ref main -f tag="$VERSION" -f notes-only=true`.
159+
160+
3. **Dispatch the release.** After approval:
161+
162+
```shell
163+
gh workflow run release.yml --ref main -f tag="$VERSION"
164+
```
165+
166+
4. **Watch and verify.** Check that `Select Qualified Prerelease` names the
167+
approved candidate, the GitHub Release has assets, no job failed, and the
168+
wheel reached PyPI:
169+
170+
```shell
171+
gh run watch <run-id> --exit-status
172+
gh release view "$VERSION" --json isPrerelease,publishedAt,assets --jq '{isPrerelease, publishedAt, assets: (.assets | length)}'
173+
```
174+
175+
5. **Close the loop.** Draft a short reply to the coordination thread. Point
176+
to the release notes rather than repeating them:
177+
178+
```text
179+
:white_check_mark: :ship: OpenShell vX.Y.Z is out: https://github.com/NVIDIA/OpenShell/releases/tag/vX.Y.Z
180+
181+
It shipped from vX.Y.Z-pre.N (<commit>). I'll be starting the <next version>-pre.* jobs shortly for next week.
182+
183+
Please check the release notes for upgrade notes that operators might care about. I'm also moving everything still open in the X.Y.Z milestone to <next milestone>.
184+
185+
Thanks everyone!
186+
```
187+
188+
After approval, move the milestone's open items to the next milestone.
189+
190+
6. **Start the next train.** This is this week's captain's last duty. Until the
191+
next `-pre.1` exists, Auto-Tag skips. Agree with the maintainers on patch
192+
(the default) or minor
193+
([version selection](../../../rfc/0014-release-stability/release-version-selection.md)).
194+
Seed it on a commit after `$VERSION` with a passing `Release Dev` run:
195+
196+
```shell
197+
for sha in $(git rev-list --max-count=10 "$VERSION..origin/main"); do
198+
state=$(gh run list --workflow release-dev.yml --commit "$sha" --json status,conclusion --jq '.[0] | if .status == "completed" then .conclusion else .status end')
199+
echo "${sha:0:9} ${state:-none} $(git log -1 --format=%s "$sha")"
200+
done
201+
```
202+
203+
After approval, push the tag. The `-c` and `/dev/null` keep `git` from
204+
signing the tag and opening an editor:
205+
206+
```shell
207+
NEXT=vX.Y.Z-pre.1 NEXT_SHA=<commit>
208+
git -c tag.gpgSign=false tag "$NEXT" "$NEXT_SHA" </dev/null
209+
git push origin "refs/tags/$NEXT"
210+
```
211+
212+
You're done when that `Pre-release` run passes, or its failure is handed to
213+
next week's captain.
214+
215+
## Recovery
216+
217+
| Symptom | Action |
218+
|---|---|
219+
| Transient infrastructure failure in a pre-release, including a registry push | After approval, `gh run rerun <run-id> --failed`. `Release` won't select the run until all of it passes. |
220+
| Pre-release run ends in `startup_failure` | The workflow file is invalid at that commit. Fix it in a new `-pre.N` (Monday step 3). |
221+
| Code defect or `Cargo Deny` advisory | Rerunning won't help. Fix it in a new `-pre.N` (Monday step 3). |
222+
| `Release` fails after selection | After approval, rerun only the failed jobs that haven't published. |
223+
224+
## Related
225+
226+
- `release-openshell`: the short procedure behind Tuesday steps 1–4.
227+
- `watch-github-actions`: general `gh run` monitoring and log retrieval.
228+
- `CI.md`: release workflows, qualification profile, and scanner policies.

‎.agents/skills/sync-agent-infra/SKILL.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ Run after adding, removing, or renaming a skill or crate; changing issue or PR c
2727
| Inference providers or native model endpoints | `debug-inference`, `openshell-cli` |
2828
| TUI behavior | `tui-development` |
2929
| Release artifacts or smoke coverage | `test-release-canary` |
30+
| Release cadence, tagging, or qualification workflows | `release-captain` |
3031
| CI diagnostics | `watch-github-actions` |
3132
| Gator supervision | `launch-openshell-gator` |
3233
| SBOM workflow | `sbom` |

‎.agents/skills/watch-github-actions/SKILL.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -196,8 +196,10 @@ GitHub API failures; it requires `LLM_INFERENCE_API_KEY`, `ANTHROPIC_BASE_URL`,
196196
Pre-release also runs `release-notes.yml` after building, alongside publication
197197
and qualification, and keeps the notes only in the `release-notes` artifact.
198198
This job is advisory: a notes failure emits a warning without failing the
199-
prerelease run or blocking promotion. Stable publication requires successful
200-
notes generation.
199+
prerelease run or blocking promotion. Stable publication reuses the promoted
200+
prerelease's `release-notes` artifact, retagged, and generates notes only when
201+
that preview is unavailable. The `Select Qualified Prerelease` summary says
202+
which happened.
201203

202204
For `Codex Compatibility Review`, inspect its job summary and the
203205
`compatibility-review-run-<run-id>-attempt-<attempt>` artifact. Confirm the

‎.github/actions/setup-buildx/action.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ runs:
2323
using: composite
2424
steps:
2525
- name: Set up Docker Buildx (local)
26-
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
26+
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
2727
with:
2828
name: ${{ inputs.name }}
2929
driver: docker-container

‎.github/prompts/compatibility-review.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,17 @@ state/migration compatibility. A surface can be marked reviewed without findings
143143
when inspection establishes that its public contract was not affected; naming
144144
every surface is not itself evidence of coverage.
145145

146+
The `cli` surface includes the `openshell`, `openshell-gateway`, and
147+
`openshell-supervisor` CLIs. Compare flags and arguments accepted by every
148+
changed executable, including helper binaries launched by compute drivers or
149+
other extensions. Trace their callers at both revisions and check launch
150+
environment and standard-stream requirements when a flag changes. A hidden flag
151+
used by an external consumer can be a compatibility contract; do not dismiss it
152+
solely because it is hidden or used by a built-in driver. Report the old
153+
invocation, the new invocation, and the failure an existing external consumer
154+
would see. Do not treat an unused internal flag as public without evidence of an
155+
affected consumer.
156+
146157
## Findings and coverage
147158

148159
For each concrete or uncertain incompatibility, state the surface, old versus

‎.github/workflows/release-publish.yml‎

Lines changed: 39 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,11 @@ on:
1919
required: false
2020
default: false
2121
type: boolean
22+
release-notes-reused:
23+
description: Whether the calling run already uploaded release-notes from the promoted prerelease's preview
24+
required: false
25+
default: false
26+
type: boolean
2227
source-sha:
2328
description: Source commit of the built artifacts in the calling run
2429
required: true
@@ -329,7 +334,7 @@ jobs:
329334

330335
release-notes:
331336
name: Write Release Notes
332-
if: ${{ !inputs.is-prerelease }}
337+
if: ${{ !inputs.is-prerelease && !inputs.release-notes-reused }}
333338
permissions:
334339
contents: read
335340
pull-requests: read
@@ -342,7 +347,7 @@ jobs:
342347
release:
343348
name: Publish Release Assets
344349
needs: [assemble, release-helm, release-notes]
345-
if: ${{ !cancelled() && needs.assemble.result == 'success' && needs.release-helm.result == 'success' && (inputs.is-prerelease || needs.release-notes.result == 'success') }}
350+
if: ${{ !cancelled() && needs.assemble.result == 'success' && needs.release-helm.result == 'success' && (inputs.is-prerelease || inputs.release-notes-reused || needs.release-notes.result == 'success') }}
346351
runs-on: linux-amd64-cpu8
347352
timeout-minutes: 10
348353
permissions:
@@ -576,6 +581,38 @@ jobs:
576581
fi
577582
npm publish --tag "${dist_tag}"
578583
584+
# Go resolves submodule versions from directory-prefixed tags, so the SDK at
585+
# sdk/go needs its own sdk/go/vX.Y.Z tag to show up as a release on pkg.go.dev.
586+
tag-sdk-go:
587+
name: Tag Go SDK
588+
needs: release
589+
if: ${{ !inputs.is-prerelease }}
590+
runs-on: ubuntu-latest
591+
timeout-minutes: 5
592+
permissions:
593+
contents: write
594+
env:
595+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
596+
VERSION: ${{ inputs.semver }}
597+
SOURCE_SHA: ${{ inputs.source-sha }}
598+
steps:
599+
- name: Create Go SDK tag
600+
run: |
601+
set -euo pipefail
602+
tag="sdk/go/v${VERSION}"
603+
# Lightweight tag, so the ref points at the commit directly.
604+
if ! existing=$(gh api "repos/${GITHUB_REPOSITORY}/git/ref/tags/${tag}" --jq .object.sha 2>/dev/null); then
605+
gh api "repos/${GITHUB_REPOSITORY}/git/refs" -f ref="refs/tags/${tag}" -f sha="${SOURCE_SHA}"
606+
elif [[ "${existing}" != "${SOURCE_SHA}" ]]; then
607+
echo "${tag} already exists at ${existing}, expected ${SOURCE_SHA}" >&2
608+
exit 1
609+
fi
610+
611+
# pkg.go.dev picks up versions the Go module proxy has fetched.
612+
- name: Request version from Go module proxy
613+
continue-on-error: true
614+
run: curl --fail --silent --show-error --max-time 60 "https://proxy.golang.org/github.com/!n!v!i!d!i!a/!open!shell/sdk/go/@v/v${VERSION}.info"
615+
579616
trigger-wheel-publish:
580617
name: Trigger Wheel Publish
581618
needs: release

0 commit comments

Comments
 (0)