Skip to content

Commit a32bdd0

Browse files
committed
chore(policy): merge main after CI repairs
Signed-off-by: Johnny Greco <[email protected]>
2 parents 5b41516 + 292559c commit a32bdd0

66 files changed

Lines changed: 3373 additions & 140 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.agents/skills/build-openshell-mxc-windows/SKILL.md‎

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,7 @@ The lane targets a Windows host with Visual Studio Build Tools and rustup.
115115
| Visual C++ ARM64 tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.ARM64 -property installationPath` | Required for native ARM64 check, build, and tests and for x64-to-ARM64 check/build. Tests always require a native runner. |
116116
| Visual C++ ARM64 Spectre-mitigated libraries | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Runtimes.ARM64.Spectre -property installationPath` | Required by `regorus` through `msvc_spectre_libs`; the build fails when the selected MSVC toolset lacks `lib\spectre\arm64`. |
117117
| Visual C++ Clang tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Llvm.Clang -property installationPath` | Provides host-native `libclang.dll` for `bindgen` and `clang-cl.exe` for ARM64 crypto dependencies such as `aws-lc-sys`. On ARM64, the wrapper uses `VC\Tools\Llvm\Arm64\bin`. |
118-
| Visual C++ CMake tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.CMake.Project -property installationPath` | Provides CMake and Ninja for bundled Z3 and other native dependencies. The x64-to-ARM64 path adds Ninja to `PATH`; Z3 uses MSVC's Visual Studio generator. |
118+
| Visual C++ CMake tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.CMake.Project -property installationPath` | Provides CMake and Ninja for native dependencies. The x64-to-ARM64 path adds Ninja to `PATH`; Z3 uses an architecture-specific prebuilt release. |
119119
| Windows SDK | `where.exe rc.exe` from a Developer PowerShell | Install an SDK containing target libraries and ARM64 tools. |
120120
| Rust via rustup | `rustc --version` | Add each target being validated: `x86_64-pc-windows-msvc` and/or `aarch64-pc-windows-msvc`. The wrapper also adds the selected target. |
121121
| mise | `mise --version` | Used as a task runner only. |
@@ -135,6 +135,8 @@ from this skill.
135135
| `CARGO_TARGET_DIR` | `target` under repo root | Override Cargo output location. Use a short absolute path when x64-to-ARM64 builds approach Windows path-length limits. |
136136
| `Z3_LIBRARY_PATH_OVERRIDE` | unset | Directory containing an x64 system `libz3.lib`; not valid for ARM64. |
137137
| `Z3_SYS_Z3_HEADER` | unset | Full `z3.h` path required with a system Z3 library. |
138+
| `Z3_SYS_Z3_VERSION` | `4.16.0` | Pinned official prebuilt Z3 release selected by the wrapper. |
139+
| `READ_ONLY_GITHUB_TOKEN` | unset | Optional token for the Z3 release lookup; GitHub Actions supplies `github.token`. |
138140
| `RUSTC_WRAPPER` | inherited | The wrapper resolves an available command to an absolute path. If it is unavailable, the wrapper warns and continues without compiler caching. |
139141

140142
Legacy fork variables such as `OPENSHELL_UPSTREAM`,
@@ -203,8 +205,7 @@ jobs in the current mirror push run, or push a new mirrored commit. The binaries
203205
The ARM64 check/build steps in this x64-host contract are cross-builds. The
204206
wrapper discovers and adds host-native LLVM and Ninja to `PATH`, requires the
205207
ARM64 compiler and Spectre-mitigated libraries, lets ARM64 crypto crates select
206-
`clang-cl`, and builds bundled Z3 with native MSVC `cl.exe` and the Visual
207-
Studio generator.
208+
`clang-cl`, and downloads the official prebuilt ARM64 Z3 static library.
208209

209210
On ARM64 hosts, validate the native ARM64 check, build, and test path. The
210211
wrapper rejects test targets that do not match the host architecture, so x64
@@ -315,12 +316,13 @@ Useful log files:
315316
| `test-x86_64-pc-windows-msvc-unsupported-*.log` | Focused unsupported-driver contract output. |
316317
| `test-aarch64-pc-windows-msvc-unsupported-*.log` | Focused native ARM64 contract output. |
317318

318-
The first check builds bundled Z3 from source through `z3-sys`. Cargo stores the
319-
native build output in its target tree, so the Windows target cache reuses it.
320-
The resulting release executables do not require `libz3.dll`. The artifact
321-
report computes SHA256 through .NET directly and does not rely on the
322-
`Get-FileHash` module being available inside the mise-launched Windows
323-
PowerShell process.
319+
The first check downloads the pinned official Z3 archive for the target
320+
architecture through `z3-sys`. GitHub Actions authenticates the lookup with its
321+
read-only workflow token; local users can set `READ_ONLY_GITHUB_TOKEN` if an
322+
unauthenticated lookup is rate-limited. Cargo stores the extracted library in
323+
its target tree, so the Windows target cache reuses it. The artifact report
324+
computes SHA256 through .NET directly and does not rely on the `Get-FileHash`
325+
module being available inside the mise-launched Windows PowerShell process.
324326

325327
## Common Fix Patterns
326328

‎.agents/skills/build-openshell-mxc-windows/reference.md‎

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -68,11 +68,11 @@ file.
6868
For ARM64, verify the Visual Studio instance contains the ARM64 MSVC tools,
6969
ARM64 Spectre-mitigated libraries, Clang tools, CMake tools, and a Windows SDK.
7070
Clang supplies host-native `libclang.dll` for `bindgen` and `clang-cl.exe` for
71-
ARM64 crypto dependencies such as `aws-lc-sys`. Native builds use the normal
72-
bundled-Z3 CMake path. An x64-to-ARM64 check/build discovers and adds
73-
host-native Ninja to `PATH`, builds bundled Z3 with native MSVC `cl.exe` and
74-
the Visual Studio generator, and lets the crypto crates select `clang-cl`. Use
75-
a short `CARGO_TARGET_DIR` if Windows path-length limits are reached.
71+
ARM64 crypto dependencies such as `aws-lc-sys`. Native and
72+
x64-to-ARM64 builds use the official prebuilt Z3 4.16.0 static library for the
73+
target architecture. An x64-to-ARM64 check/build discovers and adds host-native
74+
Ninja to `PATH`, while the crypto crates select `clang-cl`. Use a short
75+
`CARGO_TARGET_DIR` if Windows path-length limits are reached.
7676

7777
## Unsupported Driver Rules
7878

@@ -112,16 +112,18 @@ top-level workspace targets for check/test:
112112
--exclude openshell-driver-vault
113113
--exclude openshell-driver-vm
114114
--exclude openshell-sandbox
115-
--exclude openshell-supervisor-network
115+
--exclude openshell-supervisor
116116
--exclude openshell-supervisor-process
117117
--exclude openshell-vfio
118118
```
119119

120120
The gateway keeps platform configuration and unsupported-operation contracts
121-
without depending on the Docker, Kubernetes, Podman, sandbox supervisor,
122-
process supervisor, VM, or VFIO runtime crates. The Kubernetes Secrets and
123-
Vault libraries still compile as gateway dependencies; only their standalone
124-
Unix-socket binaries and package-level tests are excluded as top-level targets.
121+
without depending on the Docker, Kubernetes, Podman, sandbox runtime,
122+
standalone supervisor, supervisor process runtime, VM, or VFIO crates. The MXC
123+
driver does depend on the cross-platform supervisor network library for its host
124+
egress proxy. The Kubernetes Secrets and Vault libraries still compile as
125+
gateway dependencies; only their standalone Unix-socket binaries and
126+
package-level tests are excluded as top-level targets.
125127

126128
## Common Errors
127129

‎.github/workflows/release-auto-tag.yml‎

Lines changed: 37 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,9 @@ name: Release Auto-Tag
55

66
on:
77
workflow_dispatch: {}
8-
# schedule:
9-
# - cron: "0 14 * * 1-5" # 7 AM PDT, weekdays only
8+
schedule:
9+
- cron: "0 14 * * 1-5"
10+
timezone: "Europe/Zurich"
1011

1112
permissions:
1213
contents: write
@@ -19,17 +20,40 @@ concurrency:
1920
jobs:
2021
create-tag:
2122
runs-on: ubuntu-latest
23+
timeout-minutes: 5
2224
steps:
2325
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
2426
with:
27+
ref: main
2528
fetch-depth: 0
2629

27-
- name: Determine next patch version
30+
- name: Determine next pre-release version
2831
id: version
2932
run: |
30-
latest=$(git tag -l 'v*.*.*' --sort=-v:refname | head -1)
31-
if [ -z "$latest" ]; then
32-
echo "::error::No existing v*.*.* tags found"
33+
set -euo pipefail
34+
tags=$(git tag -l)
35+
latest=$(printf '%s\n' "$tags" | \
36+
{ grep -E '^v[0-9]+\.[0-9]+\.[0-9]+(-pre\.[1-9][0-9]*)?$' || true; } | \
37+
sort -V | tail -1)
38+
if [[ ! "$latest" =~ ^(v[0-9]+\.[0-9]+\.[0-9]+)-pre\.([1-9][0-9]*)$ ]]; then
39+
echo "No active pre-release series — skipping tag creation"
40+
echo "skip=true" >> "$GITHUB_OUTPUT"
41+
exit 0
42+
fi
43+
base=${BASH_REMATCH[1]}
44+
number=${BASH_REMATCH[2]}
45+
if ! git show-ref --verify --quiet "refs/tags/${base}-pre.1"; then
46+
echo "No ${base}-pre.1 seed — skipping tag creation"
47+
echo "skip=true" >> "$GITHUB_OUTPUT"
48+
exit 0
49+
fi
50+
if git show-ref --verify --quiet "refs/tags/${base}"; then
51+
echo "$base is already stable — skipping tag creation"
52+
echo "skip=true" >> "$GITHUB_OUTPUT"
53+
exit 0
54+
fi
55+
if ! git merge-base --is-ancestor "$latest" HEAD; then
56+
echo "::error::Latest pre-release $latest is not an ancestor of main"
3357
exit 1
3458
fi
3559
echo "Latest tag: $latest"
@@ -43,10 +67,7 @@ jobs:
4367
exit 0
4468
fi
4569
46-
major=$(echo "$latest" | sed 's/^v//' | cut -d. -f1)
47-
minor=$(echo "$latest" | sed 's/^v//' | cut -d. -f2)
48-
patch=$(echo "$latest" | sed 's/^v//' | cut -d. -f3)
49-
next="v${major}.${minor}.$((patch + 1))"
70+
next="${base}-pre.$((number + 1))"
5071
5172
if git tag -l "$next" | grep -q .; then
5273
echo "::error::Tag $next already exists"
@@ -58,15 +79,18 @@ jobs:
5879
5980
- name: Create and push tag
6081
if: steps.version.outputs.skip != 'true'
82+
env:
83+
NEXT_TAG: ${{ steps.version.outputs.next }}
6184
run: |
62-
git tag ${{ steps.version.outputs.next }}
63-
git push origin ${{ steps.version.outputs.next }}
85+
git tag "$NEXT_TAG"
86+
git push origin "refs/tags/$NEXT_TAG"
6487
6588
- name: Trigger Release Tag workflow
6689
if: steps.version.outputs.skip != 'true'
6790
env:
6891
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
92+
NEXT_TAG: ${{ steps.version.outputs.next }}
6993
run: |
7094
gh workflow run release-tag.yml \
7195
--ref main \
72-
-f tag=${{ steps.version.outputs.next }}
96+
-f tag="$NEXT_TAG"

‎.github/workflows/windows-msvc.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ jobs:
5656
rust_target: aarch64-pc-windows-msvc
5757
runs-on: ${{ matrix.runner }}
5858
env:
59+
READ_ONLY_GITHUB_TOKEN: ${{ github.token }}
5960
SCCACHE_GHA_ENABLED: "true"
6061
SCCACHE_GHA_VERSION: windows-msvc-${{ matrix.arch }}
6162
SCCACHE_CLIENT_SIDE: "1"
@@ -121,6 +122,7 @@ jobs:
121122
rust_target: aarch64-pc-windows-msvc
122123
runs-on: ${{ matrix.runner }}
123124
env:
125+
READ_ONLY_GITHUB_TOKEN: ${{ github.token }}
124126
SCCACHE_GHA_ENABLED: "true"
125127
SCCACHE_GHA_VERSION: windows-msvc-${{ matrix.arch }}
126128
SCCACHE_CLIENT_SIDE: "1"
@@ -178,6 +180,7 @@ jobs:
178180
rust_target: aarch64-pc-windows-msvc
179181
runs-on: ${{ matrix.runner }}
180182
env:
183+
READ_ONLY_GITHUB_TOKEN: ${{ github.token }}
181184
SCCACHE_GHA_ENABLED: "true"
182185
SCCACHE_GHA_VERSION: windows-msvc-${{ matrix.arch }}
183186
SCCACHE_CLIENT_SIDE: "1"

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -186,6 +186,9 @@ temp/
186186
*.tmp
187187
e2e/gpu/images/.build/
188188

189+
# Ansible Galaxy roles
190+
/tests/ansible/.roles/
191+
189192
# Secrets/credentials (should never be committed)
190193
*.pem
191194
*.key

‎CONTRIBUTING.md‎

Lines changed: 17 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -340,14 +340,19 @@ cargo build -p openshell-prover --features bundled-z3
340340

341341
For x86-64 and ARM64 Windows MSVC builds, use one of these Z3 paths:
342342

343-
- Bundled Z3 (the default for `windows:*` tasks): `z3-sys` builds Z3 from
344-
source and links it into the OpenShell binaries. This keeps the release
345-
executables independent of `libz3.dll` when they are copied to another host.
343+
- Prebuilt Z3 (the default for `windows:*` tasks): `z3-sys` downloads the
344+
pinned Z3 4.16.0 GitHub release for the target architecture on the first
345+
build. Cargo reuses the extracted archive from its target directory. Windows
346+
CI authenticates the GitHub API request with `READ_ONLY_GITHUB_TOKEN` and
347+
preserves the archive in the architecture-specific Cargo target cache. For
348+
cold local builds, you may set `READ_ONLY_GITHUB_TOKEN` to avoid anonymous
349+
GitHub API rate limits.
346350
- System Z3: point `Z3_LIBRARY_PATH_OVERRIDE` at the directory containing the
347-
target-compatible MSVC Z3 library and `Z3_SYS_Z3_HEADER` at the full path to
348-
`z3.h`. The `windows:*` tasks use this path automatically when
349-
`Z3_LIBRARY_PATH_OVERRIDE` is set. Deploy the matching `libz3.dll` with any
350-
binaries built against a dynamic system Z3.
351+
target-compatible MSVC Z3 library and `Z3_SYS_Z3_HEADER` at the full path to `z3.h`.
352+
The `windows:*` tasks use this path automatically when `Z3_LIBRARY_PATH_OVERRIDE`
353+
is set.
354+
- Bundled Z3: for direct Cargo builds, pass `--features bundled-z3` so `z3-sys`
355+
builds Z3 from source.
351356

352357
`openshell-prover` itself has no `bindgen`/`libclang` dependency, so building
353358
just this crate does not require `LIBCLANG_PATH`:
@@ -360,17 +365,17 @@ cargo build -p openshell-prover --target x86_64-pc-windows-msvc --features bundl
360365

361366
To build the full set of Windows binaries, including `openshell-gateway.exe`
362367
and `openshell.exe`, use the `windows:build:x64` mise task instead of a
363-
single-crate `cargo build`. It builds Z3 from source (bundled) by default. A full
364-
build also compiles crates that use `bindgen` (e.g. the MXC driver on Windows),
365-
so it requires `libclang.dll`; if LLVM is not on the default search path, set
366-
`LIBCLANG_PATH` to the directory containing `libclang.dll`:
368+
single-crate `cargo build`. It downloads the pinned prebuilt Z3 release by default. A
369+
full build also compiles crates that use `bindgen` (e.g. the MXC driver on
370+
Windows), so it requires `libclang.dll`; if LLVM is not on the default search
371+
path, set `LIBCLANG_PATH` to the directory containing `libclang.dll`:
367372

368373
```powershell
369374
$env:LIBCLANG_PATH='C:\Program Files\Microsoft Visual Studio\2022\<Edition>\VC\Tools\Llvm\x64\bin'
370375
mise run --skip-tools windows:build:x64
371376
```
372377

373-
To use a local x64 Z3 release instead of the bundled build, set
378+
To use a local x64 Z3 release instead of the prebuilt download, set
374379
`Z3_LIBRARY_PATH_OVERRIDE` and `Z3_SYS_Z3_HEADER` before running the task:
375380

376381
```powershell

‎architecture/build.md‎

Lines changed: 31 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -227,6 +227,8 @@ The Nix test guest harness under `nix/test-guest` boots native-architecture clou
227227
through QEMU for package, release, and E2E validation. A prepared cache entry is
228228
captured after the exact ordered Ansible configuration list and before
229229
test-specific packages, copied binaries, forwarded ports, or commands.
230+
On macOS, the test guest and tmachine paths use the same pinned QEMU and OVMF
231+
package set so the hypervisor and firmware remain compatible.
230232

231233
Prepared disks are flattened, sanitized QCOW2 images. The local cache keeps them
232234
read-only and each test receives a fresh writable overlay and cloud-init
@@ -242,6 +244,23 @@ CLI conformance runs after target provisioning and operates only through the
242244
configured OpenShell CLI. The smoke scenario verifies the black-box sandbox
243245
lifecycle by creating, inspecting, executing in, and deleting a sandbox.
244246

247+
The `tests/tmachine` setup and installation caches include a digest of the
248+
entire directory containing `ANSIBLE_CONFIG`, including local roles, task
249+
includes, templates, inventory, and requirements. The digest uses sorted
250+
relative paths, file contents, and executable permissions; source symlinks
251+
are unsupported. Both keys also retain the ordered playbook paths and contents,
252+
their base disk contents, and whether Galaxy is enabled; installation keys
253+
include named binary inputs. The top-level `.roles` directory is excluded:
254+
Galaxy release pins in `requirements.yaml` are treated as immutable, including
255+
any transitive dependency pins. Cache misses with Galaxy enabled reinstall
256+
the required roles and their dependencies before running playbooks.
257+
258+
The `tests/artifacts.nix` helpers build the CLI, conformance CLI, and sandbox
259+
with musl, and the gateway and supervisor with GNU. Image assembly stages
260+
the gateway, sandbox, and supervisor as separate binaries for their respective
261+
Dockerfiles. The Ubuntu Docker and Fedora Podman scenarios import both local
262+
runtime images and configure the gateway to use them.
263+
245264
## Python Wheel Packaging
246265

247266
The generated protobuf/gRPC stubs under `python/openshell/_proto/` are gitignored
@@ -420,10 +439,18 @@ job republishes the analysis job's outcome as the
420439
required statuses, so they do not gate merges.
421440

422441
Codex Security findings are informational during the observation phase, and the
423-
workflow only reports on candidates that already exist. Creating pre-release
424-
tags and gating stable promotion on qualification results are part of
425-
[RFC 0014](../rfc/0014-release-stability/release-qualification.md) and are not
426-
implemented yet.
442+
workflow only reports on candidates that already exist. Gating stable promotion
443+
on qualification results remains proposed in
444+
[RFC 0014](../rfc/0014-release-stability/release-qualification.md).
445+
446+
`release-auto-tag.yml` runs at 14:00 Europe/Zurich on weekdays (including daylight
447+
saving time changes) and supports manual dispatch. Maintainers start weekday
448+
pre-release publishing by tagging the initial `vX.Y.Z-pre.1` release candidate.
449+
The workflow increments the highest release series' pre-release number on `main`
450+
only when that seed exists, its stable tag does not exist, and new commits are
451+
available. It never chooses a minor or patch version or creates the initial seed.
452+
After pushing the tag, it explicitly dispatches `release-tag.yml` to build the
453+
candidate.
427454

428455
See `CI.md` for the contributor workflow, labels, and maintainer merge-queue workflow.
429456

0 commit comments

Comments
 (0)