Skip to content

Commit 08bac03

Browse files
feat(site-router): add routed site-to-site IPsec gateway app (Phase 1)
Adds `site-router`, a catalog app for routed (L3), source-IP-preserving tenant site-to-site connectivity over an IKEv2 IPsec tunnel terminated in a VyOS KubeVirt gateway VM. The chart materializes the gateway: VM + boot DataVolume, tunnel Service type LoadBalancer (UDP 500/4500, native loadBalancerClass), PSK and RBAC-isolated api-key Secrets, first-boot cloud-init, WorkloadMonitor, and two net-new Cilium policies (gateway egressDeny + gateway ingress). `site-router-controller` mediates what the chart cannot express: deny-set validation of the tunnel's remote networks, the kube-ovn return-route annotation, gateway-port `port_security` relaxation gated on the guest source filter, the live VyOS configuration push over the management API, tunnel/BGP observability, and status. There is no new tenant CRD — the `apps.cozystack.io/SiteRouter` app instance is the whole contract. A shared pure `internal/siterouter/denyset` validator backs both a SiteRouter-scoped admission check in `pkg/registry/apps/application` and the reconcile-time check, so a cluster-overlapping `remoteCIDR` is rejected identically at apply time and reconcile time. The admission runs fail-closed on create and on update. NAT/DNAT (a future `site-gateway`) and HA/VRRP are out of scope for Phase 1. The negative-security acceptance suite is authored but parked: its live run needs the published appliance image. Contents: - app chart packages/apps/site-router (10 helm-unittest suites) - site-router-controller: internal/controller/siterouter, cmd/site-router-controller, packages/system/site-router-controller - VyOS core library internal/vyos (client/parse/observation) and routed render internal/vyos/render - internal/siterouter/denyset + the admission check - the appliance build package packages/system/vyos-router-image - platform wiring: PackageSources for the application and the controller, both emitted by the naas bundle behind bundles.iaas.enabled (the gateway needs KubeVirt + CDI, which isp-hosted does not have) - docs: README, security-model, image-lifecycle, followups The e2e suite is committed parked as chainsaw-test.yaml.disabled, the way hack/e2e-chainsaw/backup/ is parked, because its appliance image is not published yet. Parking it as a file rather than as a name filtered out of hack/select-e2e.sh's output is deliberate: the selector decides coverage from the same enumeration, so a name-based filter leaves the graph walk still reaching the suite. site-router-application depends on cozystack.kubevirt-cdi, which would give kubevirt-cdi — and cozystack-basics, which reaches it — exactly one suite, stripped to an empty selection that both e2e lanes read as "skip Chainsaw" before posting the required "E2E Tests" status green (#3392). Parking the file keeps the suite out of the selector's universe and out of select-install.sh's round-trip validation at once, so those packages keep escalating to the full run as they do on main, and un-parking is renaming the file back. The appliance boot reference is committed digest-less (`…/vyos-router-disk:v0.0.0`). A first-party image that has never been built has no digest to commit, and nothing in the PR lane can create one: build-vyos ends at `git diff --binary HEAD > vyos.patch` and uploads it, so the stamp reaches e2e through pr.patch and never reaches git; only tags.yaml's post-merge `Prepare release` commits a digest. site-router-controller ships :v0.0.0 the same way in this PR, and securitygroup-controller sat at :v0.0.0 on main until a release build stamped it. An all-zero placeholder is strictly worse than no digest — it passes `required`, looks like a valid pin and resolves nowhere — so hack/image-refs-no-placeholder.bats keeps rejecting that shape untouched. While the ref is digest-less it is invisible to hack/lib/image-refs.sh, so promote/retag/mirror skip the appliance; tests/dv_test.yaml accepts either form and carries a TODO to re-tighten after the first release build. VYOS_BUILD_REF is pinned to e14a4895 (upstream `rolling` tip, 2026-08-11), whose kernel_version 6.18.44 is the only kernel the rolling mirror serves; the ref this started from asked for 6.18.41, which is gone, and live-build fails hard on that rather than building something stale. VYOS_VERSION tracks the ref's commit date so the label stays a function of the pin. The pin comment previously claimed Renovate kept these from aging silently — both custom managers do match the file, but Renovate only evaluates the default branch and neither the Makefile nor the managers exist there until this merges, so the pins were unwatched by construction; even after merge they raise a PR rather than holding the build up, while the breakage is on the mirror's kernel-retention clock. The comment and docs/image-lifecycle.md now say that. Two shared checks move with the app: cozystack-api's ClusterRole gains a name-scoped `get` on the cozystack ConfigMap that the deny-set admission reads (its pinned rule count goes 16 -> 17), and the site-router-controller chart ships the RBAC and deployment suites the sibling controller packages ship. Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent d000d02 commit 08bac03

107 files changed

Lines changed: 15430 additions & 22 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/renovate.json‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,23 @@
3232
"matchStrings": ["clastix/kubectl:(?<currentValue>v[0-9]+\\.[0-9]+(?:\\.[0-9]+)?)@(?<currentDigest>sha256:[a-f0-9]+)"],
3333
"datasourceTemplate": "docker",
3434
"depNameTemplate": "docker.io/clastix/kubectl"
35+
},
36+
{
37+
"customType": "regex",
38+
"description": "Track the vyos-build container the VyOS router appliance is built in. It is pinned by digest behind the floating `current` tag in a Makefile variable, so neither the dockerfile manager (it is not a FROM line) nor gomod can see it, and the package's `make update` is a deliberate no-op because there is no chart to pull. Without this manager the build-tooling pin — the reproducibility lever for an image that ships as a tenant-facing gateway OS — would silently age.",
39+
"managerFilePatterns": ["/^packages/system/vyos-router-image/Makefile$/"],
40+
"matchStrings": ["VYOS_BUILD_IMAGE\\s*=\\s*vyos/vyos-build:(?<currentValue>\\S+)@(?<currentDigest>sha256:[a-f0-9]+)"],
41+
"datasourceTemplate": "docker",
42+
"depNameTemplate": "docker.io/vyos/vyos-build"
43+
},
44+
{
45+
"customType": "regex",
46+
"description": "Track the vyos-build git ref the VyOS router appliance flavor is built from. It is a bare commit SHA in a Makefile variable with no package manifest behind it, so only a regex manager can see it. Pinned to a commit rather than a branch on purpose — the appliance and the site-router cloud-init contract advance atomically — so this raises a PR when the upstream `current` branch moves and the bump stays a reviewed decision.",
47+
"managerFilePatterns": ["/^packages/system/vyos-router-image/Makefile$/"],
48+
"matchStrings": ["VYOS_BUILD_REF\\s*=\\s*(?<currentDigest>[0-9a-f]{40})"],
49+
"datasourceTemplate": "git-refs",
50+
"depNameTemplate": "https://github.com/vyos/vyos-build",
51+
"currentValueTemplate": "rolling"
3552
}
3653
],
3754
"packageRules": [

‎.github/workflows/pull-requests.yaml‎

Lines changed: 101 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,7 @@ jobs:
7979
matrix: ${{ steps.p.outputs.matrix }}
8080
any: ${{ steps.p.outputs.any }}
8181
touched: ${{ steps.p.outputs.touched }}
82+
vyos: ${{ steps.p.outputs.vyos }}
8283
steps:
8384
# Claim the required context for THIS run before anything can be merged on
8485
# the strength of the last one. Commit statuses are latest-wins per (sha,
@@ -162,6 +163,17 @@ jobs:
162163
else
163164
echo "code=false" >> "$GITHUB_OUTPUT"
164165
fi
166+
# build-vyos runs a ~60-min privileged VyOS live-build whose image is
167+
# not consumed by the general test path (vm-default-images is an opt-in
168+
# bundle package), and a floating apt-repo failure in it would otherwise
169+
# block every unrelated PR. Gate it to the VyOS image pipeline and its
170+
# consumers (mirrors the docs-only `code` gate above: git-diff + grep);
171+
# finalize tolerates it being skipped.
172+
if grep -qE '^(packages/system/vyos-router-image/|packages/apps/site-router/|packages/system/vm-default-images/|\.github/workflows/pull-requests\.yaml$)' /tmp/changed.txt; then
173+
echo "vyos=true" >> "$GITHUB_OUTPUT"
174+
else
175+
echo "vyos=false" >> "$GITHUB_OUTPUT"
176+
fi
165177
# docs/agents/contributing.md carries the downstream trigger map, which
166178
# hack/downstream-trigger-map.bats asserts against: every path the map
167179
# cites still exists, and its repository list still matches the PR
@@ -527,6 +539,93 @@ jobs:
527539
path: _out/oci/*.oci.tar
528540
if-no-files-found: error
529541

542+
# VyOS is its own leg for the same reason Talos is: `make -C
543+
# packages/system/vyos-router-image image` runs a full VyOS live-build
544+
# (debootstrap + apt + squashfs) in a privileged container, far heavier than a
545+
# normal package image build, so it does not belong in the parallel matrix
546+
# (build-matrix.sh excludes it). It builds the qcow2, packages it as a
547+
# digest-pinned KubeVirt containerDisk pushed to the registry, and stamps the
548+
# ref@digest into packages/apps/site-router/images/vyos-router-disk.tag.
549+
# That stamp is captured as a patch fragment so finalize bundles the real digest
550+
# into the installer artifact (the deferred site-router e2e resolves it from
551+
# there). vm-default-images is an opt-in bundle package, so a non-opted-in e2e is
552+
# unaffected either way.
553+
#
554+
# Path-scoped (plan.outputs.vyos): this heavy, privileged build only runs when
555+
# the VyOS image pipeline or its consumers change (vyos-router-image /
556+
# site-router / vm-default-images / this workflow) — otherwise its image is never
557+
# consumed by the run and a floating apt-repo failure would needlessly block
558+
# unrelated PRs. When skipped it must not fail the gate: finalize accepts a
559+
# skipped result (see its `if`).
560+
build-vyos:
561+
name: Build VyOS
562+
needs: ["plan"]
563+
if: |
564+
needs.plan.outputs.code == 'true'
565+
&& needs.plan.outputs.vyos == 'true'
566+
&& !contains(github.event.pull_request.labels.*.name, 'release')
567+
runs-on: oracle-vm-4cpu-16gb-x86-64
568+
timeout-minutes: 60
569+
permissions:
570+
contents: read
571+
packages: write
572+
steps:
573+
- name: Checkout code
574+
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
575+
with:
576+
fetch-depth: 0
577+
fetch-tags: true
578+
- name: Set up Docker config
579+
run: |
580+
if [ -d ~/.docker ]; then
581+
cp -r ~/.docker "${{ runner.temp }}/.docker"
582+
fi
583+
- name: Login to GitHub Container Registry
584+
if: ${{ !github.event.pull_request.head.repo.fork }}
585+
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3
586+
with:
587+
username: ${{ secrets.OCIR_USER }}
588+
password: ${{ secrets.OCIR_TOKEN }}
589+
registry: iad.ocir.io
590+
env:
591+
DOCKER_CONFIG: ${{ runner.temp }}/.docker
592+
# buildkitd-config routes docker.io base-image pulls through mirror.gcr.io
593+
# (hack/buildkitd.toml); see the build job for the full rationale. The
594+
# docker-container builder is used for the containerDisk buildx push; the
595+
# heavy VyOS live-build itself runs as a privileged `docker run` on the
596+
# runner's default docker daemon (it needs --privileged + /dev for the
597+
# loop/kpartx disk operations, exactly like the Talos imager).
598+
- name: Set up Buildx (docker-container driver, docker.io mirror)
599+
id: buildx
600+
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
601+
with:
602+
driver: docker-container
603+
buildkitd-config: hack/buildkitd.toml
604+
env:
605+
DOCKER_CONFIG: ${{ runner.temp }}/.docker
606+
- name: Build VyOS router image (qcow2 + containerDisk)
607+
run: make -C packages/system/vyos-router-image image
608+
env:
609+
DOCKER_CONFIG: ${{ runner.temp }}/.docker
610+
BUILDER: ${{ steps.buildx.outputs.name }}
611+
IMAGE_TAG: pr-${{ github.event.pull_request.number }}-${{ github.event.pull_request.head.sha }}
612+
PUBLISH_VERSIONED: '0'
613+
PUBLISH_FLOATING: '0'
614+
- name: Capture VyOS patch fragment
615+
run: |
616+
mkdir -p _out/fragments
617+
# Captures the stamped apps/site-router/images/vyos-router-disk.tag.
618+
# Artifact only — this never commits or pushes, so the committed .tag
619+
# keeps whatever it holds until a release build stamps it.
620+
git diff --binary HEAD > _out/fragments/vyos.patch
621+
echo "VyOS fragment:"; git --no-pager diff --stat HEAD
622+
- name: Upload VyOS patch fragment
623+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
624+
with:
625+
name: pr-patch-fragment-vyos
626+
path: _out/fragments/vyos.patch
627+
if-no-files-found: error
628+
530629
# Reassemble the unified pr.patch from every fragment, then build the installer
531630
# LAST: `make -C packages/core/installer image` runs `flux push artifact
532631
# --path=packages`, which bundles the WHOLE digest-patched tree into the OCI
@@ -538,14 +637,15 @@ jobs:
538637
# to e2e (which needs finalize), so a red unit test skips the expensive
539638
# install+e2e path instead of wasting ~1h on it. Restores the gating the old
540639
# in-build-job tests gave, without re-coupling the cheap parallel builds.
541-
needs: ["plan", "checks", "build", "build-talos"]
640+
needs: ["plan", "checks", "build", "build-talos", "build-vyos"]
542641
if: |
543642
!cancelled()
544643
&& needs.plan.outputs.code == 'true'
545644
&& !contains(github.event.pull_request.labels.*.name, 'release')
546645
&& needs.checks.result == 'success'
547646
&& (needs.build.result == 'success' || needs.build.result == 'skipped')
548647
&& needs.build-talos.result == 'success'
648+
&& (needs.build-vyos.result == 'success' || needs.build-vyos.result == 'skipped')
549649
runs-on: oracle-vm-4cpu-16gb-x86-64
550650
timeout-minutes: 30
551651
permissions:

‎Makefile‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,10 +33,12 @@ build: build-deps
3333
make -C packages/system/bucket image
3434
make -C packages/system/objectstorage-controller image
3535
make -C packages/system/securitygroup-controller image
36+
make -C packages/system/site-router-controller image
3637
make -C packages/system/grafana-operator image
3738
make -C packages/system/redis-operator image
3839
make -C packages/core/testing image
3940
make -C packages/core/talos image
41+
make -C packages/system/vyos-router-image image
4042
make -C packages/core/platform image
4143
make -C packages/core/installer image
4244
make manifests

‎api/apps/v1alpha1/siterouter/types.go‎

Lines changed: 124 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)