Skip to content

Commit 90a1a94

Browse files
authored
feat(kube-ovn): bump kube-ovn to v1.15.26 (#4331)
## What this PR does Bumps kube-ovn from `v1.15.10` to `v1.15.26` (package `cozy-kubeovn` `0.38.0` -> `0.39.0`). The motivation is a cold-start failure mode in the OVN NB/SB raft leader checker. In `v1.15.10`, `checkDBClusterIntegrity` (`pkg/ovn_leader_checker/ovn.go:336`) called `os.Remove` on a healthy local raft DB (`ovn.go:367`) whenever a leader observed fewer members than expected. That count drops below the target transiently during cluster growth, a rolling restart, or recovery after a node address change, so the check could delete a valid DB and leave `ovn-central` unable to elect a leader. `v1.15.26` removes that function entirely, so the DB is no longer deleted. It is a removal of the faulty check, not a grace-period around it. The chart is re-vendored with `make update`, not hand-edited, so the templates match the image: - `charts/kube-ovn` is re-pulled from the `v1.15.26` tag and `patches/mtu.diff` is re-applied; the image build args are regenerated. - This brings the whole v1.15.26 chart: the reworked master-node affinity helpers, the widened kube-ovn CRD schema (`bandwidth.ingress`/`egress` from `integer` to int-or-string, `u2oFeatures` added), and the new `OVN_CENTRAL_MODE` (`cluster`/`single`) support with its `central-svc`, `central-pvc` and `central-hcp-rbac` templates. `OVN_CENTRAL_MODE` defaults to `cluster` (raft on hostPath), so an in-place upgrade keeps the existing topology. - Two vendored patches now included upstream in `v1.15.26` are dropped: `fix-keepvmip-portgroup.diff` (`hasAliveSiblingVMPod`, `pod.go:2437`) and `fix-vmim-scheduling-retry.diff` (scheduling requeue, `kubevirt.go:195`). `fix-vmim-target-pod-retry.diff` is kept: `v1.15.26` still returns nil at `kubevirt.go:215`. The image also picks up `kube-ovn-bfdd-supervisor`, which v1.15.26 adds (`makefiles/build.mk:33`) and upstream's `dist/images/Dockerfile:12` copies into `/kube-ovn`. `pkg/controller/vpc_egress_gateway.go:47` hardcodes that path for the `bfdd` container a `VpcEgressGateway` with `spec.bfd` gets, so a copy that stops at `kube-ovn-controller` ships a container that cannot exec. `hack/check-kubeovn-image-copies.sh` compares what our Dockerfile and upstream's copy into the image, `make update` runs it against the tag it pulls and `hack/check-kubeovn-image-copies.bats` runs it on every CI run, because `make update` otherwise rewrites only the ARG lines and the copy drifts at each bump. The `multus` commit moves the `kube-ovn-cni-plugins-checked-against` marker to `v1.15.26`. `CNI_PLUGINS_VERSION` is `v1.9.1` in kube-ovn's base image at both releases and in multus, so portmap and macvlan stay in lockstep; the marker is the only change. The image ref stays digest-pinned (`repository` + `tag@sha256`), which the promote-retag release contract requires for a cozystack-built image. The digest is carried over from `v1.15.10` as a placeholder and is refreshed to the real `v1.15.26` image by the release-candidate digest-vendoring step; no PR or main job rebuilds it. Verification: `helm unittest .` passes (22 tests), `helm template .` renders clean, the `cni-plugins-staging-contract` and `promote-retag` unit contracts are green, and the image-build `TestVMIMigrationSchedulingRetry` gate passes 6/6 on the patched v1.15.26 tree. > [!NOTE] > `start-db.sh` was substantially reworked upstream between `v1.15.10` and `v1.15.26`. The privileged fork E2E run installs into a sandbox and passes, but the in-tree E2E job is skipped on a fork PR, and a fresh install is not an upgrade. An in-place upgrade on a multi-node control plane (image rebuild plus OVN raft re-convergence) is still untested before merge. ### Screenshots N/A: no UI changes. ### Downstream repositories Walked the trigger map against the diff. The change is confined to `packages/system/kubeovn` (re-vendored chart, image build args, version pins) and a one-line marker in `packages/system/multus`. It does not add, rename or remove a package, does not touch platform values, variants, components, release assets, `ApplicationDefinition` semantics, the Talos version, or any hand-typed CRD schema in a downstream provider. No downstream trigger matches. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(kube-ovn): bump kube-ovn to v1.15.26, which removes the OVN leader-checker integrity check that could delete a healthy raft DB and prevent ovn-central from electing a leader after a node address change or rolling restart ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Added hosted control-plane support for OVN Central with StatefulSet deployment, persistent storage, service exposure, and dedicated RBAC. - Added split-cluster and data-plane-only configuration with external OVN endpoints. - Added configurable TLS, leader election, service, and storage settings. - Enhanced CRDs with overlay-only routing and flexible bandwidth quantities. - **Bug Fixes** - Improved VM migration retries during scheduling and preserved connectivity during live migration. - Improved upgrade handling and service configuration validation. - **Updates** - Upgraded Kube-OVN components and images to version 1.15.26. - Documented required OVN IPsec directory preparation for read-only filesystems. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 0f21d06 + 47a611c commit 90a1a94

43 files changed

Lines changed: 1692 additions & 207 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
#!/usr/bin/env bats
2+
# -----------------------------------------------------------------------------
3+
# Unit tests for hack/check-kubeovn-image-copies.sh.
4+
#
5+
# packages/system/kubeovn/images/kubeovn/Dockerfile is a hand-kept copy of
6+
# upstream's dist/images/Dockerfile, and `make update` rewrites only ARG VERSION
7+
# and ARG TAG in it. When v1.15.26 added kube-ovn-bfdd-supervisor to the image,
8+
# our copy kept the v1.15.10 set and the binary went missing, which nothing in
9+
# the tree noticed. The checker compares what each file copies, and this suite
10+
# runs it over the real Dockerfile against a verbatim copy of upstream's for the
11+
# vendored appVersion -- a synthetic pair alone would only assert the shape we
12+
# already imagined.
13+
#
14+
# cozytest.sh's awk parser recognizes only @test blocks and a bare `}` on its
15+
# own line; there is no bats `run` or `$status`. Assertions are direct shell
16+
# tests, and an expected failure is written as `if CHECK ...; then ... exit 1`.
17+
# -----------------------------------------------------------------------------
18+
19+
REPO_ROOT="$(cd "$(dirname "${BATS_TEST_FILENAME:-$0}")/.." && pwd)"
20+
CHECK="$REPO_ROOT/hack/check-kubeovn-image-copies.sh"
21+
OURS="$REPO_ROOT/packages/system/kubeovn/images/kubeovn/Dockerfile"
22+
CHART="$REPO_ROOT/packages/system/kubeovn/charts/kube-ovn/Chart.yaml"
23+
24+
# Upstream dist/images/Dockerfile at v1.15.26, copied verbatim. Bump it in the
25+
# same change that bumps the chart: `make update` re-runs the same comparison
26+
# against the tag it pulls, so the two cannot disagree for long.
27+
UPSTREAM_VERSION=1.15.26
28+
upstream_dockerfile() {
29+
cat <<'EOF'
30+
# syntax=docker/dockerfile:1
31+
ARG VERSION
32+
ARG BASE_TAG=$VERSION
33+
FROM kubeovn/kube-ovn-base:$BASE_TAG AS setcap
34+
35+
COPY *.sh /kube-ovn/
36+
COPY kubectl-ko /kube-ovn/kubectl-ko
37+
COPY 01-kube-ovn.conflist /kube-ovn/01-kube-ovn.conflist
38+
39+
COPY kube-ovn /kube-ovn/kube-ovn
40+
COPY kube-ovn-cmd /kube-ovn/kube-ovn-cmd
41+
COPY kube-ovn-bfdd-supervisor /kube-ovn/kube-ovn-bfdd-supervisor
42+
COPY kube-ovn-daemon /kube-ovn/kube-ovn-daemon
43+
COPY kube-ovn-controller /kube-ovn/kube-ovn-controller
44+
RUN ln -s /kube-ovn/kube-ovn-cmd /kube-ovn/kube-ovn-monitor && \
45+
ln -s /kube-ovn/kube-ovn-cmd /kube-ovn/kube-ovn-speaker && \
46+
ln -s /kube-ovn/kube-ovn-cmd /kube-ovn/kube-ovn-webhook && \
47+
ln -s /kube-ovn/kube-ovn-cmd /kube-ovn/kube-ovn-leader-checker && \
48+
ln -s /kube-ovn/kube-ovn-cmd /kube-ovn/kube-ovn-ic-controller && \
49+
ln -s /kube-ovn/kube-ovn-controller /kube-ovn/kube-ovn-pinger && \
50+
setcap CAP_NET_BIND_SERVICE+eip /kube-ovn/kube-ovn-cmd && \
51+
setcap CAP_NET_RAW,CAP_NET_BIND_SERVICE+eip /kube-ovn/kube-ovn-controller && \
52+
setcap CAP_NET_ADMIN,CAP_NET_RAW,CAP_NET_BIND_SERVICE,CAP_SYS_ADMIN+eip /kube-ovn/kube-ovn-daemon
53+
54+
FROM kubeovn/kube-ovn-base:$BASE_TAG
55+
56+
COPY --chmod=0644 logrotate/* /etc/logrotate.d/
57+
COPY grace_stop_ovn_controller /usr/share/ovn/scripts/grace_stop_ovn_controller
58+
59+
COPY --from=setcap /kube-ovn /kube-ovn
60+
RUN /kube-ovn/iptables-wrapper-installer.sh --no-sanity-check
61+
62+
WORKDIR /kube-ovn
63+
EOF
64+
}
65+
66+
@test "the fixture tracks the vendored chart version" {
67+
# Chart.yaml quotes the value; the package Makefile leans on the shell to
68+
# strip those, this runs the awk output through a comparison instead.
69+
appversion=$(awk '$1 == "appVersion:" {gsub(/"/, "", $2); print $2}' "$CHART")
70+
if [ "$appversion" != "$UPSTREAM_VERSION" ]; then
71+
echo "FAIL: chart appVersion is $appversion, fixture is v$UPSTREAM_VERSION" >&2
72+
echo "Refresh upstream_dockerfile() from dist/images/Dockerfile at v$appversion." >&2
73+
return 1
74+
fi
75+
}
76+
77+
@test "our Dockerfile copies everything upstream's does" {
78+
upstream_dockerfile | "$CHECK" "$OURS" -
79+
}
80+
81+
@test "a binary upstream added and we did not is reported" {
82+
ours=$(mktemp)
83+
upstream=$(mktemp)
84+
upstream_dockerfile > "$upstream"
85+
grep -v 'kube-ovn-bfdd-supervisor' "$OURS" > "$ours"
86+
87+
if "$CHECK" "$ours" "$upstream" 2>/dev/null; then
88+
echo "FAIL: a missing binary passed the check" >&2
89+
return 1
90+
fi
91+
if ! "$CHECK" "$ours" "$upstream" 2>&1 | grep -q 'kube-ovn-bfdd-supervisor'; then
92+
echo "FAIL: the report does not name the missing binary" >&2
93+
return 1
94+
fi
95+
}
96+
97+
@test "a copy we make and upstream does not is reported" {
98+
ours=$(mktemp)
99+
upstream=$(mktemp)
100+
upstream_dockerfile > "$upstream"
101+
sed -e 's|^COPY --from=builder /source/dist/images/kube-ovn-cmd .*|&\nCOPY --from=builder /source/dist/images/kube-ovn-invented /kube-ovn/kube-ovn-invented|' "$OURS" > "$ours"
102+
103+
if "$CHECK" "$ours" "$upstream" 2>/dev/null; then
104+
echo "FAIL: an extra copy passed the check" >&2
105+
return 1
106+
fi
107+
if ! "$CHECK" "$ours" "$upstream" 2>&1 | grep -q 'kube-ovn-invented'; then
108+
echo "FAIL: the report does not name the extra copy" >&2
109+
return 1
110+
fi
111+
}
112+
113+
@test "COPY lines in the builder stage are ours alone and not compared" {
114+
# `COPY patches /patches` and the regression test have no upstream
115+
# counterpart: they feed our clone-and-patch build, which upstream does not
116+
# have. Reading them would make the check fail on every run.
117+
ours=$(mktemp)
118+
upstream=$(mktemp)
119+
upstream_dockerfile > "$upstream"
120+
sed -e 's|^COPY patches /patches$|COPY patches /patches\nCOPY extra-build-input /some/where|' "$OURS" > "$ours"
121+
122+
"$CHECK" "$ours" "$upstream"
123+
}
124+
125+
@test "a destination that differs is reported even when the source matches" {
126+
ours=$(mktemp)
127+
upstream=$(mktemp)
128+
upstream_dockerfile > "$upstream"
129+
sed -e 's|/kube-ovn/kube-ovn-daemon$|/kube-ovn/kube-ovn-daemon-typo|' "$OURS" > "$ours"
130+
131+
if "$CHECK" "$ours" "$upstream" 2>/dev/null; then
132+
echo "FAIL: a retargeted copy passed the check" >&2
133+
return 1
134+
fi
135+
}
136+
137+
@test "the checker reads the file it is given and not the tree" {
138+
empty=$(mktemp)
139+
upstream=$(mktemp)
140+
upstream_dockerfile > "$upstream"
141+
printf 'FROM scratch\n' > "$empty"
142+
143+
if "$CHECK" "$empty" "$upstream" 2>/dev/null; then
144+
echo "FAIL: a Dockerfile with no copies at all passed the check" >&2
145+
return 1
146+
fi
147+
}

‎hack/check-kubeovn-image-copies.sh‎

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
#!/bin/sh
2+
# Compare what packages/system/kubeovn/images/kubeovn/Dockerfile copies into the
3+
# kube-ovn image against upstream's dist/images/Dockerfile, which it is a
4+
# hand-kept copy of.
5+
#
6+
# `make update` in that package rewrites only ARG VERSION and ARG TAG, so a
7+
# binary upstream starts shipping does not reach our image on its own: v1.15.26
8+
# added kube-ovn-bfdd-supervisor, our copy kept the v1.15.10 set, and a
9+
# VpcEgressGateway with spec.bfd got a container whose command did not exist.
10+
#
11+
# Usage: hack/check-kubeovn-image-copies.sh OURS UPSTREAM
12+
# Either path may be `-` to read that side from stdin.
13+
14+
set -eu
15+
16+
[ $# -eq 2 ] || {
17+
echo "Usage: $0 OURS UPSTREAM (either path may be '-' for stdin)" >&2
18+
exit 2
19+
}
20+
21+
OURS=$1
22+
UPSTREAM=$2
23+
24+
# Normalize the COPY instructions of every stage built FROM kubeovn/kube-ovn-base
25+
# to `<source>... <destination>` lines.
26+
#
27+
# Only those stages are comparable. Our builder stage clones and patches
28+
# kube-ovn, which upstream does not do at all, so its COPY lines have no
29+
# counterpart and reading them would make every run fail.
30+
#
31+
# Flags are dropped so that `--from=builder /source/dist/images/kube-ovn` and a
32+
# bare `kube-ovn` read the same: upstream copies out of a build context that our
33+
# multi-stage build reproduces under /source/dist/images.
34+
copies() {
35+
if [ "$1" = "-" ]; then
36+
cat
37+
else
38+
cat -- "$1"
39+
fi | awk '
40+
function strip(path) {
41+
sub(/^\/source\/dist\/images\//, "", path)
42+
return path
43+
}
44+
# Fold a continued instruction into one logical line before reading it.
45+
{
46+
line = line $0
47+
if (line ~ /\\$/) {
48+
sub(/\\$/, " ", line)
49+
next
50+
}
51+
}
52+
line ~ /^[[:space:]]*FROM[[:space:]]/ {
53+
in_scope = (line ~ /kubeovn\/kube-ovn-base/)
54+
line = ""
55+
next
56+
}
57+
in_scope && line ~ /^[[:space:]]*COPY[[:space:]]/ {
58+
$0 = line
59+
out = ""
60+
for (i = 2; i <= NF; i++) {
61+
if ($i ~ /^--/) continue
62+
out = out (out == "" ? "" : " ") strip($i)
63+
}
64+
if (out != "") print out
65+
}
66+
{ line = "" }
67+
' | LC_ALL=C sort
68+
}
69+
70+
# Lines on stdin that do not appear in the newline-separated set $1.
71+
#
72+
# The set is fed through the same stream rather than `awk -v`: a multi-line -v
73+
# value is a GNU extension, and the awk on a maintainer's macOS rejects it.
74+
lines_not_in() {
75+
{
76+
printf '%s\n' "$1" | sed 's/^/S /'
77+
sed 's/^/C /'
78+
} | awk '
79+
$1 == "S" { sub(/^S /, ""); seen[$0] = 1; next }
80+
{ sub(/^C /, ""); if ($0 != "" && !($0 in seen)) print }
81+
'
82+
}
83+
84+
ours_copies=$(copies "$OURS")
85+
upstream_copies=$(copies "$UPSTREAM")
86+
87+
# A side with no copies at all means the stage markers stopped matching, not
88+
# that the two agree. Reporting that as a pass is how this check would quietly
89+
# stop working the next time the base image is renamed.
90+
if [ -z "$ours_copies" ]; then
91+
echo "ERROR: no COPY instructions found in a kubeovn/kube-ovn-base stage of $OURS" >&2
92+
exit 1
93+
fi
94+
if [ -z "$upstream_copies" ]; then
95+
echo "ERROR: no COPY instructions found in a kubeovn/kube-ovn-base stage of $UPSTREAM" >&2
96+
exit 1
97+
fi
98+
99+
missing=$(printf '%s\n' "$upstream_copies" | lines_not_in "$ours_copies")
100+
extra=$(printf '%s\n' "$ours_copies" | lines_not_in "$upstream_copies")
101+
102+
status=0
103+
104+
if [ -n "$missing" ]; then
105+
echo "ERROR: upstream copies these into the image and we do not:" >&2
106+
printf '%s\n' "$missing" | sed 's/^/ COPY /' >&2
107+
status=1
108+
fi
109+
110+
if [ -n "$extra" ]; then
111+
echo "ERROR: we copy these into the image and upstream does not:" >&2
112+
printf '%s\n' "$extra" | sed 's/^/ COPY /' >&2
113+
status=1
114+
fi
115+
116+
if [ "$status" -ne 0 ]; then
117+
echo >&2
118+
echo "packages/system/kubeovn/images/kubeovn/Dockerfile is a copy of upstream's" >&2
119+
echo "dist/images/Dockerfile. Bring the two back in step, keeping our" >&2
120+
echo "--from=builder /source/dist/images/ prefix." >&2
121+
fi
122+
123+
exit "$status"

‎packages/system/kubeovn/Chart.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
apiVersion: v2
22
name: cozy-kubeovn
3-
version: 0.38.0
3+
version: 0.39.0

‎packages/system/kubeovn/Makefile‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,17 @@ update:
2020
version=$$(awk '$$1 == "appVersion:" {print $$2}' charts/kube-ovn/Chart.yaml) && \
2121
$(SED_INPLACE) -e "s/ARG VERSION=.*/ARG VERSION=$${version}/" images/kubeovn/Dockerfile && \
2222
$(SED_INPLACE) -e "s/ARG TAG=.*/ARG TAG=v$${version}/" images/kubeovn/Dockerfile
23+
# images/kubeovn/Dockerfile is a copy of upstream's dist/images/Dockerfile
24+
# and only its ARG lines are rewritten above, so a binary upstream starts
25+
# shipping does not reach our image on its own. v1.15.26 added
26+
# kube-ovn-bfdd-supervisor that way and nothing noticed until the container
27+
# it backs could not exec. hack/check-kubeovn-image-copies.bats makes the
28+
# same comparison on every CI run against a fixture of upstream's file; the
29+
# test named "the fixture tracks the vendored chart version" is the one that
30+
# asks for that fixture to be refreshed, and it fires after a bump lands.
31+
version=$$(awk '$$1 == "appVersion:" {gsub(/"/, "", $$2); print $$2}' charts/kube-ovn/Chart.yaml) && \
32+
curl -sSfL https://raw.githubusercontent.com/kubeovn/kube-ovn/v$${version}/dist/images/Dockerfile | \
33+
../../../hack/check-kubeovn-image-copies.sh images/kubeovn/Dockerfile -
2334

2435
image:
2536
docker buildx build images/kubeovn \

‎packages/system/kubeovn/charts/kube-ovn/Chart.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,12 +15,12 @@ type: application
1515
# This is the chart version. This version number should be incremented each time you make changes
1616
# to the chart and its templates, including the app version.
1717
# Versions are expected to follow Semantic Versioning (https://semver.org/)
18-
version: v1.15.10
18+
version: v1.15.26
1919

2020
# This is the version number of the application being deployed. This version number should be
2121
# incremented each time you make changes to the application. Versions are not expected to
2222
# follow Semantic Versioning. They should reflect the version the application is using.
2323
# It is recommended to use it with quotes.
24-
appVersion: "1.15.10"
24+
appVersion: "1.15.26"
2525

2626
kubeVersion: ">= 1.29.0-0"

‎packages/system/kubeovn/charts/kube-ovn/README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,5 +50,9 @@ and use the following options to install this Helm-chart:
5050
--set cni_conf.MOUNT_LOCAL_BIN_DIR=false
5151
--set OPENVSWITCH_DIR=/var/lib/openvswitch
5252
--set OVN_DIR=/var/lib/ovn
53+
--set OVN_IPSEC_KEY_DIR=/var/lib/ovs_ipsec_keys
5354
--set DISABLE_MODULES_MANAGEMENT=true
5455
```
56+
57+
`OVN_IPSEC_KEY_DIR` only becomes a host mount when `func.ENABLE_OVN_IPSEC` is enabled, but setting it up
58+
front keeps IPSEC from failing later with `mkdir /etc/origin: read-only file system`.

0 commit comments

Comments
 (0)