Skip to content

Commit 867f5ba

Browse files
fix(site-router): label the diagnostics disk, and make a failing probe say what it saw
The suite finally got a tunnel up and failed on the first step that pushes a packet through it, assert-inbound-source-preserved. Neither of the two channels that exist to explain such a failure produced anything, and both were broken in the same way: silently. The diagnostics disk never had a label. KubeVirt takes the iso9660 label for a Secret volume from Secret.VolumeLabel, not from the volume name (kubevirt pkg/config/secret.go, getLabel returns v.Secret.VolumeLabel), and the chart set no volumeLabel. So `blkid -L cozydiag` in the appliance seed matched nothing on every run since the disk was introduced, the guest emitter was never installed, and the comment above the volume asserted the opposite as if it were checked. Both A's chart and B's manifest now set it. The seed also says when the disk is absent instead of skipping in silence, which is what let this sit unnoticed: the one channel that could have explained a failing guest was itself missing, and nothing reported that. The probe's own message does not reach the job log. A chainsaw script whose command exits non-zero is reported as "($error == null): Invalid value: false" and nothing else, so the echo written specifically to say which source address the backend observed went nowhere. That address is the entire content of the failure. _probe-lib.sh gains a `record` helper that appends observations to the cozyreport snapshot the catch already collects, so they survive whatever the log is doing, and the source-preservation probe uses it. No cause for the source-preservation failure here. This is the round that makes the next one able to name it. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 06b444e commit 867f5ba

5 files changed

Lines changed: 50 additions & 5 deletions

File tree

‎hack/e2e-chainsaw/site-router/_probe-lib.sh‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,26 @@ transfer_from_b() {
134134
[ "${sent:-0}" -ge "$bytes" ] 2>/dev/null
135135
}
136136

137+
# record <label> <value...>
138+
#
139+
# Writes one observation into the cozyreport snapshot for this test, and echoes
140+
# it as well. The echo is the part that does NOT survive: a chainsaw `script`
141+
# whose command fails reports only the wrapper, "($error == null): Invalid
142+
# value: false", and the message the script printed to say WHAT it saw is
143+
# nowhere in the job log. Measured on the run that first got a tunnel up: the
144+
# source-preservation probe failed and the observed client address, which is the
145+
# entire content of that failure, could not be recovered from the log at all.
146+
#
147+
# The snapshot directory is the same one .chainsaw.yaml's catch collects into,
148+
# so anything written here lands in cozyreport.tgz whatever the log verbosity.
149+
record() {
150+
_rlabel="$1"; shift
151+
_rsnap="${COZY_REPORT_DIR:-/workspace/_out/cozyreport}/snapshots/${TEST_NAME:-site-router}"
152+
mkdir -p "$_rsnap" 2>/dev/null || true
153+
printf '%s: %s\n' "$_rlabel" "$*" >> "$_rsnap/probe-observations.txt" 2>/dev/null || true
154+
echo "$_rlabel: $*"
155+
}
156+
137157
# hubble_dropped <dst> <reason-substring>
138158
#
139159
# NO CURRENT CALLER, deliberately kept. The two cases that used it (a foreign

‎hack/e2e-chainsaw/site-router/chainsaw-test.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -643,6 +643,10 @@ spec:
643643
# curl http://$BACKEND_IP:8080/clientip from B, sourced 192.0.2.1,
644644
# and confirm the returned client IP is in B's site range.
645645
observed=$(probe_from_b_stdout http "$BACKEND_IP" 8080 "/clientip")
646+
# record, not echo: a failing chainsaw script reports only its
647+
# wrapper error, so the observed address has to be written where
648+
# the report collector picks it up or the failure says nothing.
649+
record inbound-source "backend=$BACKEND_IP observed='$observed'"
646650
case "$observed" in
647651
192.0.2.*) echo "source preserved: backend saw $observed" ;;
648652
*) echo "FAIL: backend saw '$observed', expected a 192.0.2.x source (NAT / not routed)"; exit 1 ;;

‎hack/e2e-chainsaw/site-router/remote-site-b.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -215,4 +215,9 @@ spec:
215215
name: remote-site-b-cloudinit
216216
- name: cozydiag
217217
secret:
218+
# volumeLabel, not the volume name: KubeVirt reads the iso9660 label
219+
# off Secret.VolumeLabel (pkg/config/secret.go getLabel), so without
220+
# this the seed's `blkid -L cozydiag` finds nothing and installs no
221+
# diagnostics, silently. Same fix as A's vm.yaml.
218222
secretName: remote-site-b-diag
223+
volumeLabel: cozydiag

‎packages/apps/site-router/templates/vm.yaml‎

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -89,15 +89,23 @@ spec:
8989
name: {{ .Release.Name }}-cloud-init
9090
{{- if .Values._logSerialConsole }}
9191
{{- /*
92-
A plain Secret volume, which KubeVirt renders as an iso9660 disk whose
93-
label is the volume name, so the appliance seed finds it with
94-
`blkid -L cozydiag`. Separate from the NoCloud disk on purpose: the
95-
configuration seed and the diagnostics must not be able to break each
96-
other (see site-router.diagFiles).
92+
A plain Secret volume, which KubeVirt renders as an iso9660 disk. The
93+
appliance seed finds it with `blkid -L cozydiag`, and volumeLabel is what
94+
puts that label on the filesystem: KubeVirt takes it from
95+
Secret.VolumeLabel and NOT from the volume name (kubevirt
96+
pkg/config/secret.go getLabel). An earlier revision of this comment said
97+
the volume name was the label; it is not, the label was empty, and the
98+
seed silently installed no diagnostics on every run — which is how the
99+
one channel that could have explained a failing guest was itself missing.
100+
101+
Separate from the NoCloud disk on purpose: the configuration seed and the
102+
diagnostics must not be able to break each other (see
103+
site-router.diagFiles).
97104
*/}}
98105
- name: cozydiag
99106
secret:
100107
secretName: {{ .Release.Name }}-diag
108+
volumeLabel: cozydiag
101109
{{- end }}
102110
networks:
103111
- name: default

‎packages/system/vyos-router-image/overlay/vyos-appliance-seed.sh‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,14 @@ umount "$MNT" 2>/dev/null || true
5252
# touch the configuration above. The chart attaches it only when the platform or
5353
# the e2e suite asked for the serial console; a tenant cannot.
5454
DIAG=$(blkid -L cozydiag 2>/dev/null) || DIAG=""
55+
if [ -z "$DIAG" ]; then
56+
# Say so. This branch used to be silent, and the label was empty for every
57+
# run because the chart set no volumeLabel on the Secret volume — KubeVirt
58+
# takes the iso9660 label from Secret.VolumeLabel, not from the volume name.
59+
# The result was a guest with no diagnostics and no hint that it had none,
60+
# on exactly the runs where the diagnostics were the thing being looked for.
61+
log "no disk labelled cozydiag; guest diagnostics not installed"
62+
fi
5563
if [ -n "$DIAG" ] && mount -o ro "$DIAG" "$MNT" 2>/dev/null; then
5664
if [ -s "$MNT/guest-diag.sh" ] && [ -s "$MNT/guest-diag.cron" ]; then
5765
mkdir -p /config/scripts

0 commit comments

Comments
 (0)