Skip to content

Commit 318be93

Browse files
authored
fix(e2e): capture Talos logs for tenant worker join failures (#3548)
## What this PR does Addresses #3513. When tenant Kubernetes workers miss the 18-minute Ready deadline, the E2E runner now mints a one-hour `os:reader` Talos client certificate from the tenant's existing issuer and runs authenticated `talosctl` from a hardened same-namespace diagnostics Pod. It captures bounded `dmesg` and kubelet output for every worker VMI in the cozyreport snapshot, records missing IPs and command exit codes, continues across unreachable workers, and preserves the original test failure. The diagnostics Pod has no service-account token or Secret volume; credentials are streamed into an `emptyDir`, and the Certificate, Secret, and Pod are labeled for cleanup. Workers that have not reached Talos `apid` can only yield a bounded connection error, while workers that reached `apid` provide guest logs. This adds the evidence needed to distinguish the remaining guest boot failure but does not claim to resolve its underlying cause. The readiness budget and success path are unchanged. Because this changes a shared E2E helper, test-impact analysis selects the full E2E suite for this PR. ### Screenshots Not applicable; no UI changes. ### Testing - `hack/cozytest.sh hack/run-kubernetes-talos-diagnostics_test.bats` (10 tests) - `hack/cozytest.sh hack/run-kubernetes-drain_test.bats` - `hack/cozytest.sh hack/cozyreport-talos.bats` - `bash -n hack/e2e-chainsaw/_lib/run-kubernetes.sh` - `git diff --check origin/main...HEAD` The full `make bats-unit-tests` run reached the unrelated `hack/migration-seaweedfs-db-adopt.bats` suite and could not continue because Docker is unavailable in the development environment. A live failure-path run was not performed because the available development environment has no Talos tenant fixture; this PR is intended to exercise that path in Cozystack CI. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note fix(e2e): capture authenticated Talos kernel and kubelet diagnostics when tenant Kubernetes workers miss the Ready deadline ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved troubleshooting when worker nodes fail to join by collecting per-node operating system and kubelet diagnostics. * Diagnostics now continue across partial failures and clean up temporary resources afterward. * Extended Kubernetes test operation timeouts to allow failure diagnostics to complete reliably. * **Tests** * Added regression coverage for diagnostic collection, timeout handling, multi-worker scenarios, missing node addresses, and cleanup. * **Chores** * Ensured the required diagnostic image is preloaded on nodes. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents b4e2031 + 0612a53 commit 318be93

5 files changed

Lines changed: 602 additions & 14 deletions

File tree

‎hack/e2e-chainsaw/_lib/run-kubernetes.sh‎

Lines changed: 318 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -267,6 +267,16 @@ cozy_cleanup() {
267267
# so they don't leak MetalLB IPs from the shared host pool. Labeled by the
268268
# test so a single selector reaps them all.
269269
kubectl -n tenant-test delete service -l cozystack-e2e.io/tenant-api-lb --ignore-not-found --wait=false 2>/dev/null || true
270+
# A failed node-join capture creates a short-lived reader Certificate and a
271+
# hardened helper Pod. They are labelled separately from the tenant API LB:
272+
# the Secret contains a Talos client key and must be reaped even if the
273+
# diagnostic collector itself returned early.
274+
if ! kubectl -n tenant-test delete certificates.cert-manager.io -l cozystack-e2e.io/tenant-talos-diagnostics --ignore-not-found --wait=true --timeout=30s 2>/dev/null; then
275+
echo "» WARNING: failed to delete tenant Talos diagnostic Certificates" >&2
276+
fi
277+
if ! kubectl -n tenant-test delete pod,secret -l cozystack-e2e.io/tenant-talos-diagnostics --ignore-not-found --wait=false 2>/dev/null; then
278+
echo "» WARNING: failed to delete tenant Talos diagnostic Pod/Secret" >&2
279+
fi
270280
kubectl -n tenant-test delete kuberneteses.apps.cozystack.io --all --ignore-not-found --wait=false 2>/dev/null || true
271281
kubectl -n tenant-test wait kuberneteses.apps.cozystack.io --all --for=delete --timeout=5m 2>/dev/null || true
272282
# The CR delete above finalizes once the Kubernetes CR is gone, which only
@@ -320,6 +330,303 @@ _tenant_snapshot_on_fail() {
320330
esac
321331
}
322332

333+
# Render name|IP rows for worker VMIs from a `kubectl get ... -o json` document.
334+
# Kept pure so an absent default interface is represented by an empty IP instead
335+
# of silently dropping the VMI from the diagnostic report.
336+
cozy_tenant_worker_vmi_rows() {
337+
jq -r '.items[] | [.metadata.name, ([.status.interfaces[]? | select(.name == "default") | .ipAddress][0] // "")] | join("|")'
338+
}
339+
340+
# Ask cert-manager for a short-lived, read-only Talos client certificate. The
341+
# tenant chart deliberately uses TalosConfigTemplate generateType=none, so CABPT
342+
# creates no client talosconfig for this worker-only Kamaji topology. Reusing the
343+
# existing Talos CA Issuer avoids materialising an os:admin credential or copying
344+
# the CA private key out of Kubernetes.
345+
cozy_apply_tenant_talos_reader_certificate() {
346+
local test_name="$1"
347+
local release="kubernetes-${test_name}"
348+
local certificate="${release}-e2e-talos-reader"
349+
350+
kubectl apply --request-timeout=30s -f - <<EOF
351+
apiVersion: cert-manager.io/v1
352+
kind: Certificate
353+
metadata:
354+
name: ${certificate}
355+
namespace: tenant-test
356+
labels:
357+
cozystack-e2e.io/tenant-talos-diagnostics: "${test_name}"
358+
spec:
359+
duration: 1h
360+
renewBefore: 10m
361+
commonName: ${certificate}
362+
secretName: ${certificate}
363+
secretTemplate:
364+
labels:
365+
cozystack-e2e.io/tenant-talos-diagnostics: "${test_name}"
366+
subject:
367+
organizations:
368+
- os:reader
369+
privateKey:
370+
algorithm: Ed25519
371+
rotationPolicy: Always
372+
usages:
373+
- digital signature
374+
- client auth
375+
issuerRef:
376+
name: ${release}-talos-ca
377+
kind: Issuer
378+
group: cert-manager.io
379+
EOF
380+
}
381+
382+
# Build a local talosconfig from the reader Certificate. All private material
383+
# stays below workdir, which the caller creates with mode 0700 and removes from
384+
# a contained subshell EXIT trap; none of it is copied into cozyreport.
385+
cozy_prepare_tenant_talosconfig() (
386+
local test_name="$1"
387+
local workdir="$2"
388+
local release="kubernetes-${test_name}"
389+
local certificate="${release}-e2e-talos-reader"
390+
391+
umask 077
392+
# An interrupted prior run may have left a reader Secret signed by the old
393+
# tenant CA. Remove the Certificate first so cert-manager cannot recreate that
394+
# Secret between deletion and the new request, then wait for both objects to
395+
# disappear before waiting for issuance below.
396+
kubectl -n tenant-test delete certificate "${certificate}" \
397+
--ignore-not-found --wait=true --timeout=10s >/dev/null 2>&1 || return 1
398+
kubectl -n tenant-test delete secret "${certificate}" \
399+
--ignore-not-found --wait=true --timeout=10s >/dev/null 2>&1 || return 1
400+
cozy_apply_tenant_talos_reader_certificate "${test_name}" || return 1
401+
402+
# cert-manager issuance is asynchronous. Wait on its actual Ready condition;
403+
# on expiry, preserve Certificate diagnostics without ever printing the
404+
# generated Secret.
405+
if ! kubectl_wait_retry -n tenant-test certificate "${certificate}" \
406+
--for=condition=Ready --timeout=30s; then
407+
echo "tenant Talos reader Certificate was not issued within 30s" >&2
408+
kubectl -n tenant-test describe certificate "${certificate}" --request-timeout=30s >&2 || true
409+
kubectl -n tenant-test get certificaterequests.cert-manager.io \
410+
-l cert-manager.io/certificate-name="${certificate}" -o wide --request-timeout=30s >&2 || true
411+
return 1
412+
fi
413+
414+
kubectl -n tenant-test get secret "${release}-talos-ca" \
415+
-o go-template='{{index .data "tls.crt" | base64decode}}' --request-timeout=30s >"${workdir}/ca.crt" || return 1
416+
kubectl -n tenant-test get secret "${certificate}" \
417+
-o go-template='{{index .data "tls.crt" | base64decode}}' --request-timeout=30s >"${workdir}/client.crt" || return 1
418+
kubectl -n tenant-test get secret "${certificate}" \
419+
-o go-template='{{index .data "tls.key" | base64decode}}' --request-timeout=30s >"${workdir}/client.key" || return 1
420+
421+
talosctl --talosconfig "${workdir}/talosconfig" config add "${release}" \
422+
--ca "${workdir}/ca.crt" --crt "${workdir}/client.crt" \
423+
--key "${workdir}/client.key" || return 1
424+
talosctl --talosconfig "${workdir}/talosconfig" config context "${release}" || return 1
425+
chmod 600 "${workdir}/talosconfig"
426+
)
427+
428+
# Apply the helper Pod separately from its readiness/copy steps so its security
429+
# invariants have focused unit coverage. The Pod runs on the management cluster
430+
# in the same namespace as bridge-networked worker VMIs, which lets talosctl use
431+
# each real VMI IP as both endpoint and node and keeps server-TLS verification
432+
# valid. A MetalLB or localhost port-forward address would not be in the Talos
433+
# serving certificate SANs.
434+
cozy_apply_tenant_talos_diagnostics_pod() {
435+
local test_name="$1"
436+
local pod_name="kubernetes-${test_name}-talos-diagnostics"
437+
438+
kubectl apply --request-timeout=30s -f - <<EOF
439+
apiVersion: v1
440+
kind: Pod
441+
metadata:
442+
name: ${pod_name}
443+
namespace: tenant-test
444+
labels:
445+
cozystack-e2e.io/tenant-talos-diagnostics: "${test_name}"
446+
spec:
447+
activeDeadlineSeconds: 900
448+
automountServiceAccountToken: false
449+
enableServiceLinks: false
450+
restartPolicy: Never
451+
terminationGracePeriodSeconds: 1
452+
securityContext:
453+
runAsNonRoot: true
454+
runAsUser: 1000
455+
runAsGroup: 1000
456+
fsGroup: 1000
457+
seccompProfile:
458+
type: RuntimeDefault
459+
containers:
460+
- name: diagnostics
461+
# The E2E sandbox ships the statically linked upstream talosctl release, so
462+
# reuse the digest-pinned Alpine helper already pulled by the test setup.
463+
image: docker.io/alpine/k8s:1.36.2@sha256:44ef4942e171939b9c665a4a84beb80e2dcdb9a24330d4651cfdfd2e9deecc47
464+
imagePullPolicy: IfNotPresent
465+
command: ["sh", "-c", "sleep 900"]
466+
securityContext:
467+
allowPrivilegeEscalation: false
468+
capabilities:
469+
drop:
470+
- ALL
471+
readOnlyRootFilesystem: true
472+
resources:
473+
requests:
474+
cpu: 10m
475+
memory: 16Mi
476+
limits:
477+
cpu: 200m
478+
memory: 256Mi
479+
volumeMounts:
480+
- name: tmp
481+
mountPath: /tmp
482+
volumes:
483+
- name: tmp
484+
emptyDir: {}
485+
EOF
486+
}
487+
488+
# Start the helper Pod and stream talosctl plus the reader talosconfig into its
489+
# emptyDir. Streaming avoids a Secret volume and keeps the Pod independent of
490+
# kubectl-cp/tar behavior. The final cleanup selector remains the authoritative
491+
# backstop if any preparation step fails.
492+
cozy_prepare_tenant_talos_diagnostics_pod() {
493+
local test_name="$1"
494+
local talosconfig="$2"
495+
local pod_name="kubernetes-${test_name}-talos-diagnostics"
496+
local talosctl_bin
497+
498+
talosctl_bin=$(command -v talosctl) || return 1
499+
kubectl -n tenant-test delete pod "${pod_name}" --ignore-not-found \
500+
--wait=true --timeout=10s >/dev/null 2>&1 || return 1
501+
cozy_apply_tenant_talos_diagnostics_pod "${test_name}" || return 1
502+
503+
if ! kubectl -n tenant-test wait pod "${pod_name}" \
504+
--for=condition=Ready --timeout=60s; then
505+
kubectl -n tenant-test describe pod "${pod_name}" --request-timeout=30s >&2 || true
506+
kubectl -n tenant-test get events \
507+
--field-selector involvedObject.name="${pod_name}" \
508+
--sort-by=.lastTimestamp --request-timeout=30s >&2 || true
509+
return 1
510+
fi
511+
512+
if ! timeout -k 5 60 kubectl -n tenant-test exec -i "${pod_name}" \
513+
-c diagnostics -- sh -ec 'cat > /tmp/talosctl && chmod 500 /tmp/talosctl' \
514+
<"${talosctl_bin}"; then
515+
echo "failed to stream talosctl into tenant diagnostics Pod" >&2
516+
return 1
517+
fi
518+
if ! timeout -k 5 20 kubectl -n tenant-test exec -i "${pod_name}" \
519+
-c diagnostics -- sh -ec 'umask 077; cat > /tmp/talosconfig; chmod 600 /tmp/talosconfig' \
520+
<"${talosconfig}"; then
521+
echo "failed to stream tenant talosconfig into diagnostics Pod" >&2
522+
return 1
523+
fi
524+
}
525+
526+
# Capture one bounded Talos command through the helper Pod and retain the exit
527+
# status alongside partial output. A timeout or unreachable pre-apid worker is
528+
# itself useful evidence and must not erase captures from the other worker.
529+
cozy_capture_tenant_talos_command() {
530+
local pod_name="$1"
531+
local node_ip="$2"
532+
local output="$3"
533+
shift 3
534+
local rc=0
535+
536+
timeout -k 5 20 kubectl -n tenant-test exec "${pod_name}" -c diagnostics -- \
537+
/tmp/talosctl --talosconfig /tmp/talosconfig \
538+
-e "${node_ip}" -n "${node_ip}" "$@" >"${output}" 2>&1 || rc=$?
539+
printf '\n[capture exit code: %s]\n' "${rc}" >>"${output}"
540+
return "${rc}"
541+
}
542+
543+
cozy_capture_tenant_talos_node() {
544+
local pod_name="$1"
545+
local node_ip="$2"
546+
local output_dir="$3"
547+
548+
mkdir -p "${output_dir}"
549+
printf 'endpoint: %s\nnode: %s\n' "${node_ip}" "${node_ip}" >"${output_dir}/target.txt"
550+
cozy_capture_tenant_talos_command "${pod_name}" "${node_ip}" \
551+
"${output_dir}/dmesg.log" dmesg || true
552+
cozy_capture_tenant_talos_command "${pod_name}" "${node_ip}" \
553+
"${output_dir}/kubelet.log" logs kubelet --tail=500 || true
554+
}
555+
556+
# Collect the guest-side evidence requested by issue #3513. This runs only after
557+
# the 18-minute Ready deadline has already failed. VMIs without a reported IP
558+
# are recorded before any Certificate or Pod is created; when at least one IP is
559+
# available, setup and each Talos command remain bounded. It runs before slower
560+
# cache diagnostics so the requested guest evidence lands first.
561+
cozy_capture_tenant_talos() (
562+
local test_name="$1"
563+
local report_dir="${COZY_REPORT_DIR:-/workspace/_out/cozyreport}/snapshots/${COZY_SNAPSHOT_NAME:-kubernetes}/tenant-talos"
564+
local selector="cluster.x-k8s.io/cluster-name=kubernetes-${test_name},cluster.x-k8s.io/role=worker"
565+
local pod_name="kubernetes-${test_name}-talos-diagnostics"
566+
local workdir vmi_name node_ip
567+
local has_node_ip=false
568+
569+
umask 077
570+
workdir=$(mktemp -d) || return 1
571+
chmod 700 "${workdir}"
572+
trap 'rm -rf "${workdir}"' EXIT
573+
mkdir -p "${report_dir}"
574+
575+
if ! kubectl -n tenant-test get virtualmachineinstances.kubevirt.io \
576+
-l "${selector}" -o json --request-timeout=30s >"${report_dir}/vmis.json" \
577+
2>"${report_dir}/vmis-error.log"; then
578+
echo "failed to list tenant worker VMIs for Talos diagnostics" >&2
579+
return 1
580+
fi
581+
cozy_tenant_worker_vmi_rows <"${report_dir}/vmis.json" >"${workdir}/vmis.rows"
582+
if [ ! -s "${workdir}/vmis.rows" ]; then
583+
echo "no tenant worker VMIs found for Talos diagnostics" >&2
584+
printf 'no tenant worker VMIs matched selector %s\n' "${selector}" \
585+
>"${report_dir}/setup-error.log"
586+
return 1
587+
fi
588+
589+
# Record missing addresses before spending time on Certificate issuance or a
590+
# helper Pod. If no worker booted far enough for KubeVirt to report an IP,
591+
# guest-side Talos is unreachable and setup cannot produce more evidence.
592+
while IFS='|' read -r vmi_name node_ip; do
593+
if [ -z "${node_ip}" ]; then
594+
mkdir -p "${report_dir}/${vmi_name}"
595+
printf '%s\n' 'default VMI interface has no reported IP address' \
596+
>"${report_dir}/${vmi_name}/capture-error.log"
597+
continue
598+
fi
599+
has_node_ip=true
600+
done <"${workdir}/vmis.rows"
601+
if [ "${has_node_ip}" != true ]; then
602+
echo "no tenant worker VMI has a reported IP for Talos diagnostics" >&2
603+
printf '%s\n' 'no tenant worker VMI has a reported IP for Talos diagnostics' \
604+
>"${report_dir}/setup-error.log"
605+
return 1
606+
fi
607+
608+
if ! cozy_prepare_tenant_talosconfig "${test_name}" "${workdir}"; then
609+
echo "failed to create short-lived tenant Talos reader config" >&2
610+
printf '%s\n' 'failed to create short-lived tenant Talos reader config' \
611+
>"${report_dir}/setup-error.log"
612+
return 1
613+
fi
614+
if ! cozy_prepare_tenant_talos_diagnostics_pod \
615+
"${test_name}" "${workdir}/talosconfig"; then
616+
echo "failed to prepare tenant Talos diagnostics Pod" >&2
617+
printf '%s\n' 'failed to prepare tenant Talos diagnostics Pod' \
618+
>"${report_dir}/setup-error.log"
619+
return 1
620+
fi
621+
622+
while IFS='|' read -r vmi_name node_ip; do
623+
[ -n "${node_ip}" ] || continue
624+
echo "--- capturing tenant Talos dmesg/kubelet: ${vmi_name} (${node_ip}) ---"
625+
cozy_capture_tenant_talos_node "${pod_name}" "${node_ip}" \
626+
"${report_dir}/${vmi_name}"
627+
done <"${workdir}/vmis.rows"
628+
)
629+
323630
run_kubernetes_test() {
324631
local version_expr="$1"
325632
local test_name="$2"
@@ -544,7 +851,7 @@ EOF
544851
# kubernetes-previous failed here at exactly 12m with zero Nodes registered
545852
# and the tenant cilium HR still mid-install. A less-loaded fleet run passed
546853
# the same suites unchanged, so this is load-induced slowness, not a stuck
547-
# bring-up; 18m restores margin and still sits well inside the 40m step
854+
# bring-up; 18m restores margin and still sits well inside the 50m step
548855
# timeout (the downstream LB/NFS/ouroboros checks add ~10-15m on the happy
549856
# path).
550857
if ! timeout 18m bash -c '
@@ -579,6 +886,16 @@ EOF
579886
kubectl -n tenant-test get pods -l kubevirt.io=virt-launcher -o wide || true
580887
kubectl -n tenant-test describe pods -l kubevirt.io=virt-launcher || true
581888

889+
# (b) In-guest Talos kernel and kubelet logs. The tenant chart intentionally
890+
# has no admin talosconfig, so mint a one-hour os:reader client from its
891+
# existing cert-manager Issuer and run talosctl from a hardened Pod that can
892+
# reach the bridge-networked VMI IPs without weakening TLS verification.
893+
# A worker that has not reached apid yet will produce a bounded connection
894+
# error while a later-stage worker remains capturable; both outcomes are
895+
# retained in cozyreport. Diagnostic setup failures never mask exit 1 below.
896+
echo "=== (b) in-guest Talos dmesg + kubelet logs ==="
897+
cozy_capture_tenant_talos "${test_name}" || true
898+
582899
# (a2) Worker DataVolume IMPORT stage. A VM stuck "Provisioning" whose
583900
# DataVolume is ImportInProgress at N/A progress with the importer pod
584901
# looping on an HTTP error is a distinct sub-mode of 2a that the VM/VMI
@@ -611,17 +928,6 @@ EOF
611928
kubectl -n tenant-test logs -l kamaji.clastix.io/name="kubernetes-${test_name}" \
612929
-c talos-csr-signer --tail=200 --prefix || true
613930

614-
# (b) In-guest Talos/kubelet state from the worker VMs is intentionally NOT
615-
# captured here. talosctl needs a client talosconfig for the TENANT cluster,
616-
# and the runner has none: the tenant workers are provisioned with their own
617-
# Talos PKI whose CA differs from the sandbox's /workspace/talosconfig (which
618-
# cozyreport.sh uses to reach the MANAGEMENT nodes only), and the chart
619-
# materialises no tenant client talosconfig Secret. Pointing talosctl at the
620-
# worker IPs with the management talosconfig would just fail mTLS and capture
621-
# nothing, so it is skipped rather than shipped as a misleading no-op. (a) +
622-
# (c) carry the 2a-vs-2b split; adding real in-guest capture later requires
623-
# wiring a tenant talosconfig into the runner first.
624-
echo "=== (b) in-guest Talos/kubelet capture skipped: no tenant talosconfig on the runner (a/c cover the 2a-vs-2b split) ==="
625931
exit 1
626932
fi
627933
kubectl --kubeconfig "tenantkubeconfig-${test_name}" get nodes -o wide

‎hack/e2e-chainsaw/kubernetes-latest/chainsaw-test.yaml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,10 @@ spec:
1919
try:
2020
- script:
2121
# ~25m Kamaji bringup + worker join + LB/NFS/ouroboros assertions.
22-
timeout: 40m
22+
# The failure path can spend ~5m10s in bounded guest Talos capture and
23+
# another 6m30s in its in-process tenant snapshot. Keep the outer op
24+
# above those inner budgets with useful scheduling/API-call slack.
25+
timeout: 50m
2326
# Co-locate the tenant crust-gather snapshot (captured by
2427
# run-kubernetes.sh on failure) under the same snapshots/<test>/ dir as
2528
# the global .chainsaw.yaml catch's host snapshot.

‎hack/e2e-chainsaw/kubernetes-previous/chainsaw-test.yaml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,10 @@ spec:
1313
- name: tenant-kubernetes-previous-version
1414
try:
1515
- script:
16-
timeout: 40m
16+
# The failure path can spend ~5m10s in bounded guest Talos capture and
17+
# another 6m30s in its in-process tenant snapshot. Keep the outer op
18+
# above those inner budgets with useful scheduling/API-call slack.
19+
timeout: 50m
1720
# Co-locate the tenant crust-gather snapshot (captured by
1821
# run-kubernetes.sh on failure) under the same snapshots/<test>/ dir as
1922
# the global .chainsaw.yaml catch's host snapshot.

0 commit comments

Comments
 (0)