Skip to content

Commit 77227c5

Browse files
themoriartimyasnikovdaniil
authored andcommitted
fix(e2e): drop the job-level runner.temp and pin the tenant kubeconfig handling
jobs.e2e.env set COZY_PROXMOX_WORKDIR from runner.temp, but the runner context is not available in job-level env, so GitHub would refuse the whole workflow at load time and the Proxmox lane would stop starting. actionlint reports it as `context "runner" is not allowed here`. Drop the variable. run-proxmox.sh already falls back to RUNNER_TEMP, which the runner exports into every step, and the cleanup step removes the file from there. Pin the behaviour the previous commit added, none of which a test covered: the fetched kubeconfig carries no group or other bits under umask 022, apply-tenant removes a file an earlier run left, and the suite's finally removes the file on the keep_tenant path, so diagnostics after it prints no tenant section. Each case goes red with its line removed. Signed-off-by: Marian Koreniuk <[email protected]> Assisted-by: LLM (cherry picked from commit 3d97e91) Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent 2af6053 commit 77227c5

3 files changed

Lines changed: 69 additions & 11 deletions

File tree

‎.github/workflows/e2e-proxmox.yaml‎

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -49,11 +49,6 @@ jobs:
4949
COZY_PVE_SSH: ${{ vars.PROXMOX_SSH }}
5050
COZY_PVE_SSH_KEY: ${{ github.workspace }}/.ssh-pve
5151
COZY_KEEP_TENANT: ${{ inputs.keep_tenant == true }}
52-
# Where run-proxmox.sh keeps the tenant's super-admin kubeconfig for the
53-
# run. runner.temp is emptied by the runner between jobs, and the cleanup
54-
# step below removes the file as well; see the script for why neither is
55-
# left to /tmp.
56-
COZY_PROXMOX_WORKDIR: ${{ runner.temp }}
5752
steps:
5853
- name: Checkout
5954
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
@@ -162,6 +157,8 @@ jobs:
162157
163158
- name: Remove staged credentials
164159
if: always()
165-
# The tenant kubeconfig is the suite's, fetched by run-proxmox.sh; its
166-
# finally removes it too, but a run killed mid-step never reaches that.
167-
run: rm -f "$GITHUB_WORKSPACE/.kubeconfig" "$COZY_PVE_SSH_KEY" "$COZY_PROXMOX_WORKDIR/proxmox-e2e-tenant.kubeconfig"
160+
# The tenant kubeconfig is the suite's, fetched by run-proxmox.sh into
161+
# RUNNER_TEMP, which the runner exports into every step (the runner
162+
# context is not allowed in job-level env); its finally removes it too,
163+
# but a run killed mid-step never reaches that.
164+
run: rm -f "$GITHUB_WORKSPACE/.kubeconfig" "$COZY_PVE_SSH_KEY" "$RUNNER_TEMP/proxmox-e2e-tenant.kubeconfig"

‎hack/e2e-chainsaw/_lib/run-proxmox.sh‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,10 @@ STORAGE="${COZY_PROXMOX_STORAGE:?COZY_PROXMOX_STORAGE is required, the PVE stora
2626
# a fixed path under /tmp: the file is a credential for the whole tenant, so it
2727
# must not outlive the run on a shared runner, and dump_diagnostics reads
2828
# whatever is at this path, so a file left by a previous run would be reported
29-
# as this run's tenant. The workflow points this at runner.temp, which the
30-
# runner empties between jobs; apply-tenant removes a leftover before it does
31-
# anything else, and the suite's finally removes the file when the run ends.
29+
# as this run's tenant. On the workflow's runner this is RUNNER_TEMP, which the
30+
# runner exports into every step and empties between jobs; apply-tenant removes
31+
# a leftover before it does anything else, and the suite's finally removes the
32+
# file when the run ends.
3233
WORKDIR="${COZY_PROXMOX_WORKDIR:-${RUNNER_TEMP:-${TMPDIR:-/tmp}}}"
3334
TENANT_KUBECONFIG="$WORKDIR/proxmox-e2e-tenant.kubeconfig"
3435
FIXTURE="$(cd "$(dirname "$0")/.." && pwd)/kubernetes-proxmox/tenant.yaml"

‎hack/proxmox-e2e-fixture_test.bats‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,3 +61,63 @@ RUNNER="$REPO_ROOT/hack/e2e-chainsaw/_lib/run-proxmox.sh"
6161
rm -rf "$stub"
6262
case "$out" in *"is gone from main-pool"*) ;; *) echo "$out" >&2; exit 1 ;; esac
6363
}
64+
65+
# The tenant's super-admin kubeconfig is a credential for the whole tenant, kept
66+
# under RUNNER_TEMP for one run. The workflow sets no path of its own, so these
67+
# cases go through the same RUNNER_TEMP fallback the runner provides, with the
68+
# override cleared.
69+
KUBECONFIG_NAME=proxmox-e2e-tenant.kubeconfig
70+
SUITE="$REPO_ROOT/hack/e2e-chainsaw/kubernetes-proxmox/chainsaw-test.yaml.disabled"
71+
72+
@test "tenant-node writes the tenant kubeconfig without group or other bits under umask 022" {
73+
work="$(mktemp -d)"
74+
stub="$(mktemp -d)"
75+
# The secret read answers a kubeconfig and the first tenant read fails, which
76+
# ends the check right after the fetch with the file in place.
77+
cat > "$stub/kubectl" <<'STUB'
78+
#!/bin/sh
79+
case "$*" in *"get secret"*) printf '%s' YXBpVmVyc2lvbjogdjEK; exit 0 ;; esac
80+
exit 1
81+
STUB
82+
chmod +x "$stub/kubectl"
83+
rc=0
84+
(umask 022; PATH="$stub:$PATH" RUNNER_TEMP="$work" COZY_PROXMOX_WORKDIR= COZY_PVE_SSH=x COZY_PROXMOX_STORAGE=s "$RUNNER" tenant-node) >/dev/null 2>&1 || rc=$?
85+
mode="$(ls -l "$work/$KUBECONFIG_NAME" | cut -c1-10)"
86+
rm -rf "$work" "$stub"
87+
[ "$rc" -ne 0 ]
88+
[ "$mode" = "-rw-------" ] || { echo "FAIL: the tenant kubeconfig was written $mode" >&2; exit 1; }
89+
}
90+
91+
@test "apply-tenant removes a tenant kubeconfig an earlier run left behind" {
92+
work="$(mktemp -d)"
93+
stub="$(mktemp -d)"
94+
printf 'an earlier run\n' > "$work/$KUBECONFIG_NAME"
95+
printf '#!/bin/sh\ncat >/dev/null\n' > "$stub/kubectl"
96+
chmod +x "$stub/kubectl"
97+
PATH="$stub:$PATH" RUNNER_TEMP="$work" COZY_PROXMOX_WORKDIR= COZY_PVE_SSH=x COZY_PROXMOX_STORAGE=s "$RUNNER" apply-tenant
98+
left=no
99+
[ ! -e "$work/$KUBECONFIG_NAME" ] || left=yes
100+
rm -rf "$work" "$stub"
101+
[ "$left" = no ] || { echo "FAIL: apply-tenant left the earlier run's kubeconfig in place" >&2; exit 1; }
102+
}
103+
104+
@test "the suite's finally removes the tenant kubeconfig, so diagnostics after it prints no tenant section" {
105+
# keep_tenant exits the finally before its deletes, and the kubeconfig has to
106+
# be gone on that path as well. Diagnostics reads whatever is at the path, so
107+
# a file left there would be reported as the next run's tenant.
108+
work="$(mktemp -d)"
109+
stub="$(mktemp -d)"
110+
printf 'this run\n' > "$work/$KUBECONFIG_NAME"
111+
printf '#!/bin/sh\necho "stub $*"\n' > "$stub/kubectl"
112+
printf '#!/bin/sh\nexit 0\n' > "$stub/ssh"
113+
chmod +x "$stub/kubectl" "$stub/ssh"
114+
finally="$(yq -r '.spec.steps[] | select(.name == "replacement-converges-unattended") | .finally[0].script.content' "$SUITE")"
115+
[ -n "$finally" ]
116+
PATH="$stub:$PATH" RUNNER_TEMP="$work" COZY_PROXMOX_WORKDIR= COZY_KEEP_TENANT=true sh -c "$finally" >/dev/null
117+
out="$(PATH="$stub:$PATH" RUNNER_TEMP="$work" COZY_PROXMOX_WORKDIR= COZY_PVE_SSH=x COZY_PROXMOX_STORAGE=s "$RUNNER" diagnostics 2>&1)"
118+
left=no
119+
[ ! -e "$work/$KUBECONFIG_NAME" ] || left=yes
120+
rm -rf "$work" "$stub"
121+
[ "$left" = no ] || { echo "FAIL: the suite's finally left the tenant kubeconfig in place" >&2; exit 1; }
122+
case "$out" in *"tenant nodes and pods"*) echo "$out" >&2; exit 1 ;; esac
123+
}

0 commit comments

Comments
 (0)