Skip to content

Commit 25f408d

Browse files
committed
fix(clickhouse): harden version generator per review
Address review feedback on the version-map generator: - Compute the server/keeper tag intersection byte-collated (LC_ALL=C sort) so comm no longer drops common tags at X.9 -> X.10 boundaries, which could silently remove a whole major from the enum. - Patch-pin the default line (v24.9 -> 24.9.2.42) so a regeneration reproduces files/versions.yaml verbatim and never bumps the image of existing default-version installs. - Fail before writing any file when a configured major has no tag common to both images, and build both artifacts in temp files moved into place at the end (atomic regeneration). - Make the script run on stock macOS: drop the bash-4 associative array and the BSD-incompatible `awk -v` newline / `sort -V`. - Note the downgrade hazard in the version parameter description. - Add hack/clickhouse-update-versions_test.bats covering the generator; make the helm-unittest version assertions robust to patch bumps and to Helm's exact schema-error phrasing. Regenerated values.schema.json, README.md, types.go and clickhouse-rd. Signed-off-by: Alexey Artamonov <[email protected]>
1 parent 5625b9e commit 25f408d

8 files changed

Lines changed: 296 additions & 137 deletions

File tree

‎api/apps/v1alpha1/clickhouse/types.go‎

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
#!/usr/bin/env bats
2+
# Unit test: packages/apps/clickhouse/hack/update-versions.sh (the ClickHouse
3+
# version-map generator).
4+
#
5+
# The generator turns the Docker Hub tag lists of clickhouse/clickhouse-server
6+
# and clickhouse/clickhouse-keeper into files/versions.yaml (a major -> full
7+
# tag map) and rewrites the version enum in values.yaml. These properties are
8+
# load-bearing and each guards a specific failure the review of #3476 caught:
9+
#
10+
# 1. The server/keeper tag intersection must be computed with a byte
11+
# (lexicographic) collation, not a version collation. `comm` walks its
12+
# inputs assuming byte order; feeding it `sort -V` output silently drops
13+
# common tags at every X.9 -> X.10 boundary, so a whole major line can
14+
# vanish from the enum and existing tenant CRs pinned to it then fail
15+
# schema validation.
16+
# 2. The default line is patch-pinned and must be reproduced verbatim, so a
17+
# regeneration never bumps the image of existing default-version installs
18+
# (the "renders byte-for-byte identically" guarantee of the PR).
19+
# 3. A configured major with no tag common to both images is a hard error,
20+
# and the generator is atomic: on that error the committed files are left
21+
# untouched rather than half-rewritten.
22+
#
23+
# The generator is exercised offline: CH_SERVER_TAGS_FILE / CH_KEEPER_TAGS_FILE
24+
# inject the tag lists (no curl), and VALUES_FILE / VERSIONS_FILE stay in a
25+
# scratch dir so the real chart is never touched. Written in the portable
26+
# subset hack/cozytest.sh understands (no `run`/`$status`/`$lines`, no
27+
# setup/teardown, no standalone `}` inside a test body): each assertion exits
28+
# non-zero on failure and the body relies on `set -e`.
29+
30+
REPO_ROOT="$(cd "$(dirname "${BATS_TEST_FILENAME:-$0}")/.." && pwd)"
31+
GEN="$REPO_ROOT/packages/apps/clickhouse/hack/update-versions.sh"
32+
33+
# A minimal values.yaml carrying the version section the generator rewrites,
34+
# plus the section boundary it inserts before when the section is absent.
35+
seed_values() {
36+
cat > "$1" <<'EOF'
37+
## @param {string} storageClass - StorageClass used to store the data.
38+
storageClass: ""
39+
40+
## @enum {string} Version
41+
## @value v24.9
42+
## @param {Version} version - ClickHouse major.minor version to deploy.
43+
version: v24.9
44+
45+
##
46+
## @section Application-specific parameters
47+
##
48+
## @param {int} logTTL - TTL.
49+
logTTL: 15
50+
EOF
51+
}
52+
53+
# Per-test scratch state (no setup() — cozytest.sh does not call it).
54+
init_case() {
55+
WORK="$(mktemp -d)"
56+
SERVER="$WORK/server.txt"
57+
KEEPER="$WORK/keeper.txt"
58+
export VERSIONS_FILE="$WORK/versions.yaml"
59+
export VALUES_FILE="$WORK/values.yaml"
60+
export CH_SERVER_TAGS_FILE="$SERVER"
61+
export CH_KEEPER_TAGS_FILE="$KEEPER"
62+
seed_values "$VALUES_FILE"
63+
: > "$VERSIONS_FILE"
64+
}
65+
66+
@test "intersection keeps tags across the X.9 -> X.10 boundary" {
67+
init_case
68+
printf '%s\n' 24.9.2.42 24.9.3.128 24.10.1.1 > "$SERVER"
69+
cp "$SERVER" "$KEEPER"
70+
CH_SUPPORTED_MAJORS="24.10 24.9" bash "$GEN"
71+
grep -q '^"v24.10": "24.10.1.1"$' "$VERSIONS_FILE" || { echo "24.10 dropped from intersection" >&2; exit 1; }
72+
grep -q '^"v24.9":' "$VERSIONS_FILE" || { echo "24.9 dropped from intersection" >&2; exit 1; }
73+
}
74+
75+
@test "the pinned default line is reproduced, not bumped to the latest patch" {
76+
init_case
77+
printf '%s\n' 24.9.2.42 24.9.3.128 > "$SERVER"
78+
cp "$SERVER" "$KEEPER"
79+
CH_SUPPORTED_MAJORS="24.9" bash "$GEN"
80+
grep -q '^"v24.9": "24.9.2.42"$' "$VERSIONS_FILE" || { echo "default not pinned to 24.9.2.42" >&2; exit 1; }
81+
if grep -q '24.9.3.128' "$VERSIONS_FILE"; then echo "default bumped to newer patch" >&2; exit 1; fi
82+
}
83+
84+
@test "non-default majors resolve to their latest common patch" {
85+
init_case
86+
printf '%s\n' 25.3.14.14 25.3.15.1 25.8.28.1 > "$SERVER"
87+
cp "$SERVER" "$KEEPER"
88+
CH_SUPPORTED_MAJORS="25.8 25.3" bash "$GEN"
89+
grep -q '^"v25.3": "25.3.15.1"$' "$VERSIONS_FILE" || { echo "25.3 not latest patch" >&2; exit 1; }
90+
grep -q '^"v25.8": "25.8.28.1"$' "$VERSIONS_FILE" || { echo "25.8 missing" >&2; exit 1; }
91+
}
92+
93+
@test "a tag present in only one image is not selected" {
94+
init_case
95+
printf '%s\n' 25.8.28.1 25.8.29.9 > "$SERVER"
96+
printf '%s\n' 25.8.28.1 > "$KEEPER"
97+
CH_SUPPORTED_MAJORS="25.8" bash "$GEN"
98+
grep -q '^"v25.8": "25.8.28.1"$' "$VERSIONS_FILE" || { echo "server-only tag was selected" >&2; exit 1; }
99+
}
100+
101+
@test "an unresolvable configured major is a hard error" {
102+
init_case
103+
printf '%s\n' 24.9.2.42 > "$SERVER"
104+
cp "$SERVER" "$KEEPER"
105+
if CH_SUPPORTED_MAJORS="26.99 24.9" bash "$GEN" >/dev/null 2>&1; then echo "expected non-zero exit" >&2; exit 1; fi
106+
}
107+
108+
@test "on error the committed files are left untouched (atomic)" {
109+
init_case
110+
printf '%s\n' 24.9.2.42 > "$SERVER"
111+
cp "$SERVER" "$KEEPER"
112+
cp "$VERSIONS_FILE" "$WORK/versions.before"
113+
cp "$VALUES_FILE" "$WORK/values.before"
114+
if CH_SUPPORTED_MAJORS="26.99" bash "$GEN" >/dev/null 2>&1; then echo "expected non-zero exit" >&2; exit 1; fi
115+
cmp -s "$VERSIONS_FILE" "$WORK/versions.before" || { echo "versions.yaml changed on failure" >&2; exit 1; }
116+
cmp -s "$VALUES_FILE" "$WORK/values.before" || { echo "values.yaml changed on failure" >&2; exit 1; }
117+
}
118+
119+
@test "values.yaml enum is rewritten newest-first with the pinned default" {
120+
init_case
121+
printf '%s\n' 24.9.2.42 25.3.14.14 25.8.28.1 > "$SERVER"
122+
cp "$SERVER" "$KEEPER"
123+
CH_SUPPORTED_MAJORS="25.8 25.3 24.9" bash "$GEN"
124+
first="$(grep -m1 '@value' "$VALUES_FILE")"
125+
[ "$first" = "## @value v25.8" ] || { echo "enum not newest-first, got: $first" >&2; exit 1; }
126+
grep -q '^version: v24.9$' "$VALUES_FILE" || { echo "default not preserved as v24.9" >&2; exit 1; }
127+
grep -q '^## @section Application-specific parameters$' "$VALUES_FILE" || { echo "section boundary lost" >&2; exit 1; }
128+
grep -q '^logTTL: 15$' "$VALUES_FILE" || { echo "trailing content lost" >&2; exit 1; }
129+
}

‎packages/apps/clickhouse/README.md‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -108,17 +108,17 @@ chart-emitted CronJob renders only when `backup.schedule` is non-empty.
108108

109109
### Common parameters
110110

111-
| Name | Description | Type | Value |
112-
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------ | ---------- | ---------- |
113-
| `replicas` | Number of ClickHouse replicas. | `int` | `2` |
114-
| `shards` | Number of ClickHouse shards. | `int` | `1` |
115-
| `resources` | Explicit CPU and memory configuration for each ClickHouse replica. When omitted, the preset defined in `resourcesPreset` is applied. | `object` | `{}` |
116-
| `resources.cpu` | CPU available to each replica. | `quantity` | `""` |
117-
| `resources.memory` | Memory (RAM) available to each replica. | `quantity` | `""` |
118-
| `resourcesPreset` | Default sizing preset used when `resources` is omitted. | `string` | `t1.small` |
119-
| `size` | Persistent Volume Claim size available for application data. | `quantity` | `10Gi` |
120-
| `storageClass` | StorageClass used to store the data. | `string` | `""` |
121-
| `version` | ClickHouse major.minor version to deploy. Applies to both the ClickHouse server and ClickHouse Keeper images. | `string` | `v24.9` |
111+
| Name | Description | Type | Value |
112+
| ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ---------- |
113+
| `replicas` | Number of ClickHouse replicas. | `int` | `2` |
114+
| `shards` | Number of ClickHouse shards. | `int` | `1` |
115+
| `resources` | Explicit CPU and memory configuration for each ClickHouse replica. When omitted, the preset defined in `resourcesPreset` is applied. | `object` | `{}` |
116+
| `resources.cpu` | CPU available to each replica. | `quantity` | `""` |
117+
| `resources.memory` | Memory (RAM) available to each replica. | `quantity` | `""` |
118+
| `resourcesPreset` | Default sizing preset used when `resources` is omitted. | `string` | `t1.small` |
119+
| `size` | Persistent Volume Claim size available for application data. | `quantity` | `10Gi` |
120+
| `storageClass` | StorageClass used to store the data. | `string` | `""` |
121+
| `version` | ClickHouse major.minor version to deploy. Applies to both the ClickHouse server and ClickHouse Keeper images. Downgrading to an older major is unsafe (ClickHouse cannot read data written by a newer server and Keeper snapshots are not backward compatible) — only increase this value. | `string` | `v24.9` |
122122

123123

124124
### Application-specific parameters

0 commit comments

Comments
 (0)