Skip to content

Commit 8d81fc5

Browse files
committed
fix(modeline): split modeline parts on top-level commas only
ParseModeline previously split the modeline body on the literal `, ` (comma+space) string, which is the separator GenerateModeline emits between key-pairs. JSON arrays serialised by encoding/json carry no space after each inner comma ("["a","b"]"), so the canonical talm-generated form parsed back cleanly. But a hand-authored modeline with whitespace inside the array ("nodes=["a", "b"]" — the natural shape) was cut at the first inner comma. The leading half ("nodes=["a"") then failed JSON parsing with "unexpected end of JSON input", with a hint that misled the operator into thinking their JSON was malformed. Replace the literal split with a depth-aware tokenizer that tracks `[`/`]` nesting and JSON string state (including backslash escapes) and only emits a key-pair boundary when it sees a `,` at depth 0 outside a string literal. The canonical no-space form still parses; the human-written form now parses; commas inside string literals never split the value. Update the contract test that pinned the old strict separator to document the relaxation, and add cases for multi-element arrays with various whitespace shapes plus comma-in-string-literal coverage. Extend the manual-test-plan with section C6f covering the parser behaviour on both anchor and side-patch slots. Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent a0bd589 commit 8d81fc5

4 files changed

Lines changed: 270 additions & 16 deletions

File tree

‎docs/manual-test-plan.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -377,6 +377,34 @@ Inspect any apply progress line from C1, C3, C6a, C6c. Expect `nodes=[n1,n2,n3]`
377377

378378
Regression anchor: the format uses `strings.Join(slice, ",")` with surrounding brackets in the format string; `%s` or `%v` directly on a `[]string` produces the space-separated `[n1 n2 n3]` artifact and is a regression. Verify with a multi-node modeline (`# talm: nodes=["1.2.3.4","5.6.7.8"], ...`) → progress line shows `nodes=[1.2.3.4,5.6.7.8]`.
379379

380+
### C6f. Multi-IP modeline parses regardless of whitespace inside the JSON array
381+
382+
```bash
383+
# Anchor with hand-written multi-IP modeline (note spaces after commas):
384+
cat > nodes/multi.yaml <<'EOF'
385+
# talm: nodes=["1.2.3.4", "5.6.7.8"], endpoints=["1.2.3.4", "5.6.7.8"], templates=["templates/controlplane.yaml"]
386+
machine: {}
387+
EOF
388+
talm apply --dry-run -f nodes/multi.yaml
389+
390+
# Side-patch slot that accidentally carries a modeline (rejection path):
391+
cat > /tmp/side-modelined.yaml <<'EOF'
392+
# talm: nodes=["1.2.3.4", "5.6.7.8"], endpoints=["1.2.3.4", "5.6.7.8"]
393+
machine:
394+
kernel:
395+
modules:
396+
- name: dm_thin_pool
397+
EOF
398+
talm apply --dry-run -f nodes/node0.yaml -f /tmp/side-modelined.yaml
399+
```
400+
401+
Expected:
402+
403+
- First invocation: progress line includes `nodes=[1.2.3.4,5.6.7.8]` (two nodes, parsed from the multi-element array). The whitespace after each comma inside `nodes=[…]` and `endpoints=[…]` is tolerated.
404+
- Second invocation: rejection error `side-patch /tmp/side-modelined.yaml carries its own modeline; the apply chain treats only the first -f file as the anchor and stacks subsequent files as side-patches` with the per-file-loop / strip-modeline hint. The error is the high-level "side-patches must not have modelines" gate from `rejectModelinedSidePatches`, NOT a low-level `error parsing JSON array for key nodes` — the parser successfully reads the multi-IP form and the rejection happens on the (correctly identified) modelined-side-patch shape.
405+
406+
Regression anchor: the modeline parser splits on commas at JSON-array depth 0 only. A regression that re-introduces a literal comma-plus-space split would silently truncate multi-element arrays — the anchor case above would fail with `unexpected end of JSON input` on `["1.2.3.4"` (array cut at the first comma). Verify the depth tracking by adding a comma inside a string literal: `# talm: nodes=["a,b"]` must parse to `Nodes=["a,b"]`, not split into two tokens.
407+
380408
### C7. Phase 1 walker rejects malformed net-addr fields before the RPC
381409

382410
When a rendered MachineConfig carries a malformed value in any of the new walker-covered fields, Phase 1 must block before the apply RPC fires:

‎pkg/modeline/contract_test.go‎

Lines changed: 129 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -94,25 +94,140 @@ func TestContract_ParseModeline_RejectsMalformedKeyValue(t *testing.T) {
9494
}
9595
}
9696

97-
// Contract: keys are separated by `, ` (comma then a single space).
98-
// This is the same separator GenerateModeline emits, so a generated
99-
// modeline always parses back. Missing the space before the next key
100-
// fails to find the part-boundary; trailing whitespace inside a JSON
101-
// array is tolerated (json.Unmarshal handles it).
102-
func TestContract_ParseModeline_KeyValueSeparatorContract(t *testing.T) {
103-
// Canonical (matches GenerateModeline output).
104-
canonical := `# talm: nodes=["a"], endpoints=["b"], templates=["c"]`
105-
got, err := ParseModeline(canonical)
106-
if err != nil {
107-
t.Fatalf("canonical line failed: %v", err)
108-
}
97+
// Contract: keys are separated by `,` at JSON-array depth 0. The
98+
// canonical separator GenerateModeline emits is `, ` (comma+space),
99+
// but the parser is depth-aware so it also accepts the no-space form
100+
// and arbitrary whitespace around the comma. A comma INSIDE a
101+
// `nodes=["a", "b"]` array is array-internal and never splits the
102+
// key-pair. This relaxation lets operators hand-author shared
103+
// side-patches with multi-IP modelines in the natural form.
104+
//
105+
// All four forms below MUST parse to the same Config — the parser is
106+
// liberal in what it accepts; GenerateModeline stays strict on output.
107+
func TestContract_ParseModeline_KeyValueSeparator(t *testing.T) {
109108
want := &Config{
110109
Nodes: []string{"a"},
111110
Endpoints: []string{"b"},
112111
Templates: []string{"c"},
113112
}
114-
if !reflect.DeepEqual(got, want) {
115-
t.Errorf("canonical parse mismatch\n got: %+v\nwant: %+v", got, want)
113+
cases := []struct {
114+
name string
115+
line string
116+
}{
117+
{"canonical (matches GenerateModeline)", `# talm: nodes=["a"], endpoints=["b"], templates=["c"]`},
118+
{"no space after comma", `# talm: nodes=["a"],endpoints=["b"],templates=["c"]`},
119+
{"multiple spaces after comma", `# talm: nodes=["a"], endpoints=["b"], templates=["c"]`},
120+
{"space before comma too", `# talm: nodes=["a"] , endpoints=["b"] , templates=["c"]`},
121+
}
122+
for _, tc := range cases {
123+
t.Run(tc.name, func(t *testing.T) {
124+
got, err := ParseModeline(tc.line)
125+
if err != nil {
126+
t.Fatalf("expected to parse, got: %v", err)
127+
}
128+
if !reflect.DeepEqual(got, want) {
129+
t.Errorf("parse mismatch\n got: %+v\nwant: %+v", got, want)
130+
}
131+
})
132+
}
133+
}
134+
135+
// Contract: a comma inside a JSON array (multi-element value) never
136+
// splits the key-pair. This is the depth-0 promise that the old
137+
// literal `, ` split broke: `nodes=["a", "b"]` was cut to
138+
// `nodes=["a"` + ` "b"]` and the first half failed JSON parsing.
139+
func TestContract_ParseModeline_CommaInsideArrayNeverSplits(t *testing.T) {
140+
cases := []struct {
141+
name string
142+
line string
143+
want *Config
144+
}{
145+
{
146+
"multi-element nodes",
147+
`# talm: nodes=["1.2.3.4", "5.6.7.8"]`,
148+
&Config{Nodes: []string{testNodeIP1, "5.6.7.8"}},
149+
},
150+
{
151+
"comma inside string literal",
152+
`# talm: nodes=["a,b","c"]`,
153+
&Config{Nodes: []string{"a,b", "c"}},
154+
},
155+
{
156+
"escaped quote inside string literal",
157+
`# talm: nodes=["a\"b","c"]`,
158+
&Config{Nodes: []string{`a"b`, "c"}},
159+
},
160+
{
161+
// Square brackets inside a string literal must not affect
162+
// the depth counter — inString short-circuits the `[`/`]`
163+
// arms of the splitter switch. A regression that drops the
164+
// short-circuit would miscount depth and either split on a
165+
// `,` inside the string, or fail to split on a legitimate
166+
// key-pair `,` that follows.
167+
"square brackets inside string literal",
168+
`# talm: nodes=["[,]","x"]`,
169+
&Config{Nodes: []string{"[,]", "x"}},
170+
},
171+
}
172+
for _, tc := range cases {
173+
t.Run(tc.name, func(t *testing.T) {
174+
got, err := ParseModeline(tc.line)
175+
if err != nil {
176+
t.Fatalf("expected to parse, got: %v", err)
177+
}
178+
if !reflect.DeepEqual(got, tc.want) {
179+
t.Errorf("parse mismatch\n got: %+v\nwant: %+v", got, tc.want)
180+
}
181+
})
182+
}
183+
}
184+
185+
// Contract: a trailing `,` at depth 0 (e.g. `nodes=["a"],` with
186+
// nothing after it) is REJECTED. The splitter emits an empty token
187+
// for the missing key=value pair, which fails the `SplitN(part, "=", 2)`
188+
// length check with `invalid format of modeline part`. Pinned so a
189+
// future "helpfully ignore empty tokens" patch can't silently flip
190+
// rejection semantics.
191+
func TestContract_ParseModeline_TrailingCommaRejected(t *testing.T) {
192+
cases := []string{
193+
`# talm: nodes=["a"],`,
194+
`# talm: nodes=["a"], `,
195+
`# talm: nodes=["a"], endpoints=["b"],`,
196+
}
197+
for _, line := range cases {
198+
t.Run(line, func(t *testing.T) {
199+
_, err := ParseModeline(line)
200+
if err == nil {
201+
t.Errorf("expected rejection for trailing comma in %q, got nil", line)
202+
}
203+
})
204+
}
205+
}
206+
207+
// Contract: an unbalanced closing `]` does not panic the depth
208+
// counter. The splitter clamps depth at zero, so a stray `]` at
209+
// depth 0 stays at depth 0; subsequent commas are still treated as
210+
// key-pair separators. The resulting token shape fails downstream
211+
// JSON unmarshalling, producing a parse error rather than a panic.
212+
// A regression that drops the `if depth > 0` clamp would let depth
213+
// go negative — at which point an inner array's content would be
214+
// misclassified.
215+
func TestContract_ParseModeline_UnbalancedClosingBracketDoesNotPanic(t *testing.T) {
216+
// No assertion on specific error text; the contract is "rejects
217+
// without panicking". A nil error would mean the parser somehow
218+
// accepted clearly malformed JSON, which is a regression too.
219+
cases := []string{
220+
`# talm: nodes="a"]`,
221+
`# talm: nodes=]`,
222+
`# talm: nodes=["a"]], endpoints=["b"]`,
223+
}
224+
for _, line := range cases {
225+
t.Run(line, func(t *testing.T) {
226+
_, err := ParseModeline(line)
227+
if err == nil {
228+
t.Errorf("expected error for unbalanced bracket in %q, got nil", line)
229+
}
230+
})
116231
}
117232
}
118233

‎pkg/modeline/modeline.go‎

Lines changed: 64 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,69 @@ import (
1010
"github.com/cockroachdb/errors"
1111
)
1212

13+
// splitModelineParts splits the modeline body (everything after the
14+
// `# talm: ` prefix) into `key=value` tokens. Splits on `,` only at
15+
// JSON nesting depth 0, so a comma inside a `nodes=["a", "b"]` array
16+
// no longer cuts the value mid-stream. Tracks `[`/`]` depth and `"`
17+
// string state with JSON-style backslash escapes; a `,` or `]` inside
18+
// a JSON string literal is treated as data, not structure.
19+
//
20+
// This is more permissive than the old `, ` literal split — both the
21+
// canonical talm-generated form (no whitespace inside arrays) and the
22+
// human-written form (whitespace after each array element) now parse.
23+
// Whitespace AROUND tokens is trimmed by the caller's SplitN step.
24+
//
25+
// Scope: JSON-array values only. The splitter does NOT track `{`/`}`
26+
// nesting because every modeline key in the current contract (nodes,
27+
// endpoints, templates) is a JSON array — a `{` at depth 0 will fall
28+
// through to the downstream json.Unmarshal which rejects non-array
29+
// inputs. If a future modeline key takes a JSON-object value, extend
30+
// the depth counter to track `{`/`}` too.
31+
func splitModelineParts(content string) []string {
32+
parts := make([]string, 0)
33+
depth := 0
34+
inString := false
35+
escape := false
36+
start := 0
37+
38+
for i := range len(content) {
39+
c := content[i]
40+
41+
if inString {
42+
switch {
43+
case escape:
44+
escape = false
45+
case c == '\\':
46+
escape = true
47+
case c == '"':
48+
inString = false
49+
}
50+
51+
continue
52+
}
53+
54+
switch c {
55+
case '"':
56+
inString = true
57+
case '[':
58+
depth++
59+
case ']':
60+
if depth > 0 {
61+
depth--
62+
}
63+
case ',':
64+
if depth == 0 {
65+
parts = append(parts, content[start:i])
66+
start = i + 1
67+
}
68+
}
69+
}
70+
71+
parts = append(parts, content[start:])
72+
73+
return parts
74+
}
75+
1376
// Config structure for storing settings from modeline.
1477
type Config struct {
1578
Nodes []string
@@ -35,8 +98,7 @@ func ParseModeline(line string) (*Config, error) {
3598
if after, ok := strings.CutPrefix(trimLine, prefix); ok {
3699
content := after
37100

38-
parts := strings.SplitSeq(content, ", ")
39-
for part := range parts {
101+
for _, part := range splitModelineParts(content) {
40102
keyVal := strings.SplitN(strings.TrimSpace(part), "=", 2)
41103
if len(keyVal) != 2 {
42104
//nolint:wrapcheck // cockroachdb/errors.WithHintf is the project's wrapping/hinting idiom

‎pkg/modeline/modeline_test.go‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,55 @@ func TestParseModeline(t *testing.T) {
3131
},
3232
wantErr: false,
3333
},
34+
{
35+
// Human-written form of a shared side-patch modeline: each
36+
// element of the JSON array is followed by a single space.
37+
// The old `SplitSeq(content, ", ")` literal split treated
38+
// the inner space-comma as the key-pair separator and cut
39+
// the array value at the first element.
40+
name: "multi-element array with space after comma",
41+
line: `# talm: nodes=["1.2.3.4", "127.0.0.1", "192.168.100.2"]`,
42+
want: &Config{
43+
Nodes: []string{testNodeIP1, testLoopback, testNodeIP2},
44+
},
45+
wantErr: false,
46+
},
47+
{
48+
// Same payload, all three keys, mixed spacing — guards the
49+
// depth-0 boundary so a comma INSIDE an array never collides
50+
// with a comma BETWEEN key-pairs.
51+
name: "all keys multi-element mixed spacing",
52+
line: `# talm: nodes=["1.2.3.4", "127.0.0.1"], endpoints=["1.2.3.4","127.0.0.1"], templates=["templates/controlplane.yaml", "templates/worker.yaml"]`,
53+
want: &Config{
54+
Nodes: []string{testNodeIP1, testLoopback},
55+
Endpoints: []string{testNodeIP1, testLoopback},
56+
Templates: []string{testTemplateControlPln, "templates/worker.yaml"},
57+
},
58+
wantErr: false,
59+
},
60+
{
61+
// Comma inside a JSON string literal must NOT split. Rare in
62+
// practice (IPs and template paths have no commas) but the
63+
// splitter's depth/string tracking promises this.
64+
name: "comma inside string literal",
65+
line: `# talm: nodes=["a,b","c"]`,
66+
want: &Config{
67+
Nodes: []string{"a,b", "c"},
68+
},
69+
wantErr: false,
70+
},
71+
{
72+
// Key-pair separator without the trailing space: a stricter
73+
// form than canonical talm output but operators may hand-edit
74+
// to this shape. Now accepted (was previously rejected).
75+
name: "key-pair separator without trailing space",
76+
line: `# talm: nodes=["1.2.3.4"],endpoints=["127.0.0.1"]`,
77+
want: &Config{
78+
Nodes: []string{testNodeIP1},
79+
Endpoints: []string{testLoopback},
80+
},
81+
wantErr: false,
82+
},
3483
}
3584

3685
for _, tc := range testCases {

0 commit comments

Comments
 (0)