Repository navigation
Commit 0724de3
fix: [#849] address code-review findings in validator framework
All issues identified in the consensus review (Claude + 8 models) are
addressed in this commit.
**MAJOR fixes (blocking merge)**
- Add subprocess timeout (`ALTIMATE_VALIDATORS_TIMEOUT_MS`, default 60 s)
to `runDbtTest` and `runSchemaVerify` — prevents the agent loop from
hanging indefinitely on stalled DuckDB connections or warehouse I/O.
Kills the child process with SIGKILL on timeout.
- Fix shadow telemetry gate: move `ValidatorRegistry.runAll()` and the
per-validator `Telemetry.track()` loop outside the `validatorsEnabled`
check in `prompt.ts`. Telemetry now fires regardless of the feature
flag, fulfilling the "measure before enforce" promise stated in the
inline comment.
- Fix nested dbt project cwd bug: replace `isDbtProject(cwd): boolean`
with `findDbtProjectRoot(cwd): Promise<string | null>` which returns
the directory that actually contains `dbt_project.yml`. Both
validators now pass that root as `cwd` to subprocess invocations and
as the base for `modelsModifiedSince`, preventing the "not a dbt
project" error when the project is one level below the working dir.
- Extract shared helpers into `validator-utils.ts`: `findDbtProjectRoot`,
`modelsModifiedSince`, `modelNameFromPath`, `extractLastJsonObject`.
Both validator files now import from the shared module. The validated
`extractLastJsonObject` rejects stray JSON fragments (checks for
`verdict`/`error`/`model`/`stdout`/`columns_*` keys) — the laxer
version that was only in `dbt-tests-pass.ts` is gone.
- Add tests: 39 unit tests covering `extractLastJsonObject` (8 cases),
`modelNameFromPath`, `findDbtProjectRoot` (5 cases),
`modelsModifiedSince` (7 cases), and `parseDbtTestOutput` (10 cases
including dbt 1.x format, ANSI prefixes, NO-OP variant, duplicate
names, `[FAIL`/`[ERROR` token exclusion).
**MINOR fixes**
- Track spawn failures separately in `dbt-tests-pass.check()` and
`dbt-schema-verify.check()`. `details.spawn_failures` now appears in
the validator result so operators can distinguish "skipped model" from
"passed model".
- Add retries-exhausted telemetry: when `validatorRetryCount >=
maxValidatorRetries` with failures outstanding, emit
`validator_retries_exhausted` event and a `log.warn` so the session
doesn't silently appear as "completed" in the operator dashboard.
- Parallel model checking: both `check()` functions now use
`Promise.all` instead of a sequential `for` loop.
**NITS**
- Named regex groups in `parseDbtTestOutput` — replaces positional
`summaryMatch[1]` / `[3]` / `[5]` captures; resilient to dbt
reordering summary fields.
- Path separator: `modelsModifiedSince` and `modelNameFromPath` now use
`path.sep` / `path.basename` instead of hardcoded `"/"`.
- Fix `ls | head -1` in `build-local-tarball.sh`: derive exact tarball
name from `VERSION` variable; error out explicitly if not present.
- Fix stale comment "Limited to two-level deep search" in
`dbt-schema-verify.ts` (actual depth was 4; comment is removed).
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>1 parent 7ca5a36 commit 0724de3
7 files changed
Lines changed: 703 additions & 287 deletions
File tree
- benchmark/ade-bench/altimate_code_agent
- packages/opencode
- src
- altimate/validators
- session
- test/altimate/validators
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
81 | 81 | | |
82 | 82 | | |
83 | 83 | | |
84 | | - | |
85 | | - | |
86 | | - | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
87 | 87 | | |
88 | 88 | | |
89 | 89 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
18 | | - | |
19 | 17 | | |
20 | 18 | | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
21 | 26 | | |
22 | 27 | | |
23 | 28 | | |
| |||
29 | 34 | | |
30 | 35 | | |
31 | 36 | | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | | - | |
80 | | - | |
81 | | - | |
82 | | - | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
91 | | - | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | 37 | | |
105 | 38 | | |
106 | 39 | | |
107 | | - | |
108 | | - | |
109 | | - | |
110 | | - | |
111 | | - | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
112 | 43 | | |
113 | 44 | | |
114 | | - | |
115 | | - | |
116 | | - | |
117 | | - | |
118 | | - | |
119 | | - | |
120 | | - | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | | - | |
127 | | - | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
140 | | - | |
141 | | - | |
142 | | - | |
143 | | - | |
144 | | - | |
145 | | - | |
146 | | - | |
147 | | - | |
148 | | - | |
149 | | - | |
150 | | - | |
151 | | - | |
152 | | - | |
153 | | - | |
154 | | - | |
155 | | - | |
156 | | - | |
157 | | - | |
158 | | - | |
159 | | - | |
160 | | - | |
161 | | - | |
162 | | - | |
163 | | - | |
164 | | - | |
165 | | - | |
166 | | - | |
167 | | - | |
168 | | - | |
169 | | - | |
170 | | - | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
171 | 48 | | |
172 | 49 | | |
173 | 50 | | |
174 | 51 | | |
175 | | - | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
176 | 58 | | |
177 | 59 | | |
178 | 60 | | |
| |||
182 | 64 | | |
183 | 65 | | |
184 | 66 | | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
185 | 71 | | |
186 | 72 | | |
187 | 73 | | |
188 | 74 | | |
189 | 75 | | |
| 76 | + | |
190 | 77 | | |
191 | 78 | | |
192 | 79 | | |
| |||
197 | 84 | | |
198 | 85 | | |
199 | 86 | | |
| 87 | + | |
200 | 88 | | |
201 | 89 | | |
202 | 90 | | |
| |||
255 | 143 | | |
256 | 144 | | |
257 | 145 | | |
258 | | - | |
| 146 | + | |
259 | 147 | | |
260 | 148 | | |
261 | 149 | | |
262 | | - | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
263 | 154 | | |
264 | 155 | | |
265 | 156 | | |
266 | 157 | | |
267 | 158 | | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
268 | 164 | | |
269 | | - | |
270 | | - | |
271 | | - | |
272 | | - | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
273 | 173 | | |
274 | 174 | | |
275 | 175 | | |
| |||
286 | 186 | | |
287 | 187 | | |
288 | 188 | | |
| 189 | + | |
289 | 190 | | |
290 | 191 | | |
291 | 192 | | |
| |||
303 | 204 | | |
304 | 205 | | |
305 | 206 | | |
| 207 | + | |
306 | 208 | | |
307 | 209 | | |
308 | 210 | | |
| |||
0 commit comments