Repository navigation
Commit 352428d
test: characterize replay-test reporter contract and extend R10 (#1505)
* test: characterize replay-test reporter contract and extend R10
P3 of #1478 moves the replay-test scheduler into `packages/replay-test` and
makes attempt identity scheduler-owned. Before production code moves, pin what
a shipped custom reporter actually observes today, and close the import
boundary the extraction has to end up satisfying.
Reporter values (all pinned as shipped, none proposed):
- the `RequestProgressEvent` -> reporter-value projection field by field,
including key presence for absent optionals and the dropped `command` events;
- `session` provenance across the seam: the start value is always `attempt-1`
(it is built before any attempt runs), step values track the running attempt
through the per-attempt AsyncLocalStorage context, and result values carry the
attempt that produced them, so a retried case reports three different
sessions to one reporter;
- the shard-scoped session prefix and device identity a sharded run reports;
- module export spelling precedence, the six optional hook names, the live-hook
vs final-hook error asymmetry, and exit-code recommendation semantics.
Late-timeout finalization/cleanup:
- finalization always runs before cleanup, and the timing trace records that
order (`finalize_start/stop` then `cleanup_start/stop`);
- a replay settling inside the 2s grace window cleans up once and is not marked
`timeout_cleanup_pending`, and the raced TIMEOUT response still wins;
- a replay that misses the window defers its second cleanup until the abandoned
replay settles, and that late cleanup's failure is swallowed.
R10 now rejects replay-test imports from `src/request/**` and engine internals
(`src/replay/`, `src/compat/`, `src/maestro/`, `src/ad-replay/`), with the four
imports that exist today recorded as shrink-only migration entries so the rule
enforces immediately and the extraction must delete them.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8
* test(daemon): assert reporter hint on the real onTestResult value
The failing-suite characterization supplied `hint` as input but never
asserted it on the hook value, so dropping `hint: error.hint` from the
scheduler path left all reporter tests green. Assert it on the real
reporter value so the ratchet catches a shipped field going missing.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01RXQLYV7etZx3gcXsUsrQJ8
---------
Co-authored-by: Claude <[email protected]>1 parent 9b5035f commit 352428d
7 files changed
Lines changed: 980 additions & 12 deletions
File tree
- scripts/layering
- src
- daemon/handlers/__tests__
- replay/test/reporters/__tests__
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
6 | 7 | | |
7 | 8 | | |
8 | 9 | | |
| |||
27 | 28 | | |
28 | 29 | | |
29 | 30 | | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
30 | 45 | | |
31 | 46 | | |
32 | 47 | | |
| |||
49 | 64 | | |
50 | 65 | | |
51 | 66 | | |
52 | | - | |
| 67 | + | |
53 | 68 | | |
54 | 69 | | |
55 | 70 | | |
56 | | - | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
57 | 75 | | |
58 | 76 | | |
59 | 77 | | |
| |||
66 | 84 | | |
67 | 85 | | |
68 | 86 | | |
69 | | - | |
| 87 | + | |
70 | 88 | | |
71 | 89 | | |
72 | 90 | | |
73 | 91 | | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
74 | 148 | | |
75 | 149 | | |
76 | 150 | | |
| |||
79 | 153 | | |
80 | 154 | | |
81 | 155 | | |
82 | | - | |
| 156 | + | |
83 | 157 | | |
84 | 158 | | |
85 | 159 | | |
| |||
89 | 163 | | |
90 | 164 | | |
91 | 165 | | |
92 | | - | |
| 166 | + | |
93 | 167 | | |
94 | 168 | | |
95 | 169 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
39 | 47 | | |
40 | 48 | | |
41 | 49 | | |
| |||
61 | 69 | | |
62 | 70 | | |
63 | 71 | | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
64 | 76 | | |
65 | 77 | | |
66 | | - | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
67 | 94 | | |
68 | 95 | | |
69 | 96 | | |
| |||
185 | 212 | | |
186 | 213 | | |
187 | 214 | | |
| 215 | + | |
188 | 216 | | |
189 | 217 | | |
190 | 218 | | |
| |||
203 | 231 | | |
204 | 232 | | |
205 | 233 | | |
206 | | - | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
207 | 258 | | |
208 | 259 | | |
209 | | - | |
210 | | - | |
211 | | - | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
212 | 263 | | |
213 | | - | |
214 | 264 | | |
215 | 265 | | |
216 | 266 | | |
| |||
237 | 287 | | |
238 | 288 | | |
239 | 289 | | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
240 | 294 | | |
241 | 295 | | |
242 | 296 | | |
243 | 297 | | |
244 | | - | |
| 298 | + | |
245 | 299 | | |
246 | 300 | | |
0 commit comments