Repository navigation
Commit 9cf617f
fix(evals): unloaded deferred calls are not graded; templates drop attachments
- Claude Code rejects a deferred tool that was not loaded, so that call never
ran. The grader no longer records it. Mocked tool errors still count.
- trim-capture keeps only text blocks of the first user turn, so user images
and documents never reach a template.
Co-Authored-By: Claude Opus 5.5 <[email protected]>1 parent 5cc594a commit 9cf617f
3 files changed
Lines changed: 28 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
| |||
287 | 288 | | |
288 | 289 | | |
289 | 290 | | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
290 | 305 | | |
291 | 306 | | |
292 | 307 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1404 | 1404 | | |
1405 | 1405 | | |
1406 | 1406 | | |
| 1407 | + | |
| 1408 | + | |
| 1409 | + | |
| 1410 | + | |
| 1411 | + | |
| 1412 | + | |
| 1413 | + | |
| 1414 | + | |
| 1415 | + | |
| 1416 | + | |
1407 | 1417 | | |
1408 | 1418 | | |
1409 | 1419 | | |
| |||
1438 | 1448 | | |
1439 | 1449 | | |
1440 | 1450 | | |
1441 | | - | |
| 1451 | + | |
1442 | 1452 | | |
1443 | 1453 | | |
1444 | 1454 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
87 | 87 | | |
88 | 88 | | |
89 | 89 | | |
| 90 | + | |
| 91 | + | |
90 | 92 | | |
91 | 93 | | |
92 | 94 | | |
| |||
0 commit comments