-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Evaluation fails because vitest expect destructured first argument on hooks.
FAIL axiom:eval myFile.eval.ts > eval-name
FixtureParseError: The 1st argument inside a fixture must use object destructuring pattern, e.g. ({ task } => {}). Instead, received "suite". If you used internal "suite" task as the 1st argument previously, access it in the 2nd argument instead.
It seems to be related to beforeAll and afterAll hooks on eval:
ai/packages/ai/src/evals/eval.ts
Lines 240 to 243 in 23fef3b
| beforeAll(async (suite) => { | |
| // Ensure worker process knows CLI overrides | |
| if (injectedOverrides && Object.keys(injectedOverrides).length > 0) { |
ai/packages/ai/src/evals/eval.ts
Lines 357 to 360 in 23fef3b
| afterAll(async (suite) => { | |
| if (instrumentationError) { | |
| throw instrumentationError; |
Upgrading to something like: afterAll(async ({}, suite) => { seems to fix the issue on my side
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels