CHANGELOG.md (1)

3-4: Nit: Link the PR and use consistent prefix casing

Make this entry consistent with others and link the PR.

-- **fix** improve parsing for custom benchmarks (#323)
+- **fix:** improve parsing for custom benchmarks ([#323](https://github.com/benchmark-action/github-action-benchmark/pull/323))

test/data/extract/customSmallerIsBetter_output2.json (1)

1-26: Add a coercion-focused fixture (stringified numbers) to exercise z.coerce paths

Current fixture has numeric value/range. Add a case where value is a string and range is numeric to prove coercion.

Example new file: test/data/extract/customSmallerIsBetter_output3.json

[
  { "name": "No instrumentation", "unit": "ns", "value": "90.9439", "range": 0.519165 },
  { "name": "Deactivated probe",  "unit": "ns", "value": "445.661",  "range": 8.21371  }
]

And include it in test/extract.spec.ts normalCases.

src/extract.ts (2)

7-13: Tighten schema and avoid type/value name collision

• Ensure non-empty strings and finite numbers.
• Avoid over-coercion turning objects into "[object Object]".
• Rename runtime schema to avoid confusion with the type alias.

-export const BenchmarkResult = z.object({
-    name: z.coerce.string(),
-    value: z.coerce.number(),
-    range: z.coerce.string().optional(),
-    unit: z.coerce.string(),
-    extra: z.coerce.string().optional(),
-});
-
-export type BenchmarkResult = z.infer<typeof BenchmarkResult>;
-
-export const BenchmarkResults = z.array(BenchmarkResult);
+export const BenchmarkResultSchema = z
+    .object({
+        name: z.string().min(1).transform((s) => s.trim()),
+        value: z.coerce.number().finite(),
+        // Accept number|string|null; avoid coercing objects/arrays to "[object Object]"
+        range: z
+            .union([z.string(), z.number(), z.null()])
+            .optional()
+            .transform((v) => (v == null ? undefined : String(v))),
+        unit: z.string().min(1).transform((s) => s.trim()),
+        extra: z
+            .union([z.string(), z.number()])
+            .optional()
+            .transform((v) => (typeof v === 'number' ? String(v) : v?.trim())),
+    })
+    .strip();
+
+export type BenchmarkResult = z.infer<typeof BenchmarkResultSchema>;
+
+export const BenchmarkResults = z.array(BenchmarkResultSchema);

Also applies to: 15-17

---

664-671: Surface Zod issues (paths/messages) for actionable errors

Report first few schema errors with paths; keep JSON.parse errors as-is.

-function extractCustomBenchmarkResult(output: string): BenchmarkResult[] {
-    try {
-        return BenchmarkResults.parse(JSON.parse(output));
-    } catch (err: unknown) {
-        const errMessage = err instanceof Error ? err.message : String(err);
-        throw new Error(
-            `Output file for 'custom-(bigger|smaller)-is-better' must be JSON file containing an array of entries in BenchmarkResult format: ${errMessage}`,
-        );
-    }
-}
+function extractCustomBenchmarkResult(output: string): BenchmarkResult[] {
+    try {
+        const parsed = JSON.parse(output);
+        const res = BenchmarkResults.safeParse(parsed);
+        if (!res.success) {
+            const details = res.error.issues
+                .slice(0, 3)
+                .map((i) => `${i.path.join('.') || '(root)'}: ${i.message}`)
+                .join('; ');
+            throw new Error(details);
+        }
+        return res.data;
+    } catch (err: unknown) {
+        const errMessage = err instanceof Error ? err.message : String(err);
+        throw new Error(
+            `Output file for 'custom-(bigger|smaller)-is-better' must be JSON file containing an array of entries in BenchmarkResult format: ${errMessage}`,
+        );
+    }
+}

test/extract.spec.ts (1)

152-166: Add targeted tests for Zod coercion and invalid-shape errors (beyond snapshot).

To exercise the new schema behavior, consider:

• a coercion case (stringified numeric values),
• and an invalid-shape case (missing required fields), asserting rejection.

TypeScript tests to add:

it('coerces custom JSON fields via Zod (value as string)', async () => {
  const outputFilePath = path.join(__dirname, 'data', 'extract', 'customSmallerIsBetter_output_coercion.json');
  const config = { tool: 'customSmallerIsBetter', outputFilePath } as Config;
  const bench = await extractResult(config);
  expect(bench).toMatchSnapshot();
});

it('fails with a clear error for invalid custom JSON shape', async () => {
  const outputFilePath = path.join(__dirname, 'data', 'extract', 'customSmallerIsBetter_output_invalid.json');
  const config = { tool: 'customSmallerIsBetter', outputFilePath } as Config;
  await expect(extractResult(config)).rejects.toThrow();
});

Fixture files to add:

test/data/extract/customSmallerIsBetter_output_coercion.json

[
  { "name": "No instrumentation", "unit": "ns", "value": "101.5", "range": "R1" },
  { "name": "Deactivated probe", "unit": "ns", "value": "98.2" }
]

test/data/extract/customSmallerIsBetter_output_invalid.json

[
  { "name": "Missing value", "unit": "ns" }
]

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

package.json (1)

47-47: Align @types/node version with Node20 runtime
action.yml declares using: 'node20', so downgrade @types/node to the matching Node.js 20 types to avoid accidental use of newer APIs.

-    "@types/node": "^24.3.1",
+    "@types/node": "^20.12.12",

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro