✅ Clarify arbitrary benchmark names by dubzzz · Pull Request #7064 · dubzzz/fast-check

dubzzz · 2026-06-09T21:55:08Z

Description

This PR refines the names (and a few of the inputs) of the arbitrary generation benchmarks in packages/fast-check/test/bench/arbitraries.bench.ts so the benchmark report is self-explanatory. There is no change to shipped library code — this only touches the benchmark suite, so end users see no behavioral difference; the benefit is for anyone reading benchmark output or adding new cases.

Changes:

constantFrom(...) → constantFrom(1, 2, 3), with the case now using fc.constantFrom(1, 2, 3) instead of the opaque 'a'..'e' so the name matches what is benchmarked.
Added a dedicated constant(1) case (fc.constant(1)), which was previously missing, placed right before constantFrom.
integer().filter(.) → integer().filter(true), using a trivially-truthy predicate () => true so the benchmark measures the filter wrapper overhead rather than a rejection-heavy predicate.
integer().map(.) → integer().map(value*2), with the mapper changed to (n) => n * 2 so the name reflects the operation.
integer().chain(.) → integer().chain(integer()), making it explicit that the chain produces another fc.integer().

Why

The previous names (constantFrom(...), .map(.), .chain(.), .filter(.)) were placeholders that didn't convey what each case exercised, which makes benchmark diffs harder to interpret. The new names spell out the inputs/operations, and the filter(true) predicate avoids letting rejection rate distort the measurement of the operator overhead itself.

Scope & impact

Patch-level, benchmark-only, single concern. No production code is affected and no new public behavior is introduced. These are the bench definitions themselves, so they act as their own "tests"; the suite was type-checked to confirm the changes compile.

Checklist

— Don't delete this checklist and make sure you do the following before opening the PR

I have a full understanding of every line in this PR — whether the code was hand-written, AI-generated, copied from external sources or produced by any other tool
I flagged the impact of my change (minor / patch / major) either by running pnpm run bump or by following the instructions from the changeset bot
I kept this PR focused on a single concern and did not bundle unrelated changes
I followed the gitmoji specification for the name of the PR, including the package scope (e.g. 🐛(vitest) Something...) when the change targets a package other than fast-check
I added relevant tests and they would have failed without my PR (when applicable)

Generated by Claude Code

changeset-bot · 2026-06-09T21:55:20Z

⚠️ No Changeset found

Latest commit: f2e570c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

pkg-pr-new · 2026-06-09T21:57:38Z

@fast-check/examples

@fast-check/ava

npm i https://pkg.pr.new/@fast-check/ava@7064

fast-check

npm i https://pkg.pr.new/fast-check@7064

@fast-check/jest

npm i https://pkg.pr.new/@fast-check/jest@7064

@fast-check/packaged

npm i https://pkg.pr.new/@fast-check/packaged@7064

@fast-check/poisoning

npm i https://pkg.pr.new/@fast-check/poisoning@7064

@fast-check/vitest

npm i https://pkg.pr.new/@fast-check/vitest@7064

@fast-check/worker

npm i https://pkg.pr.new/@fast-check/worker@7064

commit: f2e570c

github-actions · 2026-06-09T21:58:35Z

⏱️ Benchmark Results

Click to expand


 RUN  v4.1.5 /home/runner/work/fast-check/fast-check


 ✓  fast-check  test/bench/arbitraries.bench.ts > generate 51007ms
     name                                                          hz     min      max    mean     p75     p99    p995     p999      rme  samples
   · boolean()                                           6,253,307.27  0.0001   0.4437  0.0002  0.0002  0.0002  0.0003   0.0006   ±0.48%  3126749  [0.88x] ⇓
     boolean()                                           7,098,732.32  0.0001   0.5614  0.0001  0.0001  0.0002  0.0002   0.0005   ±0.47%  3549367  (baseline)
   · integer()                                           6,966,603.96  0.0001   0.2552  0.0001  0.0002  0.0002  0.0003   0.0003   ±0.16%  3483302  [1.01x] ⇑
     integer()                                           6,914,044.06  0.0001   0.0901  0.0001  0.0002  0.0003  0.0003   0.0003   ±0.19%  3457068  (baseline)
   · maxSafeInteger()                                    5,521,092.20  0.0001   0.2363  0.0002  0.0002  0.0002  0.0003   0.0004   ±0.18%  2760547  [1.04x] ⇑
     maxSafeInteger()                                    5,284,353.50  0.0001   0.3858  0.0002  0.0002  0.0003  0.0003   0.0004   ±0.24%  2642177  (baseline)
   · bigInt()                                            1,195,617.78  0.0006   0.4270  0.0008  0.0012  0.0013  0.0017   0.0044   ±0.38%   597809  [1.00x] ⇑
     bigInt()                                            1,192,711.57  0.0006   0.4030  0.0008  0.0012  0.0013  0.0017   0.0039   ±0.36%   596356  (baseline)
   · float()                                             4,351,833.98  0.0002   0.2564  0.0002  0.0003  0.0003  0.0004   0.0007   ±0.22%  2175918  [0.99x] ⇓
     float()                                             4,397,269.07  0.0002   0.5351  0.0002  0.0002  0.0003  0.0004   0.0008   ±0.36%  2198635  (baseline)
   · double()                                            1,690,665.20  0.0004   0.4557  0.0006  0.0007  0.0009  0.0011   0.0019   ±0.39%   845333  [1.00x] ⇑
     double()                                            1,690,259.96  0.0004   0.4585  0.0006  0.0007  0.0009  0.0011   0.0019   ±0.41%   845130  (baseline)
   · date()                                              3,042,439.06  0.0003   0.5233  0.0003  0.0003  0.0004  0.0005   0.0012   ±0.34%  1521220  [1.00x] ⇓
     date()                                              3,049,221.79  0.0003   0.2837  0.0003  0.0003  0.0004  0.0005   0.0011   ±0.27%  1524611  (baseline)
   · string()                                            1,007,803.16  0.0002   0.6536  0.0010  0.0012  0.0022  0.0033   0.0115   ±0.50%   503902  [1.00x] ⇓
     string()                                            1,011,909.94  0.0003   0.3897  0.0010  0.0012  0.0022  0.0032   0.0114   ±0.46%   505955  (baseline)
   · string({ maxLength: 100, size: 'max' })               182,640.17  0.0003   0.4179  0.0055  0.0079  0.0158  0.0170   0.0252   ±0.63%    91322  [1.01x] ⇑
     string({ maxLength: 100, size: 'max' })               181,188.29  0.0003   0.4839  0.0055  0.0080  0.0160  0.0175   0.0257   ±0.65%    90595  (baseline)
   · string({ unit: 'grapheme' })                          661,723.31  0.0002   0.5118  0.0015  0.0020  0.0035  0.0041   0.0132   ±0.44%   330862  [1.00x] ⇓
     string({ unit: 'grapheme' })                          661,895.85  0.0002   0.5175  0.0015  0.0020  0.0036  0.0041   0.0128   ±0.49%   330949  (baseline)
   · base64String()                                        969,817.12  0.0003   0.4652  0.0010  0.0012  0.0022  0.0033   0.0118   ±0.46%   484909  [1.01x] ⇑
     base64String()                                        963,242.73  0.0003   0.5052  0.0010  0.0013  0.0022  0.0033   0.0115   ±0.50%   481622  (baseline)
   · array(integer())                                    1,689,837.76  0.0002   0.5212  0.0006  0.0006  0.0015  0.0020   0.0039   ±0.45%   844919  [1.00x] ⇓
     array(integer())                                    1,695,151.02  0.0002   0.5312  0.0006  0.0006  0.0014  0.0019   0.0039   ±0.52%   847577  (baseline)
   · array(integer(), { maxLength: 100, size: 'max' })     319,681.19  0.0002  23.7258  0.0031  0.0039  0.0108  0.0141   0.0214   ±9.33%   159841  [1.00x] ⇑
     array(integer(), { maxLength: 100, size: 'max' })     318,375.05  0.0002  23.6170  0.0031  0.0039  0.0108  0.0141   0.0213   ±9.28%   159189  (baseline)
   · uniqueArray(integer())                              1,260,496.23  0.0002   0.4114  0.0008  0.0010  0.0019  0.0020   0.0041   ±0.38%   630249  [1.00x] ⇑
     uniqueArray(integer())                              1,257,861.38  0.0002   0.4600  0.0008  0.0010  0.0019  0.0020   0.0038   ±0.41%   628932  (baseline)
   · set(integer())                                        998,227.92  0.0003   0.4496  0.0010  0.0012  0.0023  0.0024   0.0115   ±0.42%   499114  [1.00x] ⇓
     set(integer())                                        999,071.28  0.0003   0.4830  0.0010  0.0012  0.0023  0.0025   0.0115   ±0.44%   499536  (baseline)
   · tuple(integer())                                    3,860,038.00  0.0002   0.4592  0.0003  0.0003  0.0004  0.0004   0.0011   ±0.50%  1930020  [1.00x] ⇑
     tuple(integer())                                    3,847,504.64  0.0002   0.4312  0.0003  0.0003  0.0004  0.0004   0.0012   ±0.46%  1923753  (baseline)
   · record({ a: integer() })                            1,860,027.30  0.0004   0.4540  0.0005  0.0006  0.0007  0.0011   0.0016   ±0.48%   930014  [1.01x] ⇑
     record({ a: integer() })                            1,849,573.05  0.0004   0.4587  0.0005  0.0006  0.0008  0.0011   0.0016   ±0.46%   924787  (baseline)
   · dictionary(string(), integer())                       114,186.11  0.0005   4.4370  0.0088  0.0126  0.0221  0.0244   0.0321   ±2.51%    57094  [1.00x] ⇓
     dictionary(string(), integer())                       114,247.68  0.0005   4.5308  0.0088  0.0125  0.0219  0.0244   0.0322   ±2.58%    57124  (baseline)
   · map(string(), integer())                              174,688.31  0.0003   0.4383  0.0057  0.0086  0.0144  0.0173   0.0244   ±0.68%    87345  [0.99x] ⇓
     map(string(), integer())                              176,265.62  0.0003   0.4530  0.0057  0.0085  0.0143  0.0171   0.0238   ±0.64%    88133  (baseline)
   · constant(1)                                         9,927,944.37  0.0001   0.5666  0.0001  0.0001  0.0001  0.0002   0.0002   ±0.26%  4963973  [1.28x] ⇑
     constantFrom(...)                                   7,754,702.76  0.0001   0.6100  0.0001  0.0001  0.0002  0.0002   0.0002   ±0.29%  3877352  (baseline)
   · constantFrom(1, 2)                                  6,946,600.33  0.0001   0.4860  0.0001  0.0002  0.0002  0.0002   0.0003   ±0.22%  3473301  [1.73x] ⇑
     oneof(integer(), integer())                         4,013,872.53  0.0002   0.0298  0.0002  0.0003  0.0004  0.0004   0.0009   ±0.11%  2006937  (baseline)
   · oneof(integer(), integer())                         3,978,121.55  0.0002   0.0552  0.0003  0.0003  0.0004  0.0004   0.0009   ±0.12%  1989061  [0.97x] ⇓
     oneof({ weight, arbitrary }, ...)                   4,106,574.88  0.0002   0.0398  0.0002  0.0003  0.0004  0.0004   0.0008   ±0.11%  2053288  (baseline)
   · oneof({ weight, arbitrary }, ...)                   3,885,472.13  0.0002  12.3055  0.0003  0.0003  0.0004  0.0005   0.0010   ±4.83%  1942737  [0.93x] ⇓
     option(integer())                                   4,175,440.86  0.0001   0.5353  0.0002  0.0003  0.0003  0.0004   0.0011   ±0.38%  2087721  (baseline)
   · option(integer())                                   4,146,950.92  0.0002   0.2935  0.0002  0.0003  0.0003  0.0004   0.0010   ±0.30%  2073476  [2.13x] ⇑
     subarray([1, 2, 3, 4, 5])                           1,945,603.85  0.0002   0.4600  0.0005  0.0007  0.0010  0.0012   0.0019   ±0.52%   972802  (baseline)
   · subarray([1, 2, 3, 4, 5])                           1,958,256.95  0.0002   0.5581  0.0005  0.0007  0.0010  0.0012   0.0019   ±0.52%   979129  [2.43x] ⇑
     letrec(tree)                                          807,217.32  0.0002   0.5818  0.0012  0.0018  0.0052  0.0059   0.0130   ±0.61%   403611  (baseline)
   · letrec(tree)                                          808,809.25  0.0002   0.4913  0.0012  0.0018  0.0052  0.0059   0.0129   ±0.57%   404405  [0.97x] ⇓
     memo(tree)                                            833,695.64  0.0002   0.5110  0.0012  0.0021  0.0040  0.0043   0.0128   ±0.58%   416848  (baseline)
   · memo(tree)                                            740,911.85  0.0002  15.3406  0.0013  0.0021  0.0058  0.0086   0.0205   ±6.04%   370456  [9.02x] ⇑
     anything()                                             82,147.31  0.0003   2.4876  0.0122  0.0201  0.0647  0.0749   0.1160   ±2.20%    41075  (baseline)
   · anything()                                             81,548.93  0.0003   2.4564  0.0123  0.0204  0.0620  0.0726   0.0968   ±2.21%    40775  [1.06x] ⇑
     json()                                                 76,575.04  0.0003   4.7395  0.0131  0.0217  0.0655  0.0744   0.0967   ±2.63%    38288  (baseline)
   · json()                                                 76,155.90  0.0003   4.8367  0.0131  0.0216  0.0662  0.0770   0.0997   ±2.70%    38078  [10.83x] ⇑
     entityGraph(node -> node)                               7,032.99  0.0076  28.2521  0.1422  0.1247  0.3285  0.4089  10.7980  ±16.93%     3517  (baseline)
   · entityGraph(node -> node)                               8,084.34  0.0077  11.1405  0.1237  0.1150  0.2163  0.2470   9.1481  ±12.84%     4044  [0.09x] ⇓
     emailAddress()                                         94,666.60  0.0023   0.4368  0.0106  0.0135  0.0211  0.0255   0.0330   ±0.63%    47334  (baseline)
   · emailAddress()                                         92,213.77  0.0023   0.6731  0.0108  0.0137  0.0237  0.0283   0.0639   ±0.88%    46107  [0.52x] ⇓
     webUrl()                                              176,127.60  0.0022   0.4378  0.0057  0.0066  0.0104  0.0166   0.0254   ±0.58%    88064  (baseline)
   · webUrl()                                              174,028.14  0.0023   0.6793  0.0057  0.0066  0.0106  0.0162   0.0225   ±0.81%    87015  [0.13x] ⇓
     ipV4()                                              1,301,897.76  0.0006   0.5868  0.0008  0.0008  0.0011  0.0013   0.0022   ±0.32%   650949  (baseline)
   · ipV4()                                              1,312,529.81  0.0006   0.6545  0.0008  0.0008  0.0010  0.0012   0.0021   ±0.44%   656265  [5.37x] ⇑
     ipV6()                                                244,641.70  0.0005   0.5479  0.0041  0.0049  0.0071  0.0108   0.0198   ±0.59%   122321  (baseline)
   · ipV6()                                                229,184.31  0.0004  15.5507  0.0044  0.0049  0.0131  0.0155   0.0216   ±6.12%   114593  [0.33x] ⇓
     uuid()                                                701,612.79  0.0010   0.5242  0.0014  0.0014  0.0025  0.0034   0.0139   ±0.46%   350807  (baseline)
   · uuid()                                                723,219.49  0.0010   0.5213  0.0014  0.0014  0.0023  0.0026   0.0125   ±0.47%   361610  [0.07x] ⇓
     stringMatching(/^abc$/)                            10,731,079.61  0.0001   0.3008  0.0001  0.0001  0.0001  0.0001   0.0002   ±0.16%  5365540  (baseline)
   · stringMatching(/^abc$/)                             9,931,047.50  0.0001   0.3954  0.0001  0.0001  0.0001  0.0001   0.0002   ±0.19%  4965524  [16.66x] ⇑
     stringMatching(/^[a-zA-Z0-9]+$/)                      596,134.43  0.0005   0.5500  0.0017  0.0022  0.0034  0.0036   0.0133   ±0.57%   298069  (baseline)
   · stringMatching(/^[a-zA-Z0-9]+$/)                      567,638.06  0.0005  13.4642  0.0018  0.0022  0.0043  0.0055   0.0135   ±5.30%   283820  [1.04x] ⇑
     mixedCase(string())                                   546,191.38  0.0005   0.5148  0.0018  0.0023  0.0034  0.0037   0.0133   ±0.55%   273096  (baseline)
   · mixedCase(string())                                   540,665.43  0.0005   0.4410  0.0018  0.0023  0.0034  0.0037   0.0135   ±0.49%   270333  [0.11x] ⇓
     integer().map(.)                                    5,092,737.01  0.0001   0.4028  0.0002  0.0002  0.0003  0.0003   0.0007   ±0.55%  2546369  (baseline)
   · integer().map(value+1)                              5,041,621.86  0.0001   0.3523  0.0002  0.0002  0.0003  0.0003   0.0006   ±0.48%  2520811  [1.70x] ⇑
     integer().chain(.)                                  2,964,012.09  0.0002   0.4367  0.0003  0.0004  0.0005  0.0006   0.0013   ±0.60%  1482007  (baseline)
   · integer().chain(integer())                          2,912,249.79  0.0002   0.5004  0.0003  0.0004  0.0005  0.0006   0.0013   ±0.54%  1456126  [0.95x] ⇓
     integer().filter(.)                                 3,063,365.74  0.0001   0.6725  0.0003  0.0004  0.0010  0.0011   0.0015   ±0.30%  1531683  (baseline)
   · integer().filter(true)                              5,592,871.67  0.0001   0.6208  0.0002  0.0002  0.0003  0.0003   0.0005   ±0.27%  2796437

 ✓  fast-check  test/bench/runners.bench.ts > runner 2429ms
     name                                                     hz     min     max    mean     p75     p99    p995    p999     rme  samples
   · assert on sync predicate not returning anything   12,611.95  0.0763  1.7498  0.0793  0.0772  0.1414  0.1673  0.1976  ±0.73%     6306  [1.00x] ⇑
     assert on sync predicate not returning anything   12,604.12  0.0763  0.3331  0.0793  0.0773  0.1483  0.1732  0.2195  ±0.38%     6303  (baseline)
   · assert on sync predicate returning true           12,638.96  0.0760  0.3430  0.0791  0.0770  0.1530  0.1684  0.2436  ±0.40%     6320  [1.01x] ⇑
     assert on sync predicate returning true           12,564.79  0.0760  0.3347  0.0796  0.0772  0.1617  0.1743  0.2387  ±0.43%     6283  (baseline)
   · assert on async predicate not returning anything   8,623.77  0.1100  0.3155  0.1160  0.1127  0.1917  0.1965  0.2202  ±0.38%     4312  [1.01x] ⇑
     assert on async predicate not returning anything   8,524.12  0.1105  0.3118  0.1173  0.1142  0.1933  0.2018  0.2196  ±0.39%     4263  (baseline)
   · check on sync predicate not returning anything    12,665.32  0.0761  0.3382  0.0790  0.0772  0.1408  0.1677  0.2236  ±0.37%     6333  [1.00x] ⇑
     check on sync predicate not returning anything    12,651.87  0.0762  0.3536  0.0790  0.0773  0.1417  0.1694  0.2245  ±0.38%     6326  (baseline)

 BENCH  Summary

   fast-check  stringMatching(/^abc$/) - test/bench/arbitraries.bench.ts > generate
    1.00x faster than constant(1)
    1.43x faster than integer()
    1.43x faster than constantFrom(1, 2)
    1.59x faster than boolean()
    1.78x faster than integer().filter(true)
    1.80x faster than maxSafeInteger()
    1.97x faster than integer().map(value+1)
    2.28x faster than float()
    2.39x faster than option(integer())
    2.50x faster than oneof(integer(), integer())
    2.56x faster than oneof({ weight, arbitrary }, ...)
    2.57x faster than tuple(integer())
    3.26x faster than date()
    3.41x faster than integer().chain(integer())
    5.07x faster than subarray([1, 2, 3, 4, 5])
    5.34x faster than record({ a: integer() })
    5.87x faster than double()
    5.88x faster than array(integer())
    7.57x faster than ipV4()
    7.88x faster than uniqueArray(integer())
    8.31x faster than bigInt()
    9.85x faster than string()
    9.95x faster than set(integer())
    10.24x faster than base64String()
    12.28x faster than letrec(tree)
    13.40x faster than memo(tree)
    13.73x faster than uuid()
    15.01x faster than string({ unit: 'grapheme' })
    17.50x faster than stringMatching(/^[a-zA-Z0-9]+$/)
    18.37x faster than mixedCase(string())
    31.07x faster than array(integer(), { maxLength: 100, size: 'max' })
    43.33x faster than ipV6()
    54.37x faster than string({ maxLength: 100, size: 'max' })
    56.85x faster than map(string(), integer())
    57.07x faster than webUrl()
    86.97x faster than dictionary(string(), integer())
    107.70x faster than emailAddress()
    121.78x faster than anything()
    130.40x faster than json()
    1228.43x faster than entityGraph(node -> node)

   fast-check  check on sync predicate not returning anything - test/bench/runners.bench.ts > runner
    1.00x faster than assert on sync predicate returning true
    1.00x faster than assert on sync predicate not returning anything
    1.47x faster than assert on async predicate not returning anything

Benchmark report written to /home/runner/work/fast-check/fast-check/benchmark.json

Commit: f2e570c (merge: fc781ce)

test: refine arbitrary benchmark names for clarity

5f6663b

test: use map(value+1) and constantFrom(1, 2) in bench names

f2e570c

dubzzz enabled auto-merge (squash) June 9, 2026 21:58

dubzzz merged commit 010e19a into main Jun 9, 2026
56 of 57 checks passed

dubzzz deleted the claude/arb-benchmark-refinements-67ej7h branch June 9, 2026 22:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

✅ Clarify arbitrary benchmark names#7064

✅ Clarify arbitrary benchmark names#7064
dubzzz merged 2 commits into
mainfrom
claude/arb-benchmark-refinements-67ej7h

dubzzz commented Jun 9, 2026 •

edited

Loading

Uh oh!

changeset-bot Bot commented Jun 9, 2026 •

edited

Loading

Uh oh!

pkg-pr-new Bot commented Jun 9, 2026

Uh oh!

github-actions Bot commented Jun 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

dubzzz commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Why

Scope & impact

Checklist

Uh oh!

changeset-bot Bot commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

pkg-pr-new Bot commented Jun 9, 2026

Uh oh!

github-actions Bot commented Jun 9, 2026

⏱️ Benchmark Results

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dubzzz commented Jun 9, 2026 •

edited

Loading

changeset-bot Bot commented Jun 9, 2026 •

edited

Loading