Skip to content

[EPIC] Add SQL based benchmarking #21706

New issue
New issue
Open
Feature
Open
[EPIC] Add SQL based benchmarking#21706
Feature
Assignees
Omega359
Labels
EPICA larger project, actively underway, with sub tasksenhancementNew feature or request

Description

@Omega359
Omega359
opened on Apr 17, 2026

Is your feature request related to a problem or challenge?

As mentioned in #21165 having sql based benchmarking would make creating some types of benchmarks much easier and would remove the necessity of writing rust code just to benchmark some sql. It could also allow us to migrate away from manual benchmarking towards using criterion or Divan for running the benchmarks with proper statistics.

Describe the solution you'd like

I would suggest that we should take inspiration from DuckDB's benchmarking tool and create a similar solution for Datafusion.

Describe alternatives you've considered

Leave things as they are.

Additional context

Tasks

Activity

  1. Omega359
    added
    enhancementNew feature or request
    on Apr 17, 2026
  2. Omega359
    added the
    Feature
    issue type on Apr 17, 2026

  3. Omega359 commented on Apr 17, 2026

    @Omega359
    Omega359
    on Apr 17, 2026
    ContributorAuthor

    take

  4. github-actions
    assigned
    Omega359
    on Apr 17, 2026

  5. alamb commented on Apr 29, 2026

    @alamb
    alamb
    on Apr 29, 2026
    Contributor

    I started adding a subtasks to this ticket so we (well really I) don't forget them

  6. alamb
    changed the title [-]feat: Add SQL based benchmarking[/-] [+][EPIC] Add SQL based benchmarking[/+] on Apr 29, 2026
  7. alamb
    added
    EPICA larger project, actively underway, with sub tasks
    on Apr 29, 2026

  8. alamb commented on Apr 29, 2026

    @alamb
    alamb
    on Apr 29, 2026
    Contributor

    I filed a few more tickets

    And relabeled this thing as an EPIC 🚀

  9. philippemnoel
    added a commit that references this issue on Apr 29, 2026

    Add SQL based benchmarking harness, port tpch to use framework (apach…

    2b95cde
  10. yongchul
    added 3 commits that reference this issue on Jun 5, 2026

    Add clickbench SQL benchmark (apache#22633)

    11caa4c

    Add imdb SQL benchmark (apache#22680)

    5073db1

    Add h2o SQL benchmark (apache#22660)

    0d3fb77
  11. martin-augment
    added 2 commits that reference this issue on Jun 9, 2026

    Add tpcds SQL benchmark (apache#22801)

    33eba61

    Add hj SQL benchmark (apache#22802)

    8b6917c
  12. zhuqi-lucas
    added a commit that references this issue on Jun 9, 2026

    add clickbench sorted SQL benchmark (apache#22807)

    8ae94dd

  13. 5 remaining items

  14. AdamGS
    added 6 commits that reference this issue on Jun 11, 2026

    Add hj SQL benchmark (apache#22802)

    7330105

    add clickbench sorted SQL benchmark (apache#22807)

    66347e4

    Add nlj SQL benchmark (apache#22805)

    2e26f24

    Add clickbench extended SQL benchmark (apache#22804)

    0dc6528

    Add smj SQL benchmark (apache#22803)

    6a811c3

    Add sort tpch SQL benchmark (apache#22814)

    982fc22
  15. mkleen
    added a commit that references this issue on Aug 5, 2026

    Add support for running sql benchmarks with command line arguments (a…

    373fab7
  16. kosiew
    added a commit that references this issue on Aug 12, 2026

    Add support for running sql benchmarks with command line arguments (a…

    6ba3654
  17. saadtajwar
    added a commit that references this issue on Aug 24, 2026

    Sql benchmark sync updated queries (apache#24583)

    c699996
  18. zzcclp
    added a commit that references this issue on Sep 1, 2026

    SQL benchmark configuration parity updates (apache#24778)

    45b2f73
  19. lyne7-sc
    added a commit that references this issue on Sep 1, 2026

    SQL benchmark configuration parity updates - apache#24778 2 (apache#…

    83022b1
  20. lyne7-sc
    added a commit that references this issue on Sep 12, 2026

    Prepare SQL runner for native benchmark migration (apache#24806)

    14b8100
  21. CaptainEureka
    added a commit that references this issue on Oct 12, 2026

    feat(benchmarks): enable Criterion automatically when a benchmark nam…

    4ab4721
  22. alamb
    added a commit that references this issue on Oct 16, 2026

    feat(benchmarks): add plan diagnostics to SQL benchmark runners (apac…

    4391a2e
  23. zzcclp
    added a commit that references this issue on Oct 21, 2026

    feat(benchmarks): add sql benchmark sort pushdown SQL suite (apache#2…

    b5e5c0e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

EPICA larger project, actively underway, with sub tasksenhancementNew feature or request

Type

Feature

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions