Skip to content

GH-32950: [Go] REE Benchmarks - #34666

Merged
zeroshade merged 1 commit into
apache:mainfrom
zeroshade:ree-benchmarks
Mar 22, 2023
Merged

zeroshade merged 1 commit into
apache:mainfrom
zeroshade:ree-benchmarks

Conversation

@zeroshade

@zeroshade zeroshade commented Mar 21, 2023 •

Copy link
Copy Markdown
Member

Rationale for this change

Adding benchmarks for run_end_encode and run_end_decode to track and hopefully improve the performance of these kernels.

@zeroshade

Copy link
Copy Markdown
Member Author

CC @felipecrv

@github-actions

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #32950 has been automatically assigned in GitHub to PR creator.

b.SetBytes(byts)
b.ResetTimer()
for n := 0; n < b.N; n++ {
res, err = compute.RunEndEncode(ctx, opts, input)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will vary very widely depending on the size of the runs. Does the REE random generator pay attention to this? Meaning: does it generate a normal distribution of run-lengths?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so right now it's using a specific seed to ensure consistency, but that's a good point that this benchmark is essentially still just benchmarking a random normal distribution of values with no guarantees to any particular distribution of runs and run-lengths. It might make more sense to restrict the min/max values to better guarantee some runs.

@zeroshade
zeroshade merged commit a1153a8 into apache:main Mar 22, 2023
@zeroshade
zeroshade deleted the ree-benchmarks branch March 22, 2023 19:35
@ursabot

ursabot commented Mar 23, 2023

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = fd5d710 and contender = a1153a8. a1153a8 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.45% ⬆️0.0%] test-mac-arm
[Finished ⬇️0.51% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.22% ⬆️0.03%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] a1153a8c ec2-t3-xlarge-us-east-2
[Finished] a1153a8c test-mac-arm
[Finished] a1153a8c ursa-i9-9960x
[Finished] a1153a8c ursa-thinkcentre-m75q
[Finished] fd5d7107 ec2-t3-xlarge-us-east-2
[Failed] fd5d7107 test-mac-arm
[Finished] fd5d7107 ursa-i9-9960x
[Finished] fd5d7107 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@ursabot

ursabot commented Mar 23, 2023

Copy link
Copy Markdown

['Python', 'R'] benchmarks have high level of regressions.
test-mac-arm

ArgusLi pushed a commit to Bit-Quill/arrow that referenced this pull request May 15, 2023
### Rationale for this change
Adding benchmarks for `run_end_encode` and `run_end_decode` to track and hopefully improve the performance of these kernels.

* Closes: apache#32950

Authored-by: Matt Topol <[email protected]>
Signed-off-by: Matt Topol <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Go] Add REE Benchmarks

3 participants