Skip to content

Adding automated CPU benchmarks#4711

Closed
simone-silvestri wants to merge 17 commits intomainfrom
ss/more-benchmarks
Closed

Adding automated CPU benchmarks#4711
simone-silvestri wants to merge 17 commits intomainfrom
ss/more-benchmarks

Conversation

@simone-silvestri
Copy link
Copy Markdown
Collaborator

@simone-silvestri simone-silvestri commented Aug 13, 2025

This PR is here in preparation for #4706. It adds some unit benchmarks that will run on the CPU.
They have to be taken with a grain of salt since Tartarus might be occupied while running the benchmarks so the performance might show high variance. However it is good to have at least a pipeline for tests

Additionally, this PR revamps and fixes the Benchmark.jl module to be used in the benchmark tests

@simone-silvestri simone-silvestri added the benchmark performance runs preconfigured benchamarks and spits out timing label Aug 13, 2025
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need the Manifest? That would pin the Oceananigans version used, right?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah we probably don't need it. I can delete it. Oceananigans is added during the tests here

$TARTARUS_HOME/julia-$JULIA_VERSION/bin/julia --color=yes --project=benchmark --check-bounds=no -e 'using Pkg; Pkg.add(path="./"); Pkg.instantiate()'

@glwagner
Copy link
Copy Markdown
Member

Why don't we use the benchmark agent for this pipeline? That way it is guaranteed that only one benchmark runs at a time.

@simone-silvestri
Copy link
Copy Markdown
Collaborator Author

Why don't we use the benchmark agent for this pipeline? That way it is guaranteed that only one benchmark runs at a time.

Right, this benchmark is run on the benchmark pipeline.

@glwagner glwagner requested a review from giordano October 5, 2025 19:54
command: |
# Instantiate
$TARTARUS_HOME/julia-$JULIA_VERSION/bin/julia --color=yes --project --check-bounds=no -e 'using Pkg; Pkg.add("CUDA"); Pkg.instantiate()'
$TARTARUS_HOME/julia-$JULIA_VERSION/bin/julia --color=yes --project=benchmark --check-bounds=no -e 'using Pkg; Pkg.add(path="./"); Pkg.instantiate()'
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With Julia v1.11 you could use [sources] instead, but I see you're still on v1.10 for the time being

@simone-silvestri
Copy link
Copy Markdown
Collaborator Author

I guess this PR is being superceded by #4882

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

benchmark performance runs preconfigured benchamarks and spits out timing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants