Skip to content

Test benchmark fix#26

Open
EwoutH wants to merge 1 commit intomainfrom
bench_test
Open

Test benchmark fix#26
EwoutH wants to merge 1 commit intomainfrom
bench_test

Conversation

@EwoutH
Copy link
Copy Markdown
Owner

@EwoutH EwoutH commented Mar 6, 2026

Test

@EwoutH EwoutH marked this pull request as draft March 6, 2026 07:56
@EwoutH EwoutH marked this pull request as ready for review March 6, 2026 07:56
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 6, 2026

Performance benchmarks:

Model Size Init time [95% CI] Run time [95% CI]
BoltzmannWealth small 🔵 -0.4% [-0.8%, -0.1%] 🔵 +0.2% [+0.1%, +0.4%]
BoltzmannWealth large 🔵 +0.6% [+0.0%, +1.1%] 🔵 -0.7% [-1.8%, +0.5%]
Schelling small 🔵 +1.2% [+1.0%, +1.5%] 🔵 -0.0% [-0.1%, +0.1%]
Schelling large 🔵 -1.0% [-1.9%, -0.2%] 🔵 -2.3% [-3.3%, -1.5%]
WolfSheep small 🔵 +1.0% [+0.7%, +1.3%] 🔵 +0.1% [-0.0%, +0.2%]
WolfSheep large 🔵 +0.7% [-0.5%, +1.9%] 🔵 +0.6% [+0.1%, +1.0%]
SugarscapeG1mt small 🔵 +0.8% [+0.4%, +1.1%] 🔵 +1.4% [+1.2%, +1.6%]
SugarscapeG1mt large 🔵 +2.9% [+1.7%, +4.0%] 🔵 +0.7% [+0.3%, +1.1%]
BoidFlockers small 🔵 +1.6% [+1.2%, +1.9%] 🔵 -0.9% [-1.1%, -0.8%]
BoidFlockers large 🔵 +1.9% [+1.5%, +2.3%] 🔵 -0.3% [-0.5%, -0.2%]

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 6, 2026

Performance benchmarks:

Model Size Init time [95% CI] Run time [95% CI]
BoltzmannWealth small 🔴 +3.7% [+3.2%, +4.3%] 🔵 +1.2% [+0.9%, +1.5%]
BoltzmannWealth large 🔵 +3.0% [+2.2%, +3.8%] 🔴 +13.4% [+11.3%, +15.5%]
Schelling small 🔴 +3.3% [+3.0%, +3.6%] 🔵 +2.7% [+2.5%, +2.8%]
Schelling large 🔵 +1.9% [+1.0%, +2.8%] 🔴 +6.1% [+5.2%, +6.8%]
WolfSheep small 🔵 +1.8% [+1.4%, +2.1%] 🔵 +2.3% [+2.1%, +2.5%]
WolfSheep large 🔵 -0.2% [-2.1%, +1.7%] 🔵 +1.0% [-2.7%, +4.5%]
SugarscapeG1mt small 🟢 -5.3% [-6.2%, -4.5%] 🔵 -1.8% [-2.5%, -1.1%]
SugarscapeG1mt large 🔵 -3.5% [-4.7%, -2.1%] 🔵 -1.0% [-1.4%, -0.6%]
BoidFlockers small 🔵 +0.9% [+0.5%, +1.4%] 🔵 +0.6% [+0.4%, +0.8%]
BoidFlockers large 🔵 +2.4% [+1.8%, +3.1%] 🔵 +0.1% [-0.0%, +0.3%]

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 6, 2026

Performance benchmarks:

Model Size Init time [95% CI] Run time [95% CI]
BoltzmannWealth small 🔵 +0.9% [+0.3%, +1.4%] 🔵 -0.3% [-0.5%, -0.1%]
BoltzmannWealth large 🔵 +3.2% [+1.2%, +5.7%] 🔴 +7.2% [+5.1%, +9.7%]
Schelling small 🔵 +3.0% [+2.3%, +3.6%] 🔵 -0.1% [-0.4%, +0.1%]
Schelling large 🔵 +3.5% [+0.9%, +6.4%] 🔴 +6.3% [+3.6%, +9.3%]
WolfSheep small 🔴 +3.7% [+3.2%, +4.2%] 🔵 +2.0% [+1.8%, +2.2%]
WolfSheep large 🔵 +4.6% [+2.6%, +6.4%] 🔴 +7.3% [+5.3%, +9.8%]
SugarscapeG1mt small 🔴 +12.2% [+11.5%, +12.9%] 🔴 +5.5% [+5.2%, +5.9%]
SugarscapeG1mt large 🔵 +5.8% [+1.7%, +10.1%] 🔵 +1.7% [+0.5%, +3.0%]
BoidFlockers small 🔵 +0.9% [+0.2%, +1.6%] 🟢 -3.2% [-3.4%, -3.0%]
BoidFlockers large 🔵 +1.1% [+0.3%, +1.9%] 🔵 -3.0% [-3.4%, -2.6%]

@EwoutH
Fix benchmark workflow permissions for fork PRs
409dce6 
Split the benchmark workflow into two files to resolve the "Resource not accessible by integration" error (403) when commenting on PRs from forks.

The `pull_request` event gives a read-only token for fork PRs, so the comment step always fails. Using a separate `workflow_run` workflow for commenting runs in the base repo context, which has write permissions.

Also removes the base64 encoding workaround by passing results through artifacts directly as text files.
@EwoutH EwoutH changed the title Update __init__.py Test benchmark fix Mar 6, 2026
@EwoutH EwoutH mentioned this pull request Mar 6, 2026
Merged
@EwoutH
Copy link
Copy Markdown
Owner Author

EwoutH commented Mar 6, 2026

Fixing it in mesa#3438.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@EwoutH