Skip to content

added RunSpec for executing Scenarios#3641

Closed
souro26 wants to merge 5 commits intomesa:mainfrom
souro26:runspec-base
Closed

added RunSpec for executing Scenarios#3641
souro26 wants to merge 5 commits intomesa:mainfrom
souro26:runspec-base

Conversation

@souro26
Copy link
Copy Markdown
Contributor

@souro26 souro26 commented Mar 28, 2026

Summary

This PR adds a RunSpec class to define how a single Scenario is executed. It has been discussed in #3483 .

Motive

After the Scenario refactor, experiment setup is now handled cleanly, but there is no clear way to define how a Scenario should actually be run. This makes execution logic implicit and inconsistent across use cases. RunSpec introduces a simple and explicit way to define how a model is built, executed, and how results are extracted for a single run.

Implementation

RunSpec acts as a small execution pipeline. By default, it builds a model from a Scenario, runs it using run_for, and returns the model’s data_registry. The class is designed to be subclassed, with clear override points for model construction, execution logic, result extraction, and output formatting.

Tests are added to cover the default behavior and each extension point, including custom execution, extraction, and output formatting.

Usage Examples

Basic usage:

spec = RunSpec(MyModel, steps=100)
sid, rid, result = spec(scenario)

Custom behavior:

class MyRunSpec(RunSpec):
    def execute(self, model):
        model.run_until(50)

    def extract(self, model):
        return model.compute_metrics()

@github-actions
Copy link
Copy Markdown

Performance benchmarks:

Model Size Init time [95% CI] Run time [95% CI]
BoltzmannWealth small 🟢 -3.6% [-4.2%, -3.1%] 🔵 +0.5% [+0.2%, +0.8%]
BoltzmannWealth large 🟢 -3.4% [-3.7%, -3.0%] 🟢 -7.3% [-8.7%, -5.6%]
Schelling small 🔵 -3.1% [-3.8%, -2.4%] 🔵 -0.9% [-1.7%, -0.1%]
Schelling large 🔵 -1.9% [-2.9%, -0.6%] 🟢 -8.0% [-10.4%, -5.2%]
WolfSheep small 🔵 -1.9% [-2.6%, -1.3%] 🔵 -1.2% [-5.5%, +3.0%]
WolfSheep large 🟢 -5.7% [-6.3%, -5.1%] 🟢 -9.3% [-10.5%, -7.8%]
SugarscapeG1mt small 🔵 -2.5% [-3.1%, -1.9%] 🔵 -3.0% [-4.5%, -1.4%]
SugarscapeG1mt large 🔵 -3.0% [-5.2%, -0.5%] 🔵 -2.1% [-3.6%, -0.7%]
BoidFlockers small 🔵 -1.5% [-2.0%, -1.1%] 🔵 -1.6% [-2.1%, -1.1%]
BoidFlockers large 🔵 -0.9% [-1.7%, -0.0%] 🔵 -1.8% [-2.1%, -1.4%]

@souro26
Copy link
Copy Markdown
Contributor Author

souro26 commented Mar 28, 2026

I am not fully sure about the placement of runspec.py . Should it stay in mesa.experimental or in mesa.experimental.scenarios?

@codebreaker32
Copy link
Copy Markdown
Collaborator

codebreaker32 commented Mar 29, 2026

Thanks for the PR but I think this is a bit premature, we haven't had yet reached a consensus on the separation of concerns. Also your extract function doesn't seem good, data_registry is no way useful for extracting data

@souro26 souro26 marked this pull request as draft March 29, 2026 08:11
@souro26
Copy link
Copy Markdown
Contributor Author

souro26 commented Mar 29, 2026

Also your extract function doesn't seem good, data_registry is no way useful for extracting data

I agree, i kept it as a placeholder. I'll try a few more things out locally before updating this PR.

@quaquel
Copy link
Copy Markdown
Member

quaquel commented Mar 29, 2026

I am in favour of first hashing this out in more detail in the appropriate discussion rather than spreading it out over PRs and the discussion.

@quaquel quaquel closed this Mar 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants