Skip to content

Add manual dirty flag optimization to AgentDataSet#3346

Merged
quaquel merged 9 commits intomesa:mainfrom
souro26:dirty-flag-agentdataset
Feb 21, 2026
Merged

Add manual dirty flag optimization to AgentDataSet#3346
quaquel merged 9 commits intomesa:mainfrom
souro26:dirty-flag-agentdataset

Conversation

@souro26
Copy link
Copy Markdown
Contributor

@souro26 souro26 commented Feb 19, 2026

Summary

Adds a manual dirty-flag optimization to AgentDataSet to avoid repeated data extraction within the same simulation instant. This is implemented as a pull-based caching mechanism that recomputes only when explicitly invalidated.

Adds the dirty flag function mentioned in #3341

Motive

AgentDataSet.data currently recomputes the snapshot on every access. When datasets are reused within the same simulation instant, this results in redundant extraction work. This change introduces a manual set_dirty_flag() method so model code can explicitly mark the dataset as stale when needed.

Implementation

Added two internal attributes to AgentDataSet, _is_dirty and _cache and modified .data to recompute only if _is_dirty is True or _cache is None, otherwise return cached snapshot

Added a function,

def set_dirty_flag(self) -> None

which marks the dataset as dirty for the next access.

Usage Examples

dataset = AgentDataSet("wealth", model.agents, fields="wealth")

# First access computes snapshot
snapshot = dataset.data

# Reuse within same instant (cached)
snapshot_again = dataset.data

# After model mutation
dataset.set_dirty_flag()
updated_snapshot = dataset.data

Additional Notes

  • This change alters identity behavior so that repeated .data access now returns the same list object until invalidated.
  • Snapshot values and timing semantics remain unchanged and returned snapshot remains mutable.
  • Added tests to cover the new changes

@github-actions
Copy link
Copy Markdown

Performance benchmarks:

Model Size Init time [95% CI] Run time [95% CI]
BoltzmannWealth small 🔵 +1.6% [+0.4%, +2.8%] 🟢 -15.8% [-16.0%, -15.5%]
BoltzmannWealth large 🔵 +2.0% [+1.6%, +2.3%] 🔵 -3.8% [-5.5%, -2.2%]
Schelling small 🔵 -0.3% [-0.8%, +0.2%] 🔵 +1.0% [+0.8%, +1.2%]
Schelling large 🔵 +1.3% [+0.6%, +2.1%] 🔴 +6.9% [+4.3%, +9.4%]
WolfSheep small 🔵 +1.5% [+0.4%, +2.6%] 🔵 +1.5% [+1.2%, +1.8%]
WolfSheep large 🔵 -1.0% [-2.8%, +0.9%] 🔵 -2.4% [-4.3%, -0.6%]
BoidFlockers small 🔵 -0.1% [-0.5%, +0.2%] 🔵 +1.3% [+1.2%, +1.5%]
BoidFlockers large 🔵 +0.0% [-0.5%, +0.6%] 🔵 +0.9% [+0.7%, +1.0%]

@quaquel
Copy link
Copy Markdown
Member

quaquel commented Feb 19, 2026

The current implementation in this PR forces the user to always handle the dirty flag. This is, in my view, less than ideal. Rather it should be an opt in behavior. So, probably in the __init__ we need an extra keyword argument that specifies whether the user wants to opt in or not.

@souro26
Copy link
Copy Markdown
Contributor Author

souro26 commented Feb 19, 2026

I’ve updated the implementation so the dirty-flag caching is now opt-in via a keyword argument in AgentDataSet. By default, behavior remains unchanged and .data recomputes on every access.

Copy link
Copy Markdown
Member

@quaquel quaquel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one minor comment left, but this looks almost ready to go.

@souro26
Copy link
Copy Markdown
Contributor Author

souro26 commented Feb 19, 2026

I’ve made the suggested changes, thanks for the review. Let me know if anything else should be adjusted.

Copy link
Copy Markdown
Member

@quaquel quaquel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good to me. I still want to test a few things locally before merging. Hope to get to this somewhere in the coming days.

Comment on lines 544 to 550
def track_agents(
self,
agents: AbstractAgentSet,
name: str,
fields: str | list[str] | None = None,
) -> AgentDataSet:
"""Track the specified fields for the agents in the AgentSet."""
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please update this as well to match the new signature of AgentDataSet

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated the track_agents signature to match AgentDataSet

@quaquel quaquel merged commit b07ca36 into mesa:main Feb 21, 2026
13 of 14 checks passed
@quaquel quaquel added the enhancement Release notes label label Feb 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Release notes label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants