Skip to content

Prevent RecursionError and data loss in Cell/Grid deepcopy#3222

Merged
quaquel merged 4 commits intomesa:mainfrom
falloficarus22:cell-deepcopy-connections
Jan 31, 2026
Merged

Prevent RecursionError and data loss in Cell/Grid deepcopy#3222
quaquel merged 4 commits intomesa:mainfrom
falloficarus22:cell-deepcopy-connections

Conversation

@falloficarus22
Copy link
Copy Markdown
Contributor

@falloficarus22 falloficarus22 commented Jan 29, 2026 •
edited
Loading

Description

This PR addresses critical bugs in the new discrete_space system when copying or pickling model states. This is essential for model branching, batch runs, and saving/loading model progress.

Problem

  • RecursionError: In large or dense grids (e.g., 100x100), the default deepcopy algorithm exceeds Python's maximum recursion depth due to the high number of interconnected circular cell references.
  • Data Loss: The previous Cell.__getstate__ implementation attempted to avoid recursion by nullifying neighbor references. This made standalone copied cells unusable (neighbors became None) and broke relational integrity within the grid.
  • Overwritten Modifications: Grid.__setstate__ was re-calling _connect_cells(), which would blindly recalculate the default grid pattern and overwrite any custom connections or removals made to the grid topology before serialization.

Solution

  • Coordinate-based Serialization: Cell.__getstate__ now extracts the neighbor's coordinate instead of the object reference during state collection. This "breaks" the circular object chain for the pickler, preventing RecursionError.
  • Structured Relinking: DiscreteSpace and Grid handle the "relinking" phase in setstate. Once all cells are restored, the parent space iterates through them and replaces the coordinate placeholders with actual Cell object pointers from its internal mapping.
  • Topology Preservation: By relinking based on the serialized state rather than re-running the generator pattern, any manual changes to the grid (like removing specific walls or adding non-standard links) are correctly preserved across the copy/pickle cycle.

Changes

  • mesa/discrete_space/cell.py: Updated getstate to use coordinate mapping.
  • mesa/discrete_space/discrete_space.py: Added coordinate-based relinking in setstate.
  • mesa/discrete_space/grid.py: Updated setstate to use the new relinking logic and removed the redundant _connect_cells() call.
  • tests/discrete_space/test_discrete_space.py: Added a new test test_cell_deepcopy to verify both standalone and grid-based relational integrity.

Closes #3221

@falloficarus22 falloficarus22 marked this pull request as draft January 29, 2026 09:28
@falloficarus22 falloficarus22 marked this pull request as ready for review January 29, 2026 09:29
@falloficarus22 falloficarus22 marked this pull request as draft January 29, 2026 09:30
@github-actions
Copy link
Copy Markdown

github-actions bot commented Jan 29, 2026

Performance benchmarks:

Model Size Init time [95% CI] Run time [95% CI]
BoltzmannWealth small 🔵 -3.0% [-3.8%, -2.3%] 🔵 -1.1% [-1.3%, -0.9%]
BoltzmannWealth large 🔵 -0.8% [-1.4%, -0.2%] 🔵 +2.5% [+0.6%, +4.3%]
Schelling small 🔵 +0.1% [-0.3%, +0.5%] 🔵 +0.5% [+0.4%, +0.7%]
Schelling large 🔵 +0.7% [+0.2%, +1.3%] 🔵 +1.2% [+0.3%, +2.3%]
WolfSheep small 🔵 -0.8% [-1.2%, -0.4%] 🔵 +0.7% [+0.5%, +0.9%]
WolfSheep large 🔵 +0.3% [-1.2%, +1.5%] 🔵 +1.1% [-0.2%, +2.4%]
BoidFlockers small 🔵 -0.8% [-1.2%, -0.4%] 🔵 +1.4% [+1.3%, +1.5%]
BoidFlockers large 🔵 -0.2% [-0.6%, +0.2%] 🔵 +0.9% [+0.7%, +1.0%]

@github-actions
Copy link
Copy Markdown

github-actions bot commented Jan 29, 2026

Performance benchmarks:

Model Size Init time [95% CI] Run time [95% CI]
BoltzmannWealth small 🔵 -0.7% [-1.1%, -0.2%] 🔵 +0.5% [+0.3%, +0.8%]
BoltzmannWealth large 🔵 -0.2% [-0.6%, +0.2%] 🔵 +0.5% [-1.4%, +2.8%]
Schelling small 🔵 +1.8% [+1.4%, +2.2%] 🔵 +2.1% [+1.9%, +2.4%]
Schelling large 🔵 -1.3% [-2.3%, -0.2%] 🔵 -6.7% [-11.0%, -2.0%]
WolfSheep small 🟢 -3.7% [-4.1%, -3.3%] 🔵 +0.5% [+0.4%, +0.7%]
WolfSheep large 🟢 -7.3% [-8.1%, -6.4%] 🟢 -9.7% [-10.8%, -8.4%]
BoidFlockers small 🔵 -1.9% [-2.3%, -1.5%] 🔵 +0.3% [+0.2%, +0.5%]
BoidFlockers large 🔵 -0.3% [-0.6%, +0.1%] 🔵 -1.0% [-1.2%, -0.7%]

@falloficarus22
fix(discrete_space): fix data loss and RecursionError in Cell/Grid de…
9b843c4 
…epcopy

- Implement coordinate-based serialization in 'Cell.__getstate__' to break
  circular references, preventing 'RecursionError' in large grids.
- Add relinking logic to [DiscreteSpace] and [Grid] [__setstate__] to
  reconstruct neighbor object references from coordinates.
- Fix a bug where standalone [Cell]deepcopies resulted in 'None' connections.
- Ensure manual grid modifications are preserved after serialization.
- Add 'test_cell_deepcopy' to verify graph integrity after copy.
@falloficarus22 falloficarus22 force-pushed the cell-deepcopy-connections branch from 5febe1d to 9b843c4 Compare January 29, 2026 09:48
@falloficarus22 falloficarus22 marked this pull request as ready for review January 29, 2026 09:49
@github-actions
Copy link
Copy Markdown

github-actions bot commented Jan 29, 2026

Performance benchmarks:

Model Size Init time [95% CI] Run time [95% CI]
BoltzmannWealth small 🔵 -2.3% [-2.7%, -2.0%] 🔵 -0.6% [-0.9%, -0.4%]
BoltzmannWealth large 🔵 +0.2% [-0.1%, +0.6%] 🔵 +0.3% [-0.0%, +0.7%]
Schelling small 🔵 +1.5% [+1.2%, +1.8%] 🔵 +1.4% [+1.2%, +1.5%]
Schelling large 🔵 +1.2% [+1.0%, +1.5%] 🔵 +3.5% [+2.6%, +4.3%]
WolfSheep small 🔵 +1.9% [+1.6%, +2.3%] 🔵 -0.4% [-0.6%, -0.2%]
WolfSheep large 🔵 +0.5% [-0.1%, +1.1%] 🔵 -0.1% [-1.7%, +1.6%]
BoidFlockers small 🔵 -1.3% [-1.7%, -1.0%] 🔵 -0.4% [-0.6%, -0.3%]
BoidFlockers large 🔵 -0.9% [-1.3%, -0.5%] 🔵 -0.8% [-0.9%, -0.6%]

@falloficarus22 falloficarus22 mentioned this pull request Jan 29, 2026
Closed
quaquel
quaquel reviewed Jan 29, 2026
View reviewed changes
quaquel
quaquel reviewed Jan 29, 2026
View reviewed changes
quaquel
quaquel reviewed Jan 29, 2026
View reviewed changes
@falloficarus22 falloficarus22 force-pushed the cell-deepcopy-connections branch from 7980279 to 99aa64c Compare January 29, 2026 14:17
@falloficarus22 falloficarus22 force-pushed the cell-deepcopy-connections branch from 99aa64c to d5da656 Compare January 29, 2026 15:15
@quaquel quaquel added the bug Release notes label label Jan 30, 2026
quaquel
quaquel reviewed Jan 30, 2026
View reviewed changes
mesa/discrete_space/grid.py Outdated
Comment on lines 200 to 204
self.cell_klass = type(
self._cells[(0, 0)]
next(iter(self._cells.values()))
) # the __reduce__ function handles this for us nicely
for layer in self._mesa_property_layers.values():
setattr(
Copy link
Copy Markdown
Member

@quaquel quaquel Jan 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reviewing your code, I realize this should be moved to the DiscreteSpace class.

quaquel
quaquel requested changes Jan 30, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@quaquel quaquel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one last minor change and then I think this is ready to be merged.

@falloficarus22 falloficarus22 force-pushed the cell-deepcopy-connections branch from 3d6bced to 6153042 Compare January 31, 2026 03:49
@quaquel quaquel merged commit b5b3598 into mesa:main Jan 31, 2026
14 checks passed
@falloficarus22 falloficarus22 deleted the cell-deepcopy-connections branch January 31, 2026 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@quaquel quaquel quaquel approved these changes

Assignees

No one assigned

Labels

bug Release notes label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Data Loss and Recursion Errors in Cell/Grid Deepcopy/Pickle

2 participants

@falloficarus22 @quaquel