Skip to content

Add time column to empty dataframe in datarecorder#3408

Merged
quaquel merged 2 commits intomesa:mainfrom
codebreaker32:add_time
Mar 2, 2026
Merged

Add time column to empty dataframe in datarecorder#3408
quaquel merged 2 commits intomesa:mainfrom
codebreaker32:add_time

Conversation

@codebreaker32
Copy link
Copy Markdown
Collaborator

@codebreaker32 codebreaker32 commented Mar 1, 2026

This PR fixes an inconsistency in get_table_dataframe where empty datasets would bypass their specific reconstruction logic if storage.blocks is empty( for eg when you call recorder.clear()). Because Datasets do not store time in their metadata, this premature short-circuit caused their empty DataFrame outputs to be missing the time column entirely.

Changes:

  • Removed the global short-circuit for empty storage.blocks.
  • Empty datasets now correctly dispatch to their specific converters
  • The global short-circuit is now strictly reserved for completely fresh/uninitialized datasets (data_type == unknown).

Previously if you run

model = BoltzmannWealth()
model.run_for(5)
model.recorder.clear()

print(model.recorder.get_table_dataframe("agent_data"))
print(model.recorder.get_table_dataframe("model_data"))

Old output

Empty DataFrame
Columns: [unique_id, wealth]
Index: []
Empty DataFrame
Columns: [gini, time]
Index: []

Note: Here time is in columns because modeldataset stores data as a dictionary( for eg. {"gini":.., "time":..) but this behavior would be changed once #3389 is merged

New Output

Empty DataFrame
Columns: [unique_id, wealth, time]
Index: []
Empty DataFrame
Columns: [gini, time]
Index: []

If you run above setup with start_time > 5( i.e data_type=="unknown") then output would be

Old output

Empty DataFrame
Columns: []
Index: []
Empty DataFrame
Columns: []
Index: []

New Output

Empty DataFrame
Columns: [time]
Index: []
Empty DataFrame
Columns: [time]
Index: []

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 1, 2026

Performance benchmarks:

Model Size Init time [95% CI] Run time [95% CI]
BoltzmannWealth small 🔵 +2.7% [+2.2%, +3.2%] 🔵 -1.5% [-1.9%, -1.2%]
BoltzmannWealth large 🔵 +1.9% [+1.4%, +2.5%] 🔴 +7.0% [+6.2%, +7.9%]
Schelling small 🔵 +1.2% [+0.8%, +1.7%] 🔵 +0.7% [+0.6%, +0.9%]
Schelling large 🔵 -0.1% [-1.0%, +0.7%] 🔴 +5.4% [+4.4%, +6.6%]
WolfSheep small 🔵 -1.4% [-1.8%, -1.0%] 🔵 -0.7% [-0.9%, -0.5%]
WolfSheep large 🟢 -4.2% [-5.1%, -3.3%] 🟢 -5.7% [-6.8%, -4.6%]
BoidFlockers small 🔵 -0.1% [-0.4%, +0.2%] 🔵 -2.3% [-2.5%, -2.2%]
BoidFlockers large 🔵 +0.3% [-0.1%, +0.7%] 🔵 -2.2% [-2.4%, -2.0%]

@quaquel quaquel merged commit ffe5f2f into mesa:main Mar 2, 2026
12 of 14 checks passed
@codebreaker32 codebreaker32 deleted the add_time branch March 9, 2026 17:14
@EwoutH EwoutH added experimental Release notes label enhancement Release notes label labels Mar 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Release notes label experimental Release notes label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants