-
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
ModelDataSet window_size eviction causes unpacking crash #3378
Description
Hi,
While testing ModelDataSet with a sliding window (DatasetConfig(window_size=N)), I ran into a crash once the window limit is reached.
In DataRecorder._store_dataset_snapshot, eviction assumes blocks are stored as (time, data) tuples:
_old_time, old_data = storage.blocks[0]However, for modeldataset, blocks are stored as a single merged dict:
row = {**data, "time": time}
storage.blocks.append(row)
So when eviction tries to unpack storage.blocks[0], it ends up unpacking a dict and raises:
ValueError: too many values to unpack
This only happens when:
-
Using ModelDataSet
-
Setting window_size
-
Running long enough to trigger eviction
It looks like an internal inconsistency — other dataset types store (time, data), but modeldataset stores a dict directly.
A minimal fix would be to store modeldataset blocks as (time, data) like the others, and handle merging "time" during DataFrame conversion instead.
I can open a small PR if this approach is correct.