Skip to content

ModelDataSet window_size eviction causes unpacking crash #3378

@ShreyasN707

Description

@ShreyasN707

Hi,
While testing ModelDataSet with a sliding window (DatasetConfig(window_size=N)), I ran into a crash once the window limit is reached.

In DataRecorder._store_dataset_snapshot, eviction assumes blocks are stored as (time, data) tuples:

_old_time, old_data = storage.blocks[0]

However, for modeldataset, blocks are stored as a single merged dict:

row = {**data, "time": time}
storage.blocks.append(row)

So when eviction tries to unpack storage.blocks[0], it ends up unpacking a dict and raises:

ValueError: too many values to unpack

This only happens when:

  • Using ModelDataSet

  • Setting window_size

  • Running long enough to trigger eviction

It looks like an internal inconsistency — other dataset types store (time, data), but modeldataset stores a dict directly.
A minimal fix would be to store modeldataset blocks as (time, data) like the others, and handle merging "time" during DataFrame conversion instead.

I can open a small PR if this approach is correct.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions