Skip to content

Improvements to DataSet and DataRecorder #3341

@quaquel

Description

@quaquel

In #3156, we added a new DataSet for tracking the state of part of the model at any given instance. This DataSet forms the basis of the new approach to data collection. In #3145, we added DataRecorder for takings snapshots of DataSets. However there is ample room for improvements

  • add DataSet.set_dirty_flag(); A major potential benefit of datasets is that we can reuse them elsewhere in the model. However, to fully benefit from this, we need a way of indicating whether the data is clean or dirty. For example, in Boltzmann, we collect wealth data from agents and use this in gini. Ideally, we gather the agent wealth data only once per step. If we have a way of setting, manually, a flag indicating whether the data is clean or needs to be collected again, that would save runtime
  • Handle run_ended; At the moment, DataRecorder subscribes to model.time ObservableSignals.CHANGED. So whenever time is changed, we collect data. This effectively means that data is collected before the event of the new time is executed. In turn this means that when a run is completed, we do not collect the data after that last event.
  • Add an ObservableAgentDataSet;
  • Align AgentDataSet and NumpyDataSet to both use an Agent class instead of the agents for the first and the class for the second.
  • Agree on and implement an API along the lines of the [conversation here](add DataSet.set_dirty_flag()
    handle run_ended
    add a dataset for observables). DataSet.record has already been added. We also agree on the idea of a Snapshot data class. The other ideas are still under consideration.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions