-
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Enhance DataCollector to Validate model_reporters Functions #2606
Description
Describe the bug
Currently, the DataCollector's model_reporters validation has limitations when handling string attributes. The issue arises because:
-
Model reporters can accept four types of inputs:
model_reporters = { "Count": lambda m: len(m.agents), # Lambda function "Status": "market_status", # Attribute name (string) "Metric": model.calculate_metric, # Method reference "Custom": [calculate_wealth, [param1, param2]] # Function with parameters }
-
When using string attributes (e.g., "market_status"), there's no validation to check if these attributes actually exist in the model. This leads to:
- Silent failures when collecting undefined attributes
- Confusion when attributes are misspelled or not initialized
- Delayed error detection until runtime
Adding a validation mechanism to ensure model_reporters keys mapped to callable functions or valid strings would greatly improve error detection and provide better debugging experiences for users.
Expected behavior
When using DataCollector, if an invalid value is provided in model_reporters (e.g., a non-callable object or undefined function):
Raise a ValueError with a clear error message, such as:
"Invalid reporter for 'Gini': must be a callable function/method or a valid attribute name as a string."
This prompt will help users:
- Understand their mistake.
- Quickly identify whether the provided function (e.g., compute_gini) is missing, incorrectly implemented, or improperly referenced.
To Reproduce
Valid Example:
`from mesa.datacollection import DataCollector
def compute_gini(model):
return 0.5
collector = DataCollector(
model_reporters={"Gini": compute_gini},
agent_reporters={"Wealth": "wealth"}
)`Invalid Example (Missing Function):
`from mesa.datacollection import DataCollector
collector = DataCollector(
model_reporters={"Gini": "compute_gini"}, # Error: compute_gini is not callable
agent_reporters={"Wealth": "wealth"}
)`Expected Output:
`ValueError: Invalid reporter for 'Gini': must be a callable function/method or a valid attribut`
Proposed Solution
Modify the DataCollector class to include validation for model_reporters, similar to the following:
def _validate_model_reporter(self, name, reporter):
# Type 1: Lambda function
if callable(reporter):
return
# Type 2: Class attributes (string)
elif isinstance(reporter, str):
return
# Type xxx: other types mechanism to warn
else:
raise ValueError(
f"Invalid reporter for '{name}': must be a callable function/method or a valid attribute name as a string."
)Additional context
- Affected Class: DataCollector
- Documentation: Introductory Tutorial
- Location in tutorial: the section of "Collecting Data"
- api documentation: Data collection