Skip to content

Enhance DataCollector to Validate model_reporters Functions #2606

@peter-kinger

Description

@peter-kinger

Describe the bug
Currently, the DataCollector's model_reporters validation has limitations when handling string attributes. The issue arises because:

  1. Model reporters can accept four types of inputs:

    model_reporters = {
        "Count": lambda m: len(m.agents),              # Lambda function
        "Status": "market_status",                     # Attribute name (string)
        "Metric": model.calculate_metric,              # Method reference
        "Custom": [calculate_wealth, [param1, param2]] # Function with parameters
    }
  2. When using string attributes (e.g., "market_status"), there's no validation to check if these attributes actually exist in the model. This leads to:

    • Silent failures when collecting undefined attributes
    • Confusion when attributes are misspelled or not initialized
    • Delayed error detection until runtime

Adding a validation mechanism to ensure model_reporters keys mapped to callable functions or valid strings would greatly improve error detection and provide better debugging experiences for users.

Expected behavior
When using DataCollector, if an invalid value is provided in model_reporters (e.g., a non-callable object or undefined function):

Raise a ValueError with a clear error message, such as:
"Invalid reporter for 'Gini': must be a callable function/method or a valid attribute name as a string."
This prompt will help users:

  1. Understand their mistake.
  2. Quickly identify whether the provided function (e.g., compute_gini) is missing, incorrectly implemented, or improperly referenced.

To Reproduce

Valid Example:

`from mesa.datacollection import DataCollector  

def compute_gini(model):     
	return 0.5  

collector = DataCollector(     
	model_reporters={"Gini": compute_gini},     
	agent_reporters={"Wealth": "wealth"} 
)`

Invalid Example (Missing Function):

`from mesa.datacollection import DataCollector  

collector = DataCollector(     
	model_reporters={"Gini": "compute_gini"},  # Error: compute_gini is not callable     
	agent_reporters={"Wealth": "wealth"} 
)`

Expected Output:

`ValueError: Invalid reporter for 'Gini': must be a callable function/method or a valid attribut`

Proposed Solution
Modify the DataCollector class to include validation for model_reporters, similar to the following:

def _validate_model_reporter(self, name, reporter): 
	# Type 1: Lambda function
	if callable(reporter): 
		return 
	# Type 2: Class attributes (string)
	elif isinstance(reporter, str): 
		return 
	# Type xxx: other types mechanism to warn
	else: 
		raise ValueError(
		 f"Invalid reporter for '{name}': must be a callable function/method or a valid attribute name as a string." 
		)

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions