Skip to content

Fix: Method Reporter Validation in DataCollector#3002

Merged
quaquel merged 4 commits intomesa:mainfrom
vedantvakharia:fix-datacollector-clean
Dec 31, 2025
Merged

Fix: Method Reporter Validation in DataCollector#3002
quaquel merged 4 commits intomesa:mainfrom
vedantvakharia:fix-datacollector-clean

Conversation

@vedantvakharia
Copy link
Copy Markdown
Contributor

Summary

This PR fixes a bug in DataCollector._validate_model_reporter() where method reporters were never validated due to incorrect conditional logic.


The Bug

Original Code (lines 166-168 in datacollection.py)

# Type 2: Method of class/instance
if not callable(reporter) and not isinstance(reporter, types.LambdaType):
    pass

Why This Was Wrong

Expression For a Method Result
callable(self.some_method) True Methods are callable
not callable(reporter) False Always False for methods
if condition False and ... Never executes

Method reporters were completely skipped during validation.


The Fix

# Type 2: Method of class/instance (bound methods are callable)
if callable(reporter) and not isinstance(reporter, types.LambdaType):
    try:
        reporter()  # Call without args for bound methods
    except Exception as e:
        raise RuntimeError(
            f"Method reporter '{name}' failed validation: {e!s}"
        ) from e

Key Changes

Aspect Original Fixed
Condition not callable(reporter) callable(reporter)
Matches methods? ❌ Never ✅ Yes
Validation pass (nothing) Calls reporter()
Error handling None Wraps in RuntimeError with reporter name

Why reporter() Instead of reporter(model)?

Looking at how collect() calls method reporters (line 341):

# Bound methods: called WITHOUT arguments (self is already bound)
self.model_vars[var].append(deepcopy(reporter()))

Bound methods like self.some_method already have self bound, so they're called with no arguments.


Comparison Table

Test Scenario Original Code Fixed Code
Method referencing non-existent attribute ❌ Not caught, unclear AttributeError RuntimeError with reporter name
Method that raises exception ❌ Raw exception ✅ Wrapped with context
Working method reporter ✅ Works ✅ Works
Lambda + method mix ✅ Works ✅ Works

Tests Added

Added TestMethodReporterValidation class to tests/test_datacollector.py:

Test Description
test_broken_method_reporter_raises_runtime_error Broken methods raise RuntimeError with reporter name
test_working_method_reporter_succeeds Valid methods continue to work
test_method_reporter_exception_is_wrapped Exceptions wrapped with context
test_mixed_lambda_and_method_reporters Lambdas and methods work together
test_method_reporter_called_without_args Methods called with reporter()

Test Results

$ pytest tests/test_datacollector.py -v
20 passed  # 15 original + 5 new

Files Changed

File Change
mesa/datacollection.py Fixed method reporter validation condition
tests/test_datacollector.py Added TestMethodReporterValidation class (5 tests)

Backwards Compatibility

✅ All 363 existing tests pass
✅ Working method reporters unaffected
✅ Only broken reporters now fail with clear error message

@github-actions
Copy link
Copy Markdown

Performance benchmarks:

Model Size Init time [95% CI] Run time [95% CI]
BoltzmannWealth small 🔵 +1.0% [+0.6%, +1.5%] 🔵 -0.4% [-0.5%, -0.2%]
BoltzmannWealth large 🔵 +3.4% [+1.3%, +7.1%] 🔵 -0.2% [-0.5%, +0.1%]
Schelling small 🔵 +0.1% [-1.2%, +1.6%] 🔵 -0.2% [-0.8%, +0.4%]
Schelling large 🔵 +1.2% [-0.6%, +3.7%] 🔵 +0.7% [-0.0%, +1.3%]
WolfSheep small 🔵 -0.8% [-1.8%, -0.1%] 🔵 +0.5% [+0.3%, +0.7%]
WolfSheep large 🔵 -2.2% [-6.1%, -0.1%] 🔵 +0.5% [+0.2%, +0.8%]
BoidFlockers small 🔵 +0.2% [-0.2%, +0.6%] 🔵 +1.1% [+0.9%, +1.3%]
BoidFlockers large 🔵 +0.2% [-0.1%, +0.5%] 🔵 +0.3% [+0.1%, +0.6%]

@quaquel quaquel added the bug Release notes label label Dec 30, 2025
@vedantvakharia
Copy link
Copy Markdown
Contributor Author

Hi @quaquel I have resolved the conflicts and accepted both changes. The tests are passing. Could you please re-approve so the pr can be merged?

@quaquel quaquel merged commit fc0d7bf into mesa:main Dec 31, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Release notes label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants