preserve explicit meta attributes/methods over inferred members#3472
preserve explicit meta attributes/methods over inferred members#3472
Conversation
|
Performance benchmarks:
|
| # Initialize meta_methods if not provided | ||
| meta_methods = {} | ||
| for agent_class in agent_classes: | ||
| seen_classes = set() |
There was a problem hiding this comment.
Why are you looping through all the agents here, instead of just looping through a set of the types as was being done previously?
There was a problem hiding this comment.
I switched away from iterating a plain set of types because that loses order, which made inherited method selection unstable when multiple constituting agent classes define the same method name. The goal was not to inspect every agent for its own sake, but to preserve first-seen type order while still deduplicating classes.
I agree the current form makes that less obvious, so I can simplify it to iterate an order-preserving deduplicated list of types instead, for example via dict.fromkeys(type(agent) for agent in agents). That keeps the determinism fix while making the intent clearer.
There was a problem hiding this comment.
This raises some key questions that need to be addressed for the larger meta-agent development. So with GSoC pending I am going to use this PR to explore some meta-agent challenges.
For consideration:
Deterministic order of "inheritance" is not necessarily the problem we want to solve/ the capability we want to provide. My thoughts are below, but curious on other thoughts.
The first question is what is the purpose of a meta-agent inheriting a method from one of its constituting agents?
- one use is the idea that a meta-agent, would have a different "view" than that of a constituting agent so would have a different outcome than that of the constituting agent. Otherwise, the user could just execute the constituting agents method
- another would be a potential efficiency where say the meta-agent has numerous constituting of the same type and can get a similar impact by executing once as the meta-agent.
Either way, if two constituting agent types have a method with the same name resulting in a collision, just selecting the first one may create a bug in their simulation if they don't realize they have a naming collision. In this dynamic it may be better to have a warning about the collision than it would be to select for them. There may be some other ways to handle this, but I am not sure effectively making the choice for them is the right answer.
The second question is what is the purpose of meta-agents inheriting attributes from its constituting agents? This raises different concerns, for example, if you take the alliance formation and lets say the constituting agents have an attribute of wealth, adding that attribute is a problem as it will only have one agent's wealth as opposed to the aggregation (as one possible approach to wealth sharing in an alliance) of all constituting agents wealth. Conversely, if one constituting agent has gold and the other has silver than this could be useful.
Based on this thought experiment, I am thinking meta-agent attribute adoption is of a different kind and should be handled differently than methods. Thoughts?
There was a problem hiding this comment.
There are really two different questions here, and I do not think this PR should answer both by accident.
For this PR, the bug I was aiming to fix is narrower: if the user explicitly passes meta_methods or meta_attributes, Mesa should not overwrite those with inferred values. That seems independent from the larger question of how inferred values should behave when they collide.
My current view is that methods and attributes should be treated differently. For methods, inference can still be useful as a convenience, but if two constituting agent types infer the same method name, I do not think Mesa should silently pick one without surfacing that collision. For attributes, I think the risk is higher: copying one constituent’s state onto the meta-agent can easily imply meaning that is not generally valid, so attribute inference probably needs a more restrictive design than method inference.
So I think this PR can stay narrowly focused on preserving explicit user intent, while broader handling of inferred collisions, especially for attributes, is better handled in the larger meta-agent work.
|
See #3538 (comment) @falloficarus22 let me know what think, in short I think there is a different solution that solves the problem identified here as well as the current brittleness of the meta-agent approach. |
|
In light of the discussion on #3538, I think the attribute side of this PR should probably follow the “do not overwrite existing meta-agent attributes” direction instead. That seems like a better fix shape for attributes than what I have here. I’m planning to narrow this PR to the method side only, since explicit |
tpike3
left a comment
There was a problem hiding this comment.
LGTM -- Thanks @falloficarus22
Note
Use this template for bug fixes only. For enhancements/new features, use the feature template and get maintainer approval in an issue/discussion before opening a PR.
Pre-PR Checklist
Summary
Fixes
create_meta_agentso explicitly providedmeta_attributesandmeta_methodsare not overwritten by values inferred from constituting agents. Inferred values now only fill missing keys.Bug / Issue
When
assume_constituting_agent_attributes=Trueand/orassume_constituting_agent_methods=True, the previous implementation could overwrite caller-provided keys inmeta_attributes/meta_methodsif constituting agents had matching names.Expected behavior: explicit caller-provided values should always win.
Actual behavior: inferred values could replace explicit ones without warning.
Related context: meta-agent determinism discussions in #3184, but this PR addresses explicit-vs-inferred priority specifically.
Implementation
create_meta_agentinmesa/experimental/meta_agents/meta_agent.py:add_methods(...)now:meta_methodssetdefault)seen_classes)add_attributes(...)now:meta_attributessetdefault)meta_methods/meta_attributes.Testing
Added regression tests in tests/experimental/test_meta_agents.py:
test_explicit_meta_attributes_take_precedence_over_inferredtest_explicit_meta_methods_take_precedence_over_inferredAdditional Notes
meta_attributes/meta_methodsare overwritten by inferred constituent values #3471