Refactor PropertyLayer to implement NumPy interface#3074
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
|
Since you are now raising a deprecation warning, all tests are failing. So, the tests need to be updated in line with the new API. Performance gains look modest but are still very welcome. |
b82907e to
34d213a
Compare
…native NumPy functionality
b740272 to
182bbb7
Compare
|
I believe it is fit to review now @quaquel |
d6a5b64 to
6842547
Compare
e623d41 to
f6675c0
Compare
|
Hi @quaquel The PR is now back to just the essentials:
Tests have been updated to handle the warnings. |
dd00441 to
f0da9a4
Compare
|
Hi @quaquel I have now removed all the dunder(magic) methods and accordingly updated the PR description |
|
How do you want to proceed? Just merge this for now or do you want to take a second stab at the dunder methods as part of this PR. I am fine either way. |
|
I think we should merge this first so that it would be easier for anyone(or us) working on #3072 |
|
@EwoutH, what do you think? I am fine either way, and we should not overthink it. |
|
I did a quick search to check the conventions. While Pandas uses I think sticking with |
|
I think I while I really appreciate this effort, I would really like to put of making an (breaking) API change until we have figured out how PropertyLayers are going to fit in the bigger picture. If some changes could be made without changing the API, like by making the implementations behind the current methods more elegant or NumPy native, that could already happen. |
Everything here is backward compatible, so there are no API-breaking changes; just a cleaner and faster alternative API. |
|
We're deprecating stuff, that's basically an announced API break. Just not enforced for now. Also I have my doubts if we should already add another API. While direct nparray access is probably useful in some cases, we might also expose it through the spaces those propertylayers are attached to. |
This is exactly what this PR does: |
|
@codebreaker32, I had a longer in person conversation with @EwoutH. Given the ongoing conversations about spaces in Mesa 4, we decided not to deprecate anything yet. So, if you can remove those warnings, we think this should be good to go. |
|
Thanks for the update, @quaquel. I will go ahead and remove the deprecation warnings. One question: Can we still optimize the existing functions (like |
|
yes, feel free to rewrite them but then also profile the difference |
Yes, this is fine and appreciated. This also could be separate PR(s), that might make testing and reviewing easier. |
edfa173 to
04f91cd
Compare
b492da1 to
80be02f
Compare
Summary of ChangesThis update simplifies the 1. Direct exposure of NumPy data
2. More efficient in-place updates
3. Improved
|
|
I have updated the PR. I initially tried a simple rename from I will change it in subsequent PRs |
|
Can you update the original PR starting post to reflect the final version? I hope to review this later today, and if I don't see anything strange, merge it. |
I am fine with leaving it as |
|
Performance benchmarks:
|
|
@codebreaker32 Thanks for the PR. Could you check if the PR description is now fully up to date with the final implementation in this merged PR? |
|
Yes, the PR description is now up to date |
Summary
Refactors
PropertyLayerto serve as a direct wrapper around the underlying NumPy array, enabling standard NumPy syntax (slicing, broadcasting and methods)Motive
The current
PropertyLayerclass acts as an overly complex wrapper that duplicates native NumPy functionality with custom methods likeset_cellsandmodify_cells. This design:modify_cellsrely onnp.vectorizefor lambdas, which is significantly slower than native ufuncs and broadcasting.model.worldvision, layers should ideally be flexible views on data. The current design tightly couples data to legacy access patterns, acting as an obstacle to the "Single Source of Truth" concept.Implementation
_mesa_datawith a public attributedatadatais now a public NumPy array, enabling slicing, masking, and aggregation without wrappersnp.copytooverhead.set_cellsandmodify_cellsnow operate directly onself.data.modify_cellssemantics