Repository navigation
fix: raise NotImplementedError for asymmetric weights in join counts - #527
Closed
Thanatos9404 wants to merge 1 commit into
Closed
Thanatos9404 wants to merge 1 commit into
Thanatos9404 wants to merge 1 commit into
Conversation
Join counts are only defined for symmetric spatial weights. With asymmetric weights, such as k-nearest neighbors, a black-white join is not the same as a white-black join, but Join_Counts and the three local join count estimators accepted them and returned a result anyway. Add a shared check that compares the links in the weights with their transpose and raises NotImplementedError when they differ. Only the links are compared, not their weights, so symmetric weights pass whatever their transformation. The cost is linear in the number of links. Closes pysal#198
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #527 +/- ##
=====================================
Coverage 86.8% 86.8%
=====================================
Files 29 29
Lines 3739 3750 +11
=====================================
+ Hits 3244 3255 +11
Misses 495 495
🚀 New features to boost your workflow:
|
Member
|
Overly verbose AI PR with an overly verbose PR description generated by an LLM. No, thank you. |
Author
|
Understood, thanks for taking a look. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #198
Join counts are only defined for symmetric spatial weights. With asymmetric weights, such as k-nearest neighbors, a black-white join is not the same as a white-black join. All four join count estimators still accept such weights and return a result. For example, with 50 random points and
w = KNN.from_array(coords, k=4),Join_Counts(y, w, permutations=0)onmainreturnsbb = 22.5, which includes half a join.As suggested in the issue, this PR raises a
NotImplementedErroras soon as asymmetric weights reach any of the join count estimators:Join_Counts,Join_Counts_Local,Join_Counts_Local_BVandJoin_Counts_Local_MV. The message is:Changes
esda/join_counts.pygets a small private helper,_raise_if_asymmetric, which the three local modules import. It works for bothWandGraph. Each estimator calls it right after preparing its weights, before computing anything.lat2Whas symmetric links but not symmetric values, and it still passes.test_significance.pyalready passes a row standardizedVoronoiW toJoin_Counts, and that test still passes.Join_Countsstill counts such a link as a join. Onmain, adding a zero-weight link from 0 to 5 on the 4 x 4 test lattice givesww = 10.5. As aGraph, the counts then no longer add up toJ.The helper compares
w.sparsewith its transpose. It does not callW.asymmetriesorGraph.asymmetry(), because both of those also build the full list of asymmetric pairs.Graph.asymmetry()additionally relabels the ids in a Python loop over every observation, which gets slow on large k-nearest neighbor graphs. Timings on k=6 nearest neighbor weights, with the sparse matrix already built:W.asymmetry()Graph.asymmetry()For symmetric weights the check is linear in the number of links. On a 200 x 200 rook lattice (40,000 observations, 159,200 links), it takes about 3 ms for a W and 4 ms for a Graph once the sparse matrix exists.
Join_Countsalready buildsw.sparsefors0, and the local estimators use it for the conditional randomization. So the only extra work is building the sparse matrix in two cases: for aGraphinJoin_Counts(about 0.016 s here), or for a local estimator run withpermutations=0.If you would rather use
w.asymmetriesfor readability, or warn instead of raising, that is an easy change.Tests
test_join_counts.py,test_ljc.py,test_ljc_bv.pyandtest_ljc_mv.pyeach get atest_asymmetric_weightstest, with bothWandGraph. The weights are k=3 nearest neighbors on a 4 x 4 lattice. Each corner takes the diagonal point as its third neighbor, but that point already has four neighbors at distance 1. So the weights are asymmetric however ties are broken.test_join_counts.pyalso gets two more tests:test_one_way_zero_weight_link, for bothWandGraph.test_transformed_weights, which checks that symmetric weights with ther,dandvtransformations still give the same counts.mainwithDID NOT RAISEand pass with this change. The existing tests and the doctests of the four modules pass unchanged.Not tested: I ran the four join count test files,
test_significance.py, the doctests of the four modules, andruff check/ruff format --check. That was on Windows with Python 3.14, libpysal 4.15 and numpy 2.5, without numba. I did not run the CI matrix, so the oldest-dependency (libpysal 4.12), numba, Linux and macOS environments are untested. In the full suite, the tests that need matplotlib or geodatasets did not run because I did not install those; none of them use join counts.