Skip to content

fix: raise NotImplementedError for asymmetric weights in join counts - #527

Closed
Thanatos9404 wants to merge 1 commit into
pysal:mainfrom
Thanatos9404:fix/join-counts-asymmetric-weights
Closed

Thanatos9404 wants to merge 1 commit into
pysal:mainfrom
Thanatos9404:fix/join-counts-asymmetric-weights

Conversation

@Thanatos9404

Copy link
Copy Markdown

Closes #198

Join counts are only defined for symmetric spatial weights. With asymmetric weights, such as k-nearest neighbors, a black-white join is not the same as a white-black join. All four join count estimators still accept such weights and return a result. For example, with 50 random points and w = KNN.from_array(coords, k=4), Join_Counts(y, w, permutations=0) on main returns bb = 22.5, which includes half a join.

As suggested in the issue, this PR raises a NotImplementedError as soon as asymmetric weights reach any of the join count estimators: Join_Counts, Join_Counts_Local, Join_Counts_Local_BV and Join_Counts_Local_MV. The message is:

Join count statistics are only defined for symmetric spatial weights, but the weights passed are asymmetric (as k-nearest neighbor weights usually are). Make the weights symmetric before computing join counts.

Changes

  • esda/join_counts.py gets a small private helper, _raise_if_asymmetric, which the three local modules import. It works for both W and Graph. Each estimator calls it right after preparing its weights, before computing anything.
  • The helper compares links, not weights:
    • A row standardized lat2W has symmetric links but not symmetric values, and it still passes. test_significance.py already passes a row standardized Voronoi W to Join_Counts, and that test still passes.
    • It also catches a one-way link with zero weight. Join_Counts still counts such a link as a join. On main, adding a zero-weight link from 0 to 5 on the 4 x 4 test lattice gives ww = 10.5. As a Graph, the counts then no longer add up to J.
  • Each estimator's docstring gets one line saying the weights must be symmetric.

The helper compares w.sparse with its transpose. It does not call W.asymmetries or Graph.asymmetry(), because both of those also build the full list of asymmetric pairs. Graph.asymmetry() additionally relabels the ids in a Python loop over every observation, which gets slow on large k-nearest neighbor graphs. Timings on k=6 nearest neighbor weights, with the sparse matrix already built:

n W.asymmetry() Graph.asymmetry() this check (W / Graph)
10,000 0.012 s 0.14 s 0.001 s / 0.001 s
40,000 0.076 s 1.9 s 0.006 s / 0.007 s
100,000 0.34 s 21 s 0.031 s / 0.029 s

For symmetric weights the check is linear in the number of links. On a 200 x 200 rook lattice (40,000 observations, 159,200 links), it takes about 3 ms for a W and 4 ms for a Graph once the sparse matrix exists. Join_Counts already builds w.sparse for s0, and the local estimators use it for the conditional randomization. So the only extra work is building the sparse matrix in two cases: for a Graph in Join_Counts (about 0.016 s here), or for a local estimator run with permutations=0.

If you would rather use w.asymmetries for readability, or warn instead of raising, that is an easy change.

Tests

  • test_join_counts.py, test_ljc.py, test_ljc_bv.py and test_ljc_mv.py each get a test_asymmetric_weights test, with both W and Graph. The weights are k=3 nearest neighbors on a 4 x 4 lattice. Each corner takes the diagonal point as its third neighbor, but that point already has four neighbors at distance 1. So the weights are asymmetric however ties are broken.
  • test_join_counts.py also gets two more tests:
    • test_one_way_zero_weight_link, for both W and Graph.
    • test_transformed_weights, which checks that symmetric weights with the r, d and v transformations still give the same counts.
  • The 10 new raising cases fail on main with DID NOT RAISE and pass with this change. The existing tests and the doctests of the four modules pass unchanged.

Not tested: I ran the four join count test files, test_significance.py, the doctests of the four modules, and ruff check / ruff format --check. That was on Windows with Python 3.14, libpysal 4.15 and numpy 2.5, without numba. I did not run the CI matrix, so the oldest-dependency (libpysal 4.12), numba, Linux and macOS environments are untested. In the full suite, the tests that need matplotlib or geodatasets did not run because I did not install those; none of them use join counts.

Join counts are only defined for symmetric spatial weights. With
asymmetric weights, such as k-nearest neighbors, a black-white join is
not the same as a white-black join, but Join_Counts and the three local
join count estimators accepted them and returned a result anyway.

Add a shared check that compares the links in the weights with their
transpose and raises NotImplementedError when they differ. Only the
links are compared, not their weights, so symmetric weights pass
whatever their transformation. The cost is linear in the number of
links.

Closes pysal#198
@codecov

codecov Bot commented Oct 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.8%. Comparing base (2054740) to head (114ae45).

Additional details and impacted files

Impacted file tree graph

@@          Coverage Diff          @@
##            main    #527   +/-   ##
=====================================
  Coverage   86.8%   86.8%           
=====================================
  Files         29      29           
  Lines       3739    3750   +11     
=====================================
+ Hits        3244    3255   +11     
  Misses       495     495           
Files with missing lines Coverage Δ
esda/join_counts.py 96.2% <100.0%> (+0.2%) ⬆️
esda/join_counts_local.py 98.3% <100.0%> (+0.1%) ⬆️
esda/join_counts_local_bv.py 98.8% <100.0%> (+<0.1%) ⬆️
esda/join_counts_local_mv.py 98.3% <100.0%> (+0.1%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@martinfleis

Copy link
Copy Markdown
Member

Overly verbose AI PR with an overly verbose PR description generated by an LLM. No, thank you.

@Thanatos9404

Copy link
Copy Markdown
Author

Understood, thanks for taking a look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

All join count methods should warn/fail for asymmetric weights

2 participants