Skip to content

Delete the requested rows when ANN ids are a NumPy array - #1303

Merged
davidmezzetti merged 1 commit into
neuml:masterfrom
kartsan03:fix/ann-delete-numpy-ids
Sep 30, 2026
Merged

davidmezzetti merged 1 commit into
neuml:masterfrom
kartsan03:fix/ann-delete-numpy-ids

Conversation

@kartsan03

Copy link
Copy Markdown
Contributor

Deleting from a dense ANN directly with ids in a NumPy array removed the wrong rows in SQLite: sqlite3 binds NumPy integers as blobs, so delete(np.array([3, 4])) deleted row 0 and kept rows 3 and 4. PGVector deleted nothing for the same call, and Milvus and Zvec raised ValueError from their if ids: guard.

Bind ids as int in the SQLite and PGVector deletes, and check len(ids) in the Milvus and Zvec guards, following #1241. Add one regression method in TestDense that deletes with a NumPy array in the four backends and checks the count and which ids search still returns.

Fixes #1302.

Validation

  • On unchanged master the new test fails for all four: Milvus and Zvec raise ValueError, SQLite ends with 9 rows and PGVector with 10 where 8 are expected. It passes after the fix.
  • testann.testdense and testann.testsparse: 58 tests pass.
  • Black, and Pylint with the repository's pre-commit options (10.00/10), pass.
  • Linux, Python 3.11. The full model/integration suite was not run.

@davidmezzetti

Copy link
Copy Markdown
Member

Please sync your fork and merge in tests into the new structure.

@kartsan03
kartsan03 force-pushed the fix/ann-delete-numpy-ids branch 2 times, most recently from 5bc0933 to 6765330 Compare September 30, 2026 15:26
@kartsan03

Copy link
Copy Markdown
Contributor Author

Done. I rebased onto master (6765330) and moved the tests into the new layout, so the TestDense method from the description is gone:

  • testdense/base.py: a deletenumpy() helper. It builds a 10-row index, deletes np.array([3, 4]), and checks the count and the ids search still returns.
  • testSQLite, testZvec and testMilvus each call it after runTests().
  • testPGVector: after the existing delete([0]), it deletes the remaining row with np.array([1]) and checks the count is 0.

The 4 tests pass on the branch. With the source files from master they fail: Milvus and Zvec raise ValueError, SQLite keeps 9 rows, and PGVector keeps 1. All of testann passes (60 tests), and black is clean.

@davidmezzetti

Copy link
Copy Markdown
Member

LGTM - Thank you!

@davidmezzetti
davidmezzetti merged commit 2d535ff into neuml:master Sep 30, 2026
4 checks passed
@davidmezzetti davidmezzetti added the bug Something isn't working label Sep 30, 2026
@davidmezzetti davidmezzetti added this to the v9.14.0 milestone Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ANN delete with a NumPy id array removes the wrong rows in SQLite, and does nothing or fails in PGVector, Milvus and Zvec

2 participants