[ENH] Wire up quantized reader in new orchestrator by Sicheng-Pan · Pull Request #6409 · chroma-core/chroma

Sicheng-Pan · 2026-02-11T20:56:52Z

Description of changes

Summarize the changes made by this PR.

Improvements & Bug fixes
- N/A
New functionality
- Wire up the quantized reader in a new quantized spann orchestrator.

Test plan

How are these changes tested?

Tests pass locally with pytest for python, yarn test for js, cargo test for rust

Migration plan

Are there any migrations, or any forwards/backwards compatibility changes needed in order to make sure this change deploys reliably?

Observability plan

What is the plan to instrument and monitor this change?

Documentation Changes

Are all docstrings for user-facing APIs updated if required? Do we need to make documentation changes in the docs section?

github-actions · 2026-02-11T20:56:59Z

Sicheng-Pan · 2026-02-11T20:57:06Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

propel-code-bot · 2026-02-12T01:35:31Z

Quantized SPANN KNN orchestrator with load/merge pipeline wiring

Adds a dedicated quantized SPANN KNN orchestrator and operator set that load the quantized reader, navigate cluster centers, fetch clusters, and run a bruteforce step before merging with log-derived distances. The segment reader API was expanded with get_cluster, get_version, and distance-function access so that workers can materialize cluster payloads, and the shared Merge operator now deduplicates results via hashing to support the quantized flow. Worker request routing now selects QuantizedSpannKnnOrchestrator whenever the vector segment type is QuantizedSpann, and knn filter/orchestrator error enums were extended to surface the new operator failures.

Key Changes

• Introduced quantized_spann_knn orchestrator that sequences knn_log, center search/load, per-cluster load, bruteforce, and merge tasks with state tracking for readers, rotated queries, and active brute-force count
• Added QuantizedSpannLoadCenter, QuantizedSpannCenterSearch, QuantizedSpannLoadCluster, and QuantizedSpannBruteforce operators plus supporting structs and errors
• Extended QuantizedSpannSegmentReader with get_cluster, get_version, and distance_function helpers (replacing in-reader bruteforce) and removed the previous asynchronous bruteforce API
• Updated worker server orchestration to dispatch the quantized orchestrator for SegmentType::QuantizedSpann requests and tightened KnnFilter error taxonomy
• Changed operator::Merge to require Clone + Eq + Hash + Ord, deduplicate records via a HashSet, and added tests covering duplicate suppression

Possible Issues

• No automated tests cover the new quantized orchestration path, so regressions (e.g., misordered pipeline, dedup edge cases) will go unnoticed
• QuantizedSpannLoadClusterOperator fails entire queries when any ID lacks a version record, which is stricter than the previous behavior and may surface in production with WAL-only or deleted points
• initial_tasks still awaits read_quantized_usearch before returning, meaning no work is queued until the full reader has loaded, hurting concurrency on large segments

This summary was automatically generated by @propel-code-bot

rust/worker/src/execution/orchestration/quantized_spann_knn.rs

rust/worker/src/execution/operators/quantized_spann_navigate.rs

rust/worker/src/execution/orchestration/quantized_spann_knn.rs

Sicheng-Pan · 2026-02-12T23:56:12Z

rust/worker/src/execution/orchestration/quantized_spann_knn.rs

+
+    // State tracking.
+    num_bruteforces: Option<usize>,
+    records: Vec<Vec<RecordMeasure>>,


TODO: bruteforce_results

shall we knock this rename out?

rust/worker/src/execution/orchestration/quantized_spann_knn.rs

rust/worker/src/execution/operators/quantized_spann_load_cluster.rs

rust/segment/src/quantized_spann.rs

propel-code-bot · 2026-02-14T02:35:29Z

rust/worker/src/execution/operators/quantized_spann_load_cluster.rs

+        let versions =
+            try_join_all(cluster.ids.iter().map(|&id| self.reader.get_version(id))).await?;
+
+        let global_versions = cluster
+            .ids
+            .iter()
+            .copied()
+            .zip(versions)
+            .collect::<HashMap<_, _>>();


[Logic] The new load‑cluster path now assumes every ID in the cluster has an associated version row. try_join_all(cluster.ids.iter().map(|&id| self.reader.get_version(id))) will now return an error as soon as one get_version call returns None, which bubbles up as a QuantizedSpannLoadClusterError and aborts the entire query. In the previous in‑reader bruteforce implementation we deliberately tolerated missing versions by flattening them away (future::try_join_all(...).await?.into_iter().flatten()), so stale WAL entries or partially compacted points were simply skipped instead of failing the request. This regression means a single dangling ID can now make every quantized query return 500.

Please make get_version return an Option<u32> (or interpret the "version not found" case as a skip) and only insert IDs whose version is present when building global_versions, restoring the old behavior of ignoring stale points rather than treating them as fatal.

Context for Agents

The new load‑cluster path now assumes every ID in the cluster has an associated version row. `try_join_all(cluster.ids.iter().map(|&id| self.reader.get_version(id)))` will now return an error as soon as one `get_version` call returns `None`, which bubbles up as a `QuantizedSpannLoadClusterError` and aborts the entire query. In the previous in‑reader `bruteforce` implementation we deliberately tolerated missing versions by flattening them away (`future::try_join_all(...).await?.into_iter().flatten()`), so stale WAL entries or partially compacted points were simply skipped instead of failing the request. This regression means a single dangling ID can now make every quantized query return `500`. Please make `get_version` return an `Option<u32>` (or interpret the "version not found" case as a skip) and only insert IDs whose version is present when building `global_versions`, restoring the old behavior of ignoring stale points rather than treating them as fatal. File: rust/worker/src/execution/operators/quantized_spann_load_cluster.rs Line: 61

Sicheng-Pan · 2026-02-14T03:39:07Z

Merge activity

Feb 14, 3:39 AM UTC: A user started a stack merge that includes this pull request via Graphite.
Feb 14, 3:41 AM UTC: Graphite rebased this pull request as part of a merge.
Feb 14, 4:15 AM UTC: @Sicheng-Pan merged this pull request with Graphite.

- **[ENH]: Cache rust git submodules in mounted volume (#6424)** - **[CHORE](k8s) increase dev CPU limits from 100m to 200-300m (#6435)** - **[ENH] replace live cloud tests with k8s integration tests (#6434)** - **[ENH] Make dirty_log_collections metric mcmr-aware. (#6353)** - **[ENH] Quantized Spann Segment Writer (#6397)** - **[ENH] Wire up quantized writer in compaction (#6399)** - **[ENH] Quantized Spann Segment Reader (#6405)** - **[ENH] Wire up quantized reader in new orchestrator (#6409)** - **[ENH] Garbage collect usearch index files (#6416)** - **[ENH] Trace quantized spann implementation (#6425)** - **[ENH]: Precompute data chunk len() (#6442)** - **[BUG]: Compaction version file flush was incomplete on MCMR (#6423)** - **[DOC]: Fixed broken links in Readme (#6440)** - **[DOC] Fix link to Rust documentation (#6443)** - **[ENH]: Allow users to disable FTS in schema (#6214)** --------- Co-authored-by: Robert Escriva <[email protected]> Co-authored-by: Macronova <[email protected]> Co-authored-by: Nilpotent <[email protected]> Co-authored-by: anderk222 <[email protected]> Co-authored-by: Sanket Kedia <[email protected]>

This was referenced Feb 11, 2026

[ENH] Quantized Spann Segment Writer #6397

Merged

[ENH] Wire up quantized writer in compaction #6399

Merged

Sicheng-Pan mentioned this pull request Feb 11, 2026

[ENH] Quantized Spann Segment Reader #6405

Merged

1 task

Sicheng-Pan force-pushed the 02-10-_enh_quantized_spann_segment_reader branch from ff6baf8 to 9a26a8f Compare February 11, 2026 21:07

Sicheng-Pan force-pushed the 02-11-_enh_wire_up_quantized_reader_in_new_orchestrator branch 2 times, most recently from be396bb to 06f50fa Compare February 11, 2026 22:35

This comment has been minimized.

Sign in to view

Sicheng-Pan force-pushed the 02-11-_enh_wire_up_quantized_reader_in_new_orchestrator branch from 06f50fa to 50beaff Compare February 12, 2026 01:32

Sicheng-Pan marked this pull request as ready for review February 12, 2026 01:34

Sicheng-Pan force-pushed the 02-10-_enh_quantized_spann_segment_reader branch from 9a26a8f to a601921 Compare February 12, 2026 01:40

Sicheng-Pan force-pushed the 02-11-_enh_wire_up_quantized_reader_in_new_orchestrator branch 2 times, most recently from 3f3ff93 to ce47c0b Compare February 12, 2026 01:41

This comment has been minimized.

Sign in to view

This was referenced Feb 12, 2026

[ENH] Garbage collect usearch index files #6416

Merged

[ENH] Trace quantized spann implementation #6425

Merged

Sicheng-Pan force-pushed the 02-10-_enh_quantized_spann_segment_reader branch from 80a5af5 to 514208a Compare February 12, 2026 21:27

Sicheng-Pan force-pushed the 02-11-_enh_wire_up_quantized_reader_in_new_orchestrator branch 2 times, most recently from 2acf8f0 to 588f737 Compare February 12, 2026 21:33

Sicheng-Pan force-pushed the 02-10-_enh_quantized_spann_segment_reader branch from 514208a to 939cc61 Compare February 12, 2026 21:33

propel-code-bot bot reviewed Feb 12, 2026

View reviewed changes

rust/worker/src/execution/orchestration/quantized_spann_knn.rs Show resolved Hide resolved

Sicheng-Pan commented Feb 12, 2026

View reviewed changes

rust/worker/src/execution/operators/quantized_spann_navigate.rs Outdated Show resolved Hide resolved

Sicheng-Pan commented Feb 12, 2026

View reviewed changes

rust/worker/src/execution/orchestration/quantized_spann_knn.rs Show resolved Hide resolved

Sicheng-Pan commented Feb 12, 2026

View reviewed changes

rust/worker/src/execution/orchestration/quantized_spann_knn.rs Outdated Show resolved Hide resolved

Sicheng-Pan force-pushed the 02-10-_enh_quantized_spann_segment_reader branch from 939cc61 to 833b8cc Compare February 13, 2026 00:43

Sicheng-Pan force-pushed the 02-11-_enh_wire_up_quantized_reader_in_new_orchestrator branch 2 times, most recently from b64d5d2 to 9debc15 Compare February 13, 2026 01:39

propel-code-bot bot reviewed Feb 13, 2026

View reviewed changes

rust/worker/src/execution/orchestration/quantized_spann_knn.rs Outdated Show resolved Hide resolved

propel-code-bot bot reviewed Feb 13, 2026

View reviewed changes

rust/worker/src/execution/operators/quantized_spann_load_cluster.rs Outdated Show resolved Hide resolved

Sicheng-Pan force-pushed the 02-11-_enh_wire_up_quantized_reader_in_new_orchestrator branch from 96c6e0e to 1a1b6e8 Compare February 13, 2026 19:19

HammadB approved these changes Feb 13, 2026

View reviewed changes

Sicheng-Pan force-pushed the 02-11-_enh_wire_up_quantized_reader_in_new_orchestrator branch 2 times, most recently from 7cefa67 to 8f4b1ec Compare February 14, 2026 00:09

propel-code-bot bot reviewed Feb 14, 2026

View reviewed changes

rust/segment/src/quantized_spann.rs Show resolved Hide resolved

Sicheng-Pan changed the base branch from 02-10-_enh_quantized_spann_segment_reader to graphite-base/6409 February 14, 2026 02:31

propel-code-bot bot reviewed Feb 14, 2026

View reviewed changes

Sicheng-Pan changed the base branch from graphite-base/6409 to main February 14, 2026 03:39

Sicheng-Pan added 5 commits February 14, 2026 03:40

[ENH] Wire up quantized reader in new orchestrator

6dccce5

Update merge impl

9b7db8a

Address comments

3597437

Pipelining

41f258a

Avoid data loading in CPU opeartor

dafbf0f

Sicheng-Pan force-pushed the 02-11-_enh_wire_up_quantized_reader_in_new_orchestrator branch from 8f4b1ec to dafbf0f Compare February 14, 2026 03:40

Sicheng-Pan merged commit 5a1ef3f into main Feb 14, 2026
67 checks passed

tanujnay112 mentioned this pull request Feb 18, 2026

[CHORE]: fast forward rc/2026-02-13 to 3cb097601ddd379ab109456727a926e593e43a5c #6456

Merged

Sicheng-Pan deleted the 02-11-_enh_wire_up_quantized_reader_in_new_orchestrator branch February 25, 2026 23:47

Conversation

Sicheng-Pan commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Test plan

Migration plan

Observability plan

Documentation Changes

Uh oh!

github-actions bot commented Feb 11, 2026

Reviewer Checklist

Testing, Bugs, Errors, Logs, Documentation

System Compatibility

Quality

Uh oh!

Sicheng-Pan commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

propel-code-bot bot commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

Sicheng-Pan Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

HammadB Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Sicheng-Pan Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

propel-code-bot bot Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

Sicheng-Pan commented Feb 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge activity

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Sicheng-Pan commented Feb 11, 2026 •

edited

Loading

Sicheng-Pan commented Feb 11, 2026 •

edited

Loading

propel-code-bot bot commented Feb 12, 2026 •

edited

Loading

Sicheng-Pan commented Feb 14, 2026 •

edited

Loading