[ENH]: Get Collection with segments impl by sanketkedia · Pull Request #6147 · chroma-core/chroma

sanketkedia · 2026-01-11T01:47:47Z

Description of changes

Summarize the changes made by this PR.

Improvements & Bug fixes
- Boring get collection with segments impl with tests
New functionality
- ...

Test plan

How are these changes tested?
Added tests

Tests pass locally with pytest for python, yarn test for js, cargo test for rust

Migration plan

None

Observability plan

None

Documentation Changes

None

github-actions · 2026-01-11T01:48:01Z

sanketkedia · 2026-01-11T01:48:11Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

propel-code-bot · 2026-01-11T01:49:15Z

Implement SysDB get_collection_with_segments feature

Adds full stack support for retrieving a collection together with its segments through the SysDB gRPC service. The feature introduces internal request/response types, backend routing, Spanner query execution, and response conversion, plus test updates and CI adjustments.

The implementation pulls collection, metadata, compaction cursor, and segment data in one Spanner join, reuses existing collection parsing, and constructs a consolidated response for the new GetCollectionWithSegments RPC.

Key Changes

• Defined GetCollectionWithSegmentsRequest / GetCollectionWithSegmentsResponse alongside Assignable and Runnable impls in rust/rust-sysdb/src/types.rs.
• Hooked the new RPC into Backend dispatch (rust/rust-sysdb/src/backend.rs) and gRPC server handling (rust/rust-sysdb/src/server.rs).
• Implemented Spanner-side execution that issues a 4-way join, deduplicates rows, converts them into Collection and Segment structs, and returns the combined payload in rust/rust-sysdb/src/spanner.rs.
• Improved timestamp and JSON parsing error handling during collection/segment hydration and introduced SpannerRowRef helpers in rust/rust-sysdb/src/types.rs.
• Renamed the integration workflow job to test-mcmr-integration in .github/workflows/_rust-tests.yml.

Affected Areas

• rust/rust-sysdb/src/spanner.rs
• rust/rust-sysdb/src/types.rs
• rust/rust-sysdb/src/server.rs
• rust/rust-sysdb/src/backend.rs
• .github/workflows/_rust-tests.yml

This summary was automatically generated by @propel-code-bot

propel-code-bot · 2026-01-11T01:51:02Z

rust/rust-sysdb/src/spanner.rs

+        // - Compaction cursor fields
+        let collection = Collection::try_from(rows)?;
+
+        let segments: Vec<Segment> = segments_map.into_values().collect();


[Maintainability] The segments vector order is non-deterministic because it comes from a HashMap. For API consistency and testability, it's best practice to return lists in a deterministic order (e.g., sorted by segment ID).

Suggested change

let segments: Vec<Segment> = segments_map.into_values().collect();

let mut segments: Vec<Segment> = segments_map.into_values().collect();

segments.sort_by_key(|s| s.id);

Context for Agents

The `segments` vector order is non-deterministic because it comes from a `HashMap`. For API consistency and testability, it's best practice to return lists in a deterministic order (e.g., sorted by segment ID). ```suggestion let mut segments: Vec<Segment> = segments_map.into_values().collect(); segments.sort_by_key(|s| s.id); ``` File: rust/rust-sysdb/src/spanner.rs Line: 932

rust/rust-sysdb/src/spanner.rs

rust/types/src/segment.rs

rust/rust-sysdb/src/types.rs

rust/rust-sysdb/src/spanner.rs

tanujnay112

Accepting with comments

This was referenced Jan 11, 2026

[ENH]: Database and Tenant Crud enhancements #6119

Merged

[ENH]: Spanner collection and segments Schemas #6123

Merged

[ENH]: Create collection impl in rust sysdb #6131

Merged

sanketkedia mentioned this pull request Jan 11, 2026

[ENH]: Get collections impl #6146

Merged

1 task

sanketkedia requested a review from tanujnay112 January 11, 2026 01:48

sanketkedia marked this pull request as ready for review January 11, 2026 01:48

propel-code-bot bot reviewed Jan 11, 2026

View reviewed changes

rust/rust-sysdb/src/spanner.rs Outdated Show resolved Hide resolved

propel-code-bot bot reviewed Jan 11, 2026

View reviewed changes

rust/types/src/segment.rs Outdated Show resolved Hide resolved

sanketkedia force-pushed the 01-10-_enh_get_collection_with_segments_impl branch from 496c205 to 8a070da Compare January 11, 2026 01:59

propel-code-bot bot reviewed Jan 11, 2026

View reviewed changes

rust/types/src/segment.rs Outdated Show resolved Hide resolved

sanketkedia force-pushed the 01-10-_enh_get_collections_impl branch from 75c2da6 to 578c9ec Compare January 11, 2026 21:25

sanketkedia force-pushed the 01-10-_enh_get_collection_with_segments_impl branch from 8a070da to 72e6926 Compare January 11, 2026 21:25

sanketkedia force-pushed the 01-10-_enh_get_collections_impl branch from 578c9ec to 6588983 Compare January 12, 2026 15:49

sanketkedia force-pushed the 01-10-_enh_get_collection_with_segments_impl branch from 72e6926 to 9edc11b Compare January 12, 2026 15:49

sanketkedia force-pushed the 01-10-_enh_get_collections_impl branch from 6588983 to 8929be5 Compare January 12, 2026 16:27

sanketkedia force-pushed the 01-10-_enh_get_collection_with_segments_impl branch from 9edc11b to 92bccd8 Compare January 12, 2026 16:27

sanketkedia force-pushed the 01-10-_enh_get_collections_impl branch from 8929be5 to 29a65f5 Compare January 13, 2026 02:20

sanketkedia force-pushed the 01-10-_enh_get_collection_with_segments_impl branch from 92bccd8 to e6a9329 Compare January 13, 2026 02:21

sanketkedia force-pushed the 01-10-_enh_get_collections_impl branch from 29a65f5 to bf5478d Compare January 13, 2026 17:20

sanketkedia force-pushed the 01-10-_enh_get_collection_with_segments_impl branch from e6a9329 to 44b85fe Compare January 13, 2026 17:21

sanketkedia force-pushed the 01-10-_enh_get_collections_impl branch from bf5478d to e480dbb Compare January 13, 2026 17:25

sanketkedia force-pushed the 01-10-_enh_get_collection_with_segments_impl branch from 44b85fe to 63068d6 Compare January 13, 2026 17:25

propel-code-bot bot reviewed Jan 13, 2026

View reviewed changes

rust/rust-sysdb/src/types.rs Outdated Show resolved Hide resolved

sanketkedia force-pushed the 01-10-_enh_get_collections_impl branch from e480dbb to 3fd3f10 Compare January 13, 2026 17:39

sanketkedia force-pushed the 01-10-_enh_get_collection_with_segments_impl branch from 63068d6 to b3d6cc1 Compare January 13, 2026 17:39

sanketkedia force-pushed the 01-10-_enh_get_collections_impl branch from 3fd3f10 to 7e28350 Compare January 13, 2026 23:40

sanketkedia force-pushed the 01-10-_enh_get_collection_with_segments_impl branch from b3d6cc1 to 67038ee Compare January 13, 2026 23:40

sanketkedia force-pushed the 01-10-_enh_get_collections_impl branch 2 times, most recently from d2ac304 to c35ebab Compare January 14, 2026 02:57

sanketkedia force-pushed the 01-10-_enh_get_collection_with_segments_impl branch from 67038ee to 1eca196 Compare January 14, 2026 02:57

sanketkedia force-pushed the 01-10-_enh_get_collections_impl branch from c35ebab to 0ca022f Compare January 14, 2026 03:01

sanketkedia force-pushed the 01-10-_enh_get_collection_with_segments_impl branch 2 times, most recently from aa7bf42 to 40e9e5c Compare January 14, 2026 08:25

propel-code-bot bot reviewed Jan 14, 2026

View reviewed changes

rust/rust-sysdb/src/types.rs Outdated Show resolved Hide resolved

tanujnay112 reviewed Jan 14, 2026

View reviewed changes

rust/rust-sysdb/src/spanner.rs Outdated Show resolved Hide resolved

sanketkedia force-pushed the 01-10-_enh_get_collection_with_segments_impl branch from 5890dce to 8554390 Compare January 15, 2026 00:16

tanujnay112 approved these changes Jan 15, 2026

View reviewed changes

sanketkedia changed the base branch from 01-10-_enh_get_collections_impl to graphite-base/6147 January 15, 2026 01:01

sanketkedia added 6 commits January 14, 2026 17:05

[ENH]: Get Collection with segments impl

f5d5d86

Merge conflicts

a127c50

add more tests

9a845d0

lots of tests

b35303c

review comments

67d5296

rename test harness name from rust sysdb to mcmr

849806c

sanketkedia force-pushed the 01-10-_enh_get_collection_with_segments_impl branch from 641be45 to 849806c Compare January 15, 2026 01:05

sanketkedia changed the base branch from graphite-base/6147 to main January 15, 2026 01:05

sanketkedia enabled auto-merge (squash) January 15, 2026 01:24

dummy commit

efed803

sanketkedia merged commit a0ad296 into main Jan 15, 2026
188 of 194 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ENH]: Get Collection with segments impl#6147

[ENH]: Get Collection with segments impl#6147
sanketkedia merged 7 commits intomainfrom
01-10-_enh_get_collection_with_segments_impl

sanketkedia commented Jan 11, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Jan 11, 2026

Uh oh!

sanketkedia commented Jan 11, 2026 •

edited

Loading

Uh oh!

propel-code-bot bot commented Jan 11, 2026 •

edited

Loading

Uh oh!

propel-code-bot bot Jan 11, 2026

Uh oh!

sanketkedia Jan 15, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tanujnay112 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	let segments: Vec<Segment> = segments_map.into_values().collect();
	let mut segments: Vec<Segment> = segments_map.into_values().collect();
	segments.sort_by_key(\|s\| s.id);

Conversation

sanketkedia commented Jan 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Test plan

Migration plan

Observability plan

Documentation Changes

Uh oh!

github-actions bot commented Jan 11, 2026

Reviewer Checklist

Testing, Bugs, Errors, Logs, Documentation

System Compatibility

Quality

Uh oh!

sanketkedia commented Jan 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

propel-code-bot bot commented Jan 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

propel-code-bot bot Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

sanketkedia Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tanujnay112 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sanketkedia commented Jan 11, 2026 •

edited

Loading

sanketkedia commented Jan 11, 2026 •

edited

Loading

propel-code-bot bot commented Jan 11, 2026 •

edited

Loading