Conversation
generall
changed the title
JojiiOfficial
approved these changes
Oct 25, 2024
agourlay
reviewed
Oct 25, 2024
Co-authored-by: Jojii <[email protected]>
Co-authored-by: Jojii <[email protected]>
agourlay
reviewed
Oct 25, 2024
|
|
||
| for (vector_name, vector_config) in &segment_config.vector_data { | ||
| let vector_index_path = get_vector_index_path(temp_dir.path(), vector_name); | ||
| let mut vector_storages_arc = HashMap::new(); |
agourlay
approved these changes
Oct 25, 2024
coszio
reviewed
Oct 25, 2024
| id_tracker: Arc<AtomicRefCell<IdTrackerSS>>, | ||
| /// Vector storages for each field, used for `has_vector` condition | ||
| #[allow(dead_code)] | ||
| vector_storages: HashMap<VectorName, Arc<AtomicRefCell<VectorStorageEnum>>>, |
Contributor
There was a problem hiding this comment.
If the payload index only reads from the vector storage, can't this just be an Arc<VectorStorageEnum>?
Ah, nvm, it is the same refcell that is used in other structs which DO modify it, so removing it does not make sense
timvisee
pushed a commit
that referenced
this pull request
Nov 8, 2024
* include vector storage into struct vector index * implement has_vector * generate schemas * refactor query filter optimizer so avoid too many function arguments * test + fix for sparse vectors * Update lib/segment/src/index/struct_payload_index.rs Co-authored-by: Jojii <[email protected]> * Update lib/segment/src/index/query_optimization/optimizer.rs Co-authored-by: Jojii <[email protected]> * fmt --------- Co-authored-by: Jojii <[email protected]>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related to: #5264 and #2737
Also required for faster selection of samples in Matrix API
This PR includes:
StructPayloadIndexnow havevector_storagesfield.optimize_filterpart of theStructPayloadIndex, so they can access context viaself., which allows to simplify interface significantlyPro tip: review by commits