Skip to content

Fix Sorting Vector - [MOD-6783]#5311

Merged
GuyAv46 merged 7 commits intomasterfrom
guyav-MOD-6783
Dec 5, 2024
Merged

Fix Sorting Vector - [MOD-6783]#5311
GuyAv46 merged 7 commits intomasterfrom
guyav-MOD-6783

Conversation

@GuyAv46
Copy link
Collaborator

@GuyAv46 GuyAv46 commented Dec 4, 2024

Refactoring our sortable fields metadata structures

This PR fixes a bug affecting indexes with more than 255 sortable fields.
The issue happened due to storing the length of the sorting vector in an unsigned char, which can only count to 255. We should support 1024 sortable fields so we need at least 10 bits for the length.

This PR also removes the RSSortingTable structure from the spec, as it is very redundant - consumes additional memory and performs string comparisons for info available in the FieldSpec. it is replaced by a counter to track the current number of sortable fields.

Which additional issues this PR fixes

  1. [BUG] Redis Search silently fails to Sort when the index schema is too large #4494

Main objects this PR modified

  1. Deleting sorting table struct
  2. fix the sorting vector issue

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

@codecov
Copy link

codecov bot commented Dec 4, 2024

Codecov Report

Attention: Patch coverage is 80.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 86.72%. Comparing base (c9834e5) to head (5ea17a5).
Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
src/document.c 33.33% 2 Missing ⚠️
src/sortable.c 90.00% 1 Missing ⚠️
src/spec.c 83.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5311      +/-   ##
==========================================
+ Coverage   86.69%   86.72%   +0.02%     
==========================================
  Files         193      193              
  Lines       34764    34704      -60     
==========================================
- Hits        30139    30096      -43     
+ Misses       4625     4608      -17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@GuyAv46 GuyAv46 marked this pull request as ready for review December 4, 2024 19:41
@GuyAv46 GuyAv46 changed the title Fix Sorting Vector MOD-6783 Fix Sorting Vector - [MOD-6783] Dec 5, 2024
Copy link

@DvirDukhan DvirDukhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!
Few comments

@GuyAv46 GuyAv46 added this pull request to the merge queue Dec 5, 2024
Merged via the queue into master with commit 47dd9d0 Dec 5, 2024
@GuyAv46 GuyAv46 deleted the guyav-MOD-6783 branch December 5, 2024 10:34
@redisearch-backport-pull-request
Copy link
Contributor

Backport failed for 2.8, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.8
git worktree add -d .worktree/backport-5311-to-2.8 origin/2.8
cd .worktree/backport-5311-to-2.8
git switch --create backport-5311-to-2.8
git cherry-pick -x 47dd9d05f8be6a56722d627106713d320dd9278b

@redisearch-backport-pull-request
Copy link
Contributor

Backport failed for 2.6, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.6
git worktree add -d .worktree/backport-5311-to-2.6 origin/2.6
cd .worktree/backport-5311-to-2.6
git switch --create backport-5311-to-2.6
git cherry-pick -x 47dd9d05f8be6a56722d627106713d320dd9278b

@redisearch-backport-pull-request
Copy link
Contributor

Backport failed for 2.10, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 2.10
git worktree add -d .worktree/backport-5311-to-2.10 origin/2.10
cd .worktree/backport-5311-to-2.10
git switch --create backport-5311-to-2.10
git cherry-pick -x 47dd9d05f8be6a56722d627106713d320dd9278b

@redisearch-backport-pull-request
Copy link
Contributor

Backport failed for 8.0, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin 8.0
git worktree add -d .worktree/backport-5311-to-8.0 origin/8.0
cd .worktree/backport-5311-to-8.0
git switch --create backport-5311-to-8.0
git cherry-pick -x 47dd9d05f8be6a56722d627106713d320dd9278b

GuyAv46 added a commit that referenced this pull request Dec 5, 2024
* refactor sortable fields

* fix tests

* make more memory efficient

* added a test

* remove unused function

* improve test

* added a comment

(cherry picked from commit 47dd9d0)
GuyAv46 added a commit that referenced this pull request Dec 5, 2024
* refactor sortable fields

* fix tests

* make more memory efficient

* added a test

* remove unused function

* improve test

* added a comment

(cherry picked from commit 47dd9d0)
GuyAv46 added a commit that referenced this pull request Dec 5, 2024
* refactor sortable fields

* fix tests

* make more memory efficient

* added a test

* remove unused function

* improve test

* added a comment

(cherry picked from commit 47dd9d0)
GuyAv46 added a commit that referenced this pull request Dec 5, 2024
* refactor sortable fields

* fix tests

* make more memory efficient

* added a test

* remove unused function

* improve test

* added a comment

(cherry picked from commit 47dd9d0)
github-merge-queue bot pushed a commit that referenced this pull request Dec 5, 2024
Fix Sorting Vector - [MOD-6783] (#5311)

* refactor sortable fields

* fix tests

* make more memory efficient

* added a test

* remove unused function

* improve test

* added a comment

(cherry picked from commit 47dd9d0)
github-merge-queue bot pushed a commit that referenced this pull request Dec 5, 2024
Fix Sorting Vector - [MOD-6783] (#5311)

* refactor sortable fields

* fix tests

* make more memory efficient

* added a test

* remove unused function

* improve test

* added a comment

(cherry picked from commit 47dd9d0)
github-merge-queue bot pushed a commit that referenced this pull request Dec 5, 2024
* Fix Sorting Vector - [MOD-6783] (#5311)

* refactor sortable fields

* fix tests

* make more memory efficient

* added a test

* remove unused function

* improve test

* added a comment

(cherry picked from commit 47dd9d0)

* cleanup
github-merge-queue bot pushed a commit that referenced this pull request Dec 5, 2024
* Fix Sorting Vector - [MOD-6783] (#5311)

* refactor sortable fields

* fix tests

* make more memory efficient

* added a test

* remove unused function

* improve test

* added a comment

(cherry picked from commit 47dd9d0)

* fix for 2.6

* another fix for 2.6

* yet another fix for 2.6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants