Fix MRIterator ownership mechanism - [MOD-8108]#5244
Merged
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #5244 +/- ##
==========================================
+ Coverage 86.60% 86.71% +0.11%
==========================================
Files 193 193
Lines 34772 34671 -101
==========================================
- Hits 30113 30065 -48
+ Misses 4659 4606 -53 ☔ View full report in Codecov by Sentry. |
GuyAv46
changed the title
Contributor
Automated performance analysis summaryThis comment was automatically generated given there is performance data available. In summary:
You can check a comparison in detail via the grafana link Comparison between master and guyav-fix_cursor_deadlock.Time Period from 30 days ago. (environment used: oss-standalone)
|
raz-mon
reviewed
Nov 25, 2024
Collaborator
raz-mon
left a comment
raz-mon
left a comment
There was a problem hiding this comment.
AWESOME 🔥
Some small comments
raz-mon
approved these changes
Nov 26, 2024
Collaborator
raz-mon
left a comment
raz-mon
left a comment
There was a problem hiding this comment.
💣
Consider naming the blocking variable block instead of wait, a bit more intuitive.
Great addition!
alonre24
reviewed
Nov 26, 2024
alonre24
approved these changes
Nov 27, 2024
Contributor
|
Backport failed for Please cherry-pick the changes locally and resolve any conflicts. git fetch origin 2.8
git worktree add -d .worktree/backport-5244-to-2.8 origin/2.8
cd .worktree/backport-5244-to-2.8
git switch --create backport-5244-to-2.8
git cherry-pick -x 6f68d69d7688807bad5b89eb53673aaaff36c9c9 |
Contributor
|
Backport failed for Please cherry-pick the changes locally and resolve any conflicts. git fetch origin 2.6
git worktree add -d .worktree/backport-5244-to-2.6 origin/2.6
cd .worktree/backport-5244-to-2.6
git switch --create backport-5244-to-2.6
git cherry-pick -x 6f68d69d7688807bad5b89eb53673aaaff36c9c9 |
redisearch-backport-pull-request bot
pushed a commit
that referenced
this pull request
Nov 27, 2024
* implement release mechanism for MRIterator * refactor MRChannel * delete CursorList_Expire * fix test for async cursor deletion * minor cleanup * more cleanup * remove a log message that is now more likely to be logged * log about stuck RQ less frequently * even less noisy * rename MRChannel_Close (cherry picked from commit 6f68d69)
Contributor
|
Successfully created backport PR for |
redisearch-backport-pull-request bot
pushed a commit
that referenced
this pull request
Nov 27, 2024
* implement release mechanism for MRIterator * refactor MRChannel * delete CursorList_Expire * fix test for async cursor deletion * minor cleanup * more cleanup * remove a log message that is now more likely to be logged * log about stuck RQ less frequently * even less noisy * rename MRChannel_Close (cherry picked from commit 6f68d69)
Contributor
|
Successfully created backport PR for |
2 tasks
GuyAv46
added a commit
that referenced
this pull request
Nov 27, 2024
* implement release mechanism for MRIterator * refactor MRChannel * delete CursorList_Expire * fix test for async cursor deletion * minor cleanup * more cleanup * remove a log message that is now more likely to be logged * log about stuck RQ less frequently * even less noisy * rename MRChannel_Close (cherry picked from commit 6f68d69)
GuyAv46
added a commit
that referenced
this pull request
Nov 27, 2024
* implement release mechanism for MRIterator * refactor MRChannel * delete CursorList_Expire * fix test for async cursor deletion * minor cleanup * more cleanup * remove a log message that is now more likely to be logged * log about stuck RQ less frequently * even less noisy * rename MRChannel_Close (cherry picked from commit 6f68d69)
This was referenced Nov 27, 2024
github-merge-queue bot
pushed a commit
that referenced
this pull request
Nov 27, 2024
* Fix MRIterator ownership mechanism - [MOD-8108] (#5244) * implement release mechanism for MRIterator * refactor MRChannel * delete CursorList_Expire * fix test for async cursor deletion * minor cleanup * more cleanup * remove a log message that is now more likely to be logged * log about stuck RQ less frequently * even less noisy * rename MRChannel_Close (cherry picked from commit 6f68d69) * Fix a potential use after free - [MOD-8108] (#5268) fix a potential use after free --------- Co-authored-by: GuyAv46 <[email protected]>
github-merge-queue bot
pushed a commit
that referenced
this pull request
Nov 27, 2024
* Fix MRIterator ownership mechanism - [MOD-8108] (#5244) * implement release mechanism for MRIterator * refactor MRChannel * delete CursorList_Expire * fix test for async cursor deletion * minor cleanup * more cleanup * remove a log message that is now more likely to be logged * log about stuck RQ less frequently * even less noisy * rename MRChannel_Close (cherry picked from commit 6f68d69) * Fix a potential use after free - [MOD-8108] (#5268) fix a potential use after free --------- Co-authored-by: GuyAv46 <[email protected]>
github-merge-queue bot
pushed a commit
that referenced
this pull request
Nov 28, 2024
* Fix MRIterator ownership mechanism - [MOD-8108] (#5244) * implement release mechanism for MRIterator * refactor MRChannel * delete CursorList_Expire * fix test for async cursor deletion * minor cleanup * more cleanup * remove a log message that is now more likely to be logged * log about stuck RQ less frequently * even less noisy * rename MRChannel_Close (cherry picked from commit 6f68d69) * additional fixes for 2.6
github-merge-queue bot
pushed a commit
that referenced
this pull request
Nov 28, 2024
* Fix MRIterator ownership mechanism - [MOD-8108] (#5244) * implement release mechanism for MRIterator * refactor MRChannel * delete CursorList_Expire * fix test for async cursor deletion * minor cleanup * more cleanup * remove a log message that is now more likely to be logged * log about stuck RQ less frequently * even less noisy * rename MRChannel_Close (cherry picked from commit 6f68d69) * additional fixes for 2.8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implement a referencing mechanism between the reader and writers of an
MRIterator, removing the need to wait for replies to arrive before releasing the iterator.This PR fixes a potential deadlock when using cursors in cluster mode, and sometimes speeds up the freeing flow of an
FT.AGGREGATEcommand (no need to sync with the shards).Main objects this PR modified
MRIteratornow has a simple referencing mechanism (using a boolean flag). When both the reader and the writers give up the iterator, the last one to do so frees it. In the case of pending cursors,FT.CURSOR DELcommands will be sent and the iterator will free itself when all the replies return, without blocking the reader.MRChannel, so there is no waiting for it to be closed anymore. It is also possible to push new items to it after it is closed, soMRChannel_Pushcannot fail. The channel and its pending items will be freed when the iterator is freed.CursorList_Expire. The mechanism is no longer needed and we can now free the coordinator's cursors without worrying about deadlocks.#5268 is a follow-up of this PR.
Mark if applicable