Skip to content

Fix the missing server.dirty increment and redundant signalModifiedKey in serveClientBlockedOnList#11326

Merged
oranagra merged 2 commits intoredis:unstablefrom
sundb:fix_dirty_and_signal
Sep 28, 2022
Merged

Fix the missing server.dirty increment and redundant signalModifiedKey in serveClientBlockedOnList#11326
oranagra merged 2 commits intoredis:unstablefrom
sundb:fix_dirty_and_signal

Conversation

@sundb
Copy link
Collaborator

@sundb sundb commented Sep 27, 2022

Mainly fix two minor bug

  1. When handle BL*POP/BLMOVE commands with blocked clients, we should increment server.dirty.
  2. listPopRangeAndReplyWithKey() in serveClientBlockedOnList() should not repeat calling signalModifiedKey() which has been called when an element was pushed on the list.

Other optimization
add signal param for listElementsRemoved to skip signalModifiedKey() to unify all pop operation.

Unifying all pop operations also prepares us for #11303, so that we can avoid having to deal with the conversion from quicklist to listpack() in various places when the list shrinks.

@sundb sundb requested a review from enjoy-binbin September 27, 2022 08:18
@sundb sundb changed the title Fix the missing server.dirty increment and redundant signalM odifiedKey in serveClientBlockedOnList Fix the missing server.dirty increment and redundant signalModifiedKey in serveClientBlockedOnList Sep 27, 2022
@oranagra
Copy link
Member

why is it wrong (or bad) to signalModifiedKey twice?
if you execute LPUSH and BLPOP in a pipeline, it'll do it twice, so why not when POP was blocked and then unblocked by a PUSH?

@sundb
Copy link
Collaborator Author

sundb commented Sep 28, 2022

why is it wrong (or bad) to signalModifiedKey twice?

It can only be considered redundant.

if you execute LPUSH and BLPOP in a pipeline, it'll do it twice,

This is equivalent to LPUSH and LPOP, LPOP has no way to know that signalModifiedKey() has been sent in LPUSH.

so why not when POP was blocked and then unblocked by a PUSH?

When we enter serveClientsBlockedOnListKey(), we already know that the signalModifiedKey() was called when some elements were inserted into this list.

@sundb
Copy link
Collaborator Author

sundb commented Sep 28, 2022

@oranagra I missed something.
when POP was blocked and then unblocked by a PUSH, if we call signalModifiedKey() twice, the second will be NOP, because the first signalModifiedKey() will unwatch all keys in touchWatchedKey() and remove this key from TrackingTable in trackingInvalidateKey().

@oranagra
Copy link
Member

ok, but still i wonder, considering it doesn't do any harm, maybe we can keep it as a statement (can be useful if the code is copied someday)

@sundb
Copy link
Collaborator Author

sundb commented Sep 28, 2022

@oranagra Not sure if you missed something.
In the old serveClientBlockedOnList() code, we would call signalModifiedKey() twice just for the blmpop command, but not for the other commands.
Not sure if we need to unify them.

@oranagra
Copy link
Member

ohh, i see.. in all blocking POP / MOVE commands, we had skipped this pop-driven signal with a comment that the push had done that. and the 'm' variant (blmpop) was the odd one.

Copy link
Member

@oranagra oranagra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so we have 3 types of changes:

  1. cleanup for sharing code.
  2. remove a redundant signalModifiedKey with no implications
  3. fix a missing dirty counter increment.

i'll argue that there's no reason to mention the dirty counter fix in the release notes, since the command did increment it, and the bug is that it incremented it once instead of twice, probably not a big issue)

@oranagra oranagra merged commit f106bee into redis:unstable Sep 28, 2022
@sundb sundb deleted the fix_dirty_and_signal branch September 29, 2022 01:59
@enjoy-binbin
Copy link
Contributor

thanks, i haven't had time to look at this yet... but the 3 types of changes LGTM, i will catch up the code later in person

madolson pushed a commit to madolson/redis that referenced this pull request Apr 19, 2023
…y in serveClientBlockedOnList (redis#11326)

Mainly fix two minor bug
1. When handle BL*POP/BLMOVE commands with blocked clients, we should increment server.dirty.
2.  `listPopRangeAndReplyWithKey()` in `serveClientBlockedOnList()` should not repeat calling
   `signalModifiedKey()` which has been called when an element was pushed on the list.
   (was skipped in all bpop commands, other than blmpop) 

Other optimization
add `signal` param for `listElementsRemoved` to skip `signalModifiedKey()` to unify all pop operation.

Unifying all pop operations also prepares us for redis#11303, so that we can avoid having to deal with the
conversion from quicklist to listpack() in various places when the list shrinks.
enjoy-binbin pushed a commit to enjoy-binbin/redis that referenced this pull request Jul 31, 2023
…y in serveClientBlockedOnList (redis#11326)

Mainly fix two minor bug
1. When handle BL*POP/BLMOVE commands with blocked clients, we should increment server.dirty.
2.  `listPopRangeAndReplyWithKey()` in `serveClientBlockedOnList()` should not repeat calling
   `signalModifiedKey()` which has been called when an element was pushed on the list.
   (was skipped in all bpop commands, other than blmpop) 

Other optimization
add `signal` param for `listElementsRemoved` to skip `signalModifiedKey()` to unify all pop operation.

Unifying all pop operations also prepares us for redis#11303, so that we can avoid having to deal with the
conversion from quicklist to listpack() in various places when the list shrinks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants