Fixed SET and BITFIELD commands being wrongly marked movablekeys#10837
Fixed SET and BITFIELD commands being wrongly marked movablekeys#10837oranagra merged 4 commits intoredis:unstablefrom
Conversation
SET and BITFIELD added `get_keys_function` in redis#10148, causing them to be wrongly marked movablekeys in `populateCommandMovableKeys`. Currently SET and BITFIELD has only one key, and key_specs are marked as VARIABLE_FLAGS, so we can rely on the VARIABLE_FLAGS flag. This was an unintended side effect introduced in redis#10148 (7.0 RC2) Fixes redis#10833
|
I don't think I wrote it right but let us discuss it in here. i also thought about something like: int variable_key_specs_num = 0;
for (int i = 0; i < cmd->key_specs_num; i++) {
if (cmd->key_specs[i].flags & CMD_KEY_VARIABLE_FLAGS &&
cmd->key_specs[i].begin_search_type == KSPEC_BS_INDEX &&
cmd->key_specs[i].find_keys_type == KSPEC_FK_RANGE)
{
variable_key_specs_num++;
}
}
if (cmd->key_specs_num != 0 && cmd->key_specs_num == variable_key_specs_num)
movablekeys = 0; |
|
i agree we need to completely re-write this function. since we define the right approach should be to check if the legacy range covers all key-specs, and if none of the key-specs have the INCOMPLETE flag. maybe the easiest way to do that is to unify populateCommandLegacyRangeSpec and populateCommandMovableKeys into one function. then, let's compare the result to the output of 6.2. p.s. in order to properly serve both old and new modules, we must probably keep relying CMD_MODULE_GETKEYS, but do that only for modules that don't declare key-specs. for ones that do, we need to take the same approach we take with native redis commands. |
|
For the record, i compared the list of commands tagged with The other difference was that STRALGO and MEMORY are no longer listed, one was removed (LSC does have a legacy range spec), and the other has a specific sub-command with a legacy range spec, i'm not sure how compatible is that with old clients. i.e. in the past memory was listed as: and not it is listed as @chayim can you check how this affects redis-py (not sure if we have anything to improve) |
|
i pushed suggested approach and some tests. |
|
@oranagra thanks for picking it up, i read the code, LGTM. and i did some comment cleanup (hope you don't mind) so i added a |
|
seems like that bug with prev_lastkey didn't have an impact in redis (maybe it did for some modules). |
…is#10837) The SET and BITFIELD command were added `get_keys_function` in redis#10148, causing them to be wrongly marked movablekeys in `populateCommandMovableKeys`. This was an unintended side effect introduced in redis#10148 (7.0 RC1) which could cause some clients an extra round trip for these commands in cluster mode. Since we define movablekeys as a way to determine if the legacy range [first, last, step] doesn't find all keys, then we need a completely different approach. The right approach should be to check if the legacy range covers all key-specs, and if none of the key-specs have the INCOMPLETE flag. This way, we don't need to look at getkeys_proc of VARIABLE_FLAG at all. Probably with the exception of modules, who may still not be using key-specs. In this PR, we removed `populateCommandMovableKeys` and put its logic in `populateCommandLegacyRangeSpec`. In order to properly serve both old and new modules, we must probably keep relying CMD_MODULE_GETKEYS, but do that only for modules that don't declare key-specs. For ones that do, we need to take the same approach we take with native redis commands. This approach was proposed by Oran. Fixes redis#10833 Co-authored-by: Oran Agra <[email protected]>
The SET and BITFIELD command were added
get_keys_functionin #10148, causingthem to be wrongly marked movablekeys in
populateCommandMovableKeys.This was an unintended side effect introduced in #10148 (7.0 RC1)
which could cause some clients an extra round trip for these commands in cluster mode.
Since we define movablekeys as a way to determine if the legacy range [first, last, step]
doesn't find all keys, then we need a completely different approach.
The right approach should be to check if the legacy range covers all key-specs,
and if none of the key-specs have the INCOMPLETE flag.
This way, we don't need to look at getkeys_proc of VARIABLE_FLAG at all.
Probably with the exception of modules, who may still not be using key-specs.
In this PR, we removed
populateCommandMovableKeysand put its logic inpopulateCommandLegacyRangeSpec.In order to properly serve both old and new modules, we must probably keep relying
CMD_MODULE_GETKEYS, but do that only for modules that don't declare key-specs.
For ones that do, we need to take the same approach we take with native redis commands.
This approach was proposed by Oran. Fixes #10833