Fix pre-size hashtables per slot when reading RDB files#2466
Merged
enjoy-binbin merged 3 commits intovalkey-io:unstablefrom Aug 19, 2025
Merged
Fix pre-size hashtables per slot when reading RDB files#2466enjoy-binbin merged 3 commits intovalkey-io:unstablefrom
enjoy-binbin merged 3 commits intovalkey-io:unstablefrom
Conversation
…e the hashtable
If we want to expand kvstoreHashtableExpand, we need to make sure the
hashtable exists. Currently, when processing RDB slot-info, our expand
has no effect because the hashtable does not exist (we initialize it only
when we need it).
```
if (server.cluster_enabled) {
/* In cluster mode we resize individual slot specific dictionaries based on the number of keys that
* slot holds. */
kvstoreHashtableExpand(db->keys, slot_id, slot_size);
kvstoreHashtableExpand(db->expires, slot_id, expires_slot_size);
should_expand_db = 0;
}
```
We also update kvstoreExpand to use the kvstoreHashtableExpand to make
sure there is only one code path. Also see valkey-io#1328 for more details.
Signed-off-by: Binbin <[email protected]>
Member
Author
|
This seem broken at the very beginning. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## unstable #2466 +/- ##
============================================
+ Coverage 72.02% 72.07% +0.04%
============================================
Files 126 126
Lines 70479 70485 +6
============================================
+ Hits 50761 50799 +38
+ Misses 19718 19686 -32
🚀 New features to boost your workflow:
|
zuiderkwast
approved these changes
Aug 11, 2025
Signed-off-by: Binbin <[email protected]>
Signed-off-by: Binbin <[email protected]>
github-project-automation
bot
moved this from In Progress
to To be backported
in Valkey 8.0
enjoy-binbin
added
the
release-notes
enjoy-binbin
changed the title
zuiderkwast
changed the title
allenss-amazon
pushed a commit
to allenss-amazon/valkey-core
that referenced
this pull request
Aug 19, 2025
valkey-io#2466) If we want to expand kvstoreHashtableExpand, we need to make sure the hashtable exists. Currently, when processing RDB slot-info, our expand has no effect because the hashtable does not exist (we initialize it only when we need it). We also update kvstoreExpand to use the kvstoreHashtableExpand to make sure there is only one code path. Also see valkey-io#1199 for more details. Signed-off-by: Binbin <[email protected]>
asagege
pushed a commit
to asagege/valkey
that referenced
this pull request
Aug 19, 2025
valkey-io#2466) If we want to expand kvstoreHashtableExpand, we need to make sure the hashtable exists. Currently, when processing RDB slot-info, our expand has no effect because the hashtable does not exist (we initialize it only when we need it). We also update kvstoreExpand to use the kvstoreHashtableExpand to make sure there is only one code path. Also see valkey-io#1199 for more details. Signed-off-by: Binbin <[email protected]>
Contributor
|
Backporting this to 8.0 is non-trivial, because of the dict/hashtable change. I suggest we make a separate PR targeting the 8.0 branch. |
enjoy-binbin
added a commit
to vitarb/valkey
that referenced
this pull request
Aug 22, 2025
When reading RDB files with information about the number of keys per cluster slot, we need to create the dicts if they don't exist. Currently, when processing RDB slot-info, our expand has no effect because the dict does not exist (we initialize it only when we need it). We also update kvstoreExpand to use the kvstoreDictExpand to make sure there is only one code path. Also see valkey-io#1199 for more details. Signed-off-by: Binbin <[email protected]>
Merged
Member
Author
|
I pushed the commit to the target branch already. |
enjoy-binbin
added a commit
to vitarb/valkey
that referenced
this pull request
Aug 22, 2025
When reading RDB files with information about the number of keys per cluster slot, we need to create the dicts if they don't exist. Currently, when processing RDB slot-info, our expand has no effect because the dict does not exist (we initialize it only when we need it). We also update kvstoreExpand to use the kvstoreDictExpand to make sure there is only one code path. Also see valkey-io#1199 for more details. Signed-off-by: Binbin <[email protected]>
zuiderkwast
pushed a commit
to vitarb/valkey
that referenced
this pull request
Aug 22, 2025
When reading RDB files with information about the number of keys per cluster slot, we need to create the dicts if they don't exist. Currently, when processing RDB slot-info, our expand has no effect because the dict does not exist (we initialize it only when we need it). We also update kvstoreExpand to use the kvstoreDictExpand to make sure there is only one code path. Also see valkey-io#1199 for more details. Signed-off-by: Binbin <[email protected]>
zuiderkwast
pushed a commit
to vitarb/valkey
that referenced
this pull request
Aug 22, 2025
When reading RDB files with information about the number of keys per cluster slot, we need to create the dicts if they don't exist. Currently, when processing RDB slot-info, our expand has no effect because the dict does not exist (we initialize it only when we need it). We also update kvstoreExpand to use the kvstoreDictExpand to make sure there is only one code path. Also see valkey-io#1199 for more details. Signed-off-by: Binbin <[email protected]>
zuiderkwast
pushed a commit
that referenced
this pull request
Aug 22, 2025
When reading RDB files with information about the number of keys per cluster slot, we need to create the dicts if they don't exist. Currently, when processing RDB slot-info, our expand has no effect because the dict does not exist (we initialize it only when we need it). We also update kvstoreExpand to use the kvstoreDictExpand to make sure there is only one code path. Also see #1199 for more details. Signed-off-by: Binbin <[email protected]>
sarthakaggarwal97
pushed a commit
to sarthakaggarwal97/valkey
that referenced
this pull request
Sep 16, 2025
When reading RDB files with information about the number of keys per cluster slot, we need to create the dicts if they don't exist. Currently, when processing RDB slot-info, our expand has no effect because the dict does not exist (we initialize it only when we need it). We also update kvstoreExpand to use the kvstoreDictExpand to make sure there is only one code path. Also see valkey-io#1199 for more details. Signed-off-by: Binbin <[email protected]>
rjd15372
pushed a commit
to rjd15372/valkey
that referenced
this pull request
Sep 19, 2025
valkey-io#2466) If we want to expand kvstoreHashtableExpand, we need to make sure the hashtable exists. Currently, when processing RDB slot-info, our expand has no effect because the hashtable does not exist (we initialize it only when we need it). We also update kvstoreExpand to use the kvstoreHashtableExpand to make sure there is only one code path. Also see valkey-io#1199 for more details. Signed-off-by: Binbin <[email protected]>
rjd15372
pushed a commit
that referenced
this pull request
Sep 23, 2025
#2466) If we want to expand kvstoreHashtableExpand, we need to make sure the hashtable exists. Currently, when processing RDB slot-info, our expand has no effect because the hashtable does not exist (we initialize it only when we need it). We also update kvstoreExpand to use the kvstoreHashtableExpand to make sure there is only one code path. Also see #1199 for more details. Signed-off-by: Binbin <[email protected]>
ranshid
pushed a commit
to ranshid/valkey
that referenced
this pull request
Sep 30, 2025
valkey-io#2466) If we want to expand kvstoreHashtableExpand, we need to make sure the hashtable exists. Currently, when processing RDB slot-info, our expand has no effect because the hashtable does not exist (we initialize it only when we need it). We also update kvstoreExpand to use the kvstoreHashtableExpand to make sure there is only one code path. Also see valkey-io#1199 for more details. Signed-off-by: Binbin <[email protected]>
ranshid
pushed a commit
to ranshid/valkey
that referenced
this pull request
Sep 30, 2025
valkey-io#2466) If we want to expand kvstoreHashtableExpand, we need to make sure the hashtable exists. Currently, when processing RDB slot-info, our expand has no effect because the hashtable does not exist (we initialize it only when we need it). We also update kvstoreExpand to use the kvstoreHashtableExpand to make sure there is only one code path. Also see valkey-io#1199 for more details. Signed-off-by: Binbin <[email protected]>
zuiderkwast
pushed a commit
that referenced
this pull request
Oct 1, 2025
#2466) If we want to expand kvstoreHashtableExpand, we need to make sure the hashtable exists. Currently, when processing RDB slot-info, our expand has no effect because the hashtable does not exist (we initialize it only when we need it). We also update kvstoreExpand to use the kvstoreHashtableExpand to make sure there is only one code path. Also see #1199 for more details. Signed-off-by: Binbin <[email protected]>
hpatro
pushed a commit
to hpatro/valkey
that referenced
this pull request
Oct 3, 2025
valkey-io#2466) If we want to expand kvstoreHashtableExpand, we need to make sure the hashtable exists. Currently, when processing RDB slot-info, our expand has no effect because the hashtable does not exist (we initialize it only when we need it). We also update kvstoreExpand to use the kvstoreHashtableExpand to make sure there is only one code path. Also see valkey-io#1199 for more details. Signed-off-by: Binbin <[email protected]> Signed-off-by: Harkrishn Patro <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When reading RDB files with information about the number of keys per cluster slot, we need to create the hashtables if they don't exist.
Currently, when processing RDB slot-info, our expand has no effect because the hashtable does not exist (we initialize it only when we need it).
We also update kvstoreExpand to use the kvstoreHashtableExpand to make
sure there is only one code path. Also see #1199 for more details.