Skip to content

Optimization: moduleLoadString should try to create embedded string#11050

Merged
oranagra merged 1 commit intoredis:unstablefrom
valentinogeron:fix-module-string-load
Jul 31, 2022
Merged

Optimization: moduleLoadString should try to create embedded string#11050
oranagra merged 1 commit intoredis:unstablefrom
valentinogeron:fix-module-string-load

Conversation

@valentinogeron
Copy link
Contributor

@valentinogeron valentinogeron commented Jul 27, 2022

Before this change, if the module has an embedded string, then uses RedisModule_SaveString and RedisModule_LoadString, the result would be a raw string instead of an embedded string.

Now the RDB_LOAD_ENC flag to moduleLoadString only affects integer encoding, but not embedded strings (which still hold an sds in the robj ptr, so they're actually still raw strings for anyone who reads them).

Release notes: Reduce memory usage on strings loaded by a module from an RDB file

@valentinogeron valentinogeron force-pushed the fix-module-string-load branch from 5530dce to f3511f1 Compare July 27, 2022 16:17
Comment on lines -550 to -551
robj *o = encode ? tryCreateStringObject(SDS_NOINIT,len) :
tryCreateRawStringObject(SDS_NOINIT,len);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i see this condition was added here: c7822bf
before that it always used to create raw string.
i went looking for it in fear that this PR reverts an intentional change and there was some reason for it.
but i suppose it just just plain mistake.
i.e. at the time, that commit should have changed the code from always creating a raw string into something that automatically decides according to the size, but instead a condition on encode was added, which i think was a mistake.

@oranagra
Copy link
Member

@yossigo i'd like a second set of eyes on this one.
also, do you think it should be mentioned in the release notes?

@oranagra oranagra changed the title Optimization: moduleLoadString should try to create embedded string if not plain Optimization: moduleLoadString should try to create embedded string Jul 28, 2022
@yossigo
Copy link
Collaborator

yossigo commented Jul 28, 2022

@oranagra It looks correct to me, but potentially non-trivial implications if it uncovers other issues. I think it should be in the release notes at least for this reason.

@oranagra
Copy link
Member

looking at it deeper, it affects anyone who called rdbGenericLoadStringObject with RDB_LOAD_NONE, since RDB_LOAD_SDS and RDB_LOAD_PLAIN don't go into that path, and RDB_LOAD_ENC was already ok.
so other RM_LoadString, it's just rdbLoadCheckModuleValue and rdbLoadStringObject.
the last of which is actually only currently used for RDB_OPCODE_AUX.

so other than the module implications, the others seem safe.
let's merge it (still long away from a release), and see

@oranagra oranagra added the release-notes indication that this issue needs to be mentioned in the release notes label Jul 28, 2022
@oranagra
Copy link
Member

actually, before i do, @MeirShpilraien please take a look too to consider any module implications

@MeirShpilraien
Copy link

Looks OK 👍 . Don't see any issue with it.

@oranagra oranagra merged commit 2029976 into redis:unstable Jul 31, 2022
enjoy-binbin pushed a commit to enjoy-binbin/redis that referenced this pull request Jul 31, 2023
…f not plain (redis#11050)

Before this change, if the module has an embedded string, then uses RedisModule_SaveString
and RedisModule_LoadString, the result would be a raw string instead of an embedded string.

Now the `RDB_LOAD_ENC` flag to `moduleLoadString` only affects integer encoding, but not
embedded strings (which still hold an sds in the robj ptr, so they're actually still raw strings for
anyone who reads them).

Co-authored-by: Valentino Geron <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-notes indication that this issue needs to be mentioned in the release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants