Modules: add defrag API support.#8149
Conversation
Add a new set of defrag functions that take a defrag context and allow defragmenting memory blocks and RedisModuleStrings. Modules can register a defrag callback which will be invoked when the defrag process handles globals. Modules with custom data types can also register a datatype-specific defrag callback which is invoked for keys that require defragmentation. The callback and associated functions support both one-step and multi-step options, depending on the complexity of the key as exposed by the free_effort callback.
oranagra
left a comment
oranagra
left a comment
There was a problem hiding this comment.
As we discussed, at some point in time we'll have to also add an option to defrag a dict (rax), both with and without the values in the dict (i.e. the rax nodes and header themselves need defragging).
Also as we discussed, at some point in time we'll need to add an option to defrag global incrementally (with cursors and such).
We probably need to mention these in some "todo" comment, or open an issue for some milestone (unless we plan on doing that in the next couple of weeks).
src/defrag.c
Outdated
| robj *obj = dictGetVal(kde); | ||
| serverAssert(obj->type == OBJ_MODULE); | ||
|
|
||
| long defragged = moduleDefragValue(dictGetKey(kde), obj); |
There was a problem hiding this comment.
do you think the module needs to know the db index?
it probably doesn't even need the key, but if we do provide the key, who knows what it needs it for...
There was a problem hiding this comment.
Not providing the key would be cleaner but we did see this become an issue with complex structures in the past.
When I come to think about it, if the db index is needed in addition to the key name it's an issue all over the place with the API.
oranagra
added
the
release-notes
|
Closes #4185 |
|
@yossigo looking at #4185 i see this suggestion wasn't covered:
also, i remind you that we're missing:
i'm not sure if we wanna cover the "forbidden" thing. i.e. if a user will enable defrag on a db with a module data type that doesn't support it, it risks running a "busy" loop that achieves nothing. but maybe that burden is for the user to read the module docs before enabling defrag. regarding the other points, shall we open tickets and put them in some milestone? |
Add a new set of defrag functions that take a defrag context and allow defragmenting memory blocks and RedisModuleStrings. Modules can register a defrag callback which will be invoked when the defrag process handles globals. Modules with custom data types can also register a datatype-specific defrag callback which is invoked for keys that require defragmentation. The callback and associated functions support both one-step and multi-step options, depending on the complexity of the key as exposed by the free_effort callback.
3 participants
Add a new set of defrag functions that take a defrag context and allow
defragmenting memory blocks and RedisModuleStrings.
Modules can register a defrag callback which will be invoked when the
defrag process handles globals.
Modules with custom data types can also register a datatype-specific
defrag callback which is invoked for keys that require defragmentation.
The callback and associated functions support both one-step and
multi-step options, depending on the complexity of the key as exposed by
the free_effort callback.