Skip to content

Add lazyfree-lazy-user-flush config to control default behavior of FLUSH[ALL|DB], SCRIPT FLUSH#8258

Merged
oranagra merged 3 commits intoredis:unstablefrom
yangbodong22011:feature-lua-async-free
Jan 15, 2021
Merged

Add lazyfree-lazy-user-flush config to control default behavior of FLUSH[ALL|DB], SCRIPT FLUSH#8258
oranagra merged 3 commits intoredis:unstablefrom
yangbodong22011:feature-lua-async-free

Conversation

@yangbodong22011
Copy link
Copy Markdown
Contributor

@yangbodong22011 yangbodong22011 commented Dec 28, 2020

  • Adds ASYNC and SYNC arguments to SCRIPT FLUSH
  • Adds SYNC argument to FLUSHDB and FLUSHALL
  • Adds new config to control the default behavior of FLUSHDB, FLUSHALL and SCRIPT FLUASH.

the new behavior is as follows:

  • FLUSH[ALL|DB],SCRIPT FLUSH: Determine sync or async according to the
    value of lazyfree-lazy-user-flush.
  • FLUSH[ALL|DB],SCRIPT FLUSH ASYNC: Always flushes the database in an async manner.
  • FLUSH[ALL|DB],SCRIPT FLUSH SYNC: Always flushes the database in a sync manner.

src/lazyfree.c Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't think this condition is sufficient.

in all other places a lazy free is dependent on a user input or a config, and this threshold just defines if it worth while to send the job to the other thread or keep it in the main thread just because the list is short.

i.e the user first needs to decide if he's willing to accept the async freeing side effects (not atomic), or not.

so to be consistent with that, we should either add a check for lazyfree_lazy_server_del or add an ASYNC argument for SCRIPT FLUSH

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with you, I think both of your suggestions should be taken:

  1. Provide SCRIPT FLUSH ASYNC for users to actively trigger.
  2. SCRIPT FLUSH judges whether to release asynchronously according to lazyfree_lazy_server_del.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on a second thought, i'm actually not sure lazyfree-lazy-server-del is the right config, maybe it should be lazyfree-lazy-user-del or neither.
looking at what FLUSHB does, it should be neither.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should use lazyfree-lazy-server-del or lazyfree-lazy-user-del, since they are already defined and don't mean this.

I also don't think we should be adding ASYNC into command arguments. It's not really something you should be deciding on a case by case basis, and it adds overhead to client developers to support and us to document.

I think we should introduce a general new flag for lazyfree-policy, that covers every other lazy free usage not covered by an existing flag. I think it would even be nice to update the default value for the other lazy-free configs to add a third option, use-default, which would use the lazyfree-policy value unless an explicit value was set.

Copy link
Copy Markdown
Contributor Author

@yangbodong22011 yangbodong22011 Dec 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should introduce a general new flag for lazyfree-policy, that covers every other lazy free usage not covered by an existing flag.

I agree with your suggestion of adding a new flag, for example: lazyfree-lazy-server-free. However, it is more difficult for us to find the remaining code that is not covered by lazyfree. The problem of scriptingRelease is encountered in our production environment, but there are many places where dictRelease is called, see [here](https:// github.com/redis/redis/search?q=dictRelease) . In addition to dictRelease, there are listRelease, raxFree, etc.

I think it would even be nice to update the default value for the other lazy-free configs to add a third option, use-default, which would use the lazyfree-policy value unless an explicit value was set.

Here I have a little doubt, for example: you mean that lazyfree-lazy-eviction will have three optional values: yes,no,use-default? If use-default is set, then the rest of the lazyfree configuration is this value?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi guys. I have some questions:

  • freeLuaScriptsAsync This function name metions the behavior will be a async way. But we still calculate the dict size. And if dictsize not reach LAZYFREE_THRESHOLD 64 . It will become a sync way. In other ways, scriptingRelease require a async arg to call freeLuaScriptsAsync. But inside freeLuaScriptsAsync, it could be a sync way
  • so do freeObjAsync function
  • and also the comment of freeTrackingRadixTreeAsync seems wrong

There is not problem with those questions. Just my thinking

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@enjoy-binbin Yes, LAZYFREE_THRESHOLD controls whether it is really async free.

About freeTrackingRadixTreeAsync comment, it's wrong, it will direct async free and do not judge object's size.

/* Free an object, if the object is huge enough, free it in async way. */
void freeTrackingRadixTreeAsync(rax *tracking) {
    atomicIncr(lazyfree_objects,tracking->numele);
    bioCreateLazyFreeJob(lazyFreeTrackingTable,1,tracking);
}

and another problem is we should use tracking->numnodes(Every macro node in the Stream is one allocation) instead of tracking->numele.

Now we have two points to fix:
1, use tracking->numnodes instead of tracking->numele
2. Judge size to async free or fix comments.
@oranagra @madolson WDYT?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Received a reply so soon. cool. I am willing to correct it if needed

And the mean time. Do we need to be consistent before and after method behavior?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

freeXxxxAsync doesn't necessarily means it'll be released by the thread.
it means it will be potentially async (which has implication on memory being reclaimed in time or not), but it doesn't mean it'll always be done by the thread.
even the UNLINK function doesn't always release asynchronously, it considers that and makes a efficiency decision (it will be inefficient to do it async if you just need to release a few pointers).
so i don't see a problem about the initial complain on freeLuaScriptsAsync, or maybe i didn't understand it.

i agree that freeTrackingRadixTreeAsync and also freeSlotsToKeysMapAsync need to be extended with a check for LAZYFREE_THRESHOLD (against numnodes, not numele).
please note that not every rax structure that we release this way needs to look at numnodes. if the rax was holding any points in it's elements that need to be released too, then we should look at numele or nuele+numnodes, but in this case but these rax structures are holding only keys (no values), so it only needs to release the nodes.

regarding lazyfree_objects, it is a statistics metric, not a real counter of released allocations. it seems to denote the number of objects / elements released, not actual allocations.
However, when we release a list object (composed of a linked list of ziplists), we increment it only by one (not the number of elements in the list, and not the number of linked-list nodes), so for from that point of view, we can increment it only by one here too. (if we do that, we need to update the call to atomicDecr too).
Again, just a metric, and i don't have any major argument for incrementing by either [1, numele, numnodes].

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

freeXxxxAsync doesn't necessarily means it'll be released by the thread.
I misunderstood....
I make pr #8969 to extend freeTrackingRadixTreeAsync and freeSlotsToKeysMapAsync
About lazyfree_objects, according your says. I did not change it

Thanks a lot for your reply. Very detailed. @oranagra

@yangbodong22011 yangbodong22011 changed the title Lua Scripts support async free Add lazyfree-lazy-user-flush configuration to control whether FLUSH[ALL|DB], SCRIPT FLUSH commands are lazyfree Jan 5, 2021
@yangbodong22011 yangbodong22011 force-pushed the feature-lua-async-free branch 2 times, most recently from afb7435 to 1b30052 Compare January 5, 2021 12:43
@yangbodong22011 yangbodong22011 force-pushed the feature-lua-async-free branch 2 times, most recently from 6654e7f to 99a4ed3 Compare January 6, 2021 01:47
…LL|DB], SCRIPT FLUSH commands are lazyfree

the current behavior is as follows:

FLUSH[ALL|DB],SCRIPT FLUSH: Determine sync or async according to the
value of lazyfree-lazy-user-flush.
FLUSH[ALL|DB],SCRIPT FLUSH ASYNC: Always flushes the database in an async manner.
FLUSH[ALL|DB],SCRIPT FLUSH SYNC: Always flushes the database in a sync manner.
madolson
madolson previously approved these changes Jan 6, 2021
Copy link
Copy Markdown
Contributor

@madolson madolson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@madolson madolson added state:major-decision Requires core team consensus approval-needed Waiting for core team approval to be merged state:needs-doc-pr requires a PR to redis-doc repository release-notes indication that this issue needs to be mentioned in the release notes labels Jan 6, 2021
oranagra
oranagra previously approved these changes Jan 6, 2021
Copy link
Copy Markdown
Member

@oranagra oranagra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@yangbodong22011 yangbodong22011 dismissed stale reviews from oranagra and madolson via 1fbd6ec January 6, 2021 07:57
@oranagra
Copy link
Copy Markdown
Member

oranagra commented Jan 6, 2021

@redis/core-team please approve or comment.
i'm not certain this is what we discussed / decided, in case it's not, here's what i was thinking later.

Lets split the lazy configs to 3 groups:

  1. lazyfree-lazy-eviction, lazyfree-lazy-expire, lazyfree-lazy-server-del, replica-lazy-flush:
    these are all about deletions that the server does implicitly, not as a result of a specific command.
  2. lazyfree-lazy-user-del and DEL / UNLINK:
    in this case there's a specific command for async, but since it's widely used we added a server config to change the legacy command behavior.
  3. FLUSHDB / FLUSHALL / SCRIPT FLUSH:
    These have (or should have) an ASYNC and SYNC arguments, and a single config defining the default behavior when no argument is specified. i.e. lazyfree-lazy-user-flush which has a default of no for backwards compatibility.

oranagra
oranagra previously approved these changes Jan 6, 2021
@oranagra oranagra added the state:to-be-merged The PR should be merged soon, even if not yet ready, this is used so that it won't be forgotten label Jan 6, 2021
yossigo
yossigo previously approved these changes Jan 7, 2021
Co-authored-by: Itamar Haber <[email protected]>
@yangbodong22011 yangbodong22011 dismissed stale reviews from yossigo and oranagra via 3bfc88b January 8, 2021 01:45
@madolson
Copy link
Copy Markdown
Contributor

madolson commented Jan 13, 2021

@yangbodong22011 Can you create PR for the doc update? I think that is the only thing blocking the merge now.

@yangbodong22011
Copy link
Copy Markdown
Contributor Author

docs PR: redis/redis-doc#1493

@oranagra oranagra changed the title Add lazyfree-lazy-user-flush configuration to control whether FLUSH[ALL|DB], SCRIPT FLUSH commands are lazyfree Add lazyfree-lazy-user-flush config to control default behavior of FLUSH[ALL|DB], SCRIPT FLUSH Jan 15, 2021
@oranagra oranagra merged commit 294f93a into redis:unstable Jan 15, 2021
itamarhaber pushed a commit to redis/redis-doc that referenced this pull request Jan 29, 2021
@oranagra oranagra mentioned this pull request Jan 31, 2021
JackieXie168 pushed a commit to JackieXie168/redis that referenced this pull request Mar 2, 2021
…USH[ALL|DB], SCRIPT FLUSH (redis#8258)

* Adds ASYNC and SYNC arguments to SCRIPT FLUSH
* Adds SYNC argument to FLUSHDB and FLUSHALL
* Adds new config to control the default behavior of FLUSHDB, FLUSHALL and SCRIPT FLUASH.

the new behavior is as follows:
* FLUSH[ALL|DB],SCRIPT FLUSH: Determine sync or async according to the
  value of lazyfree-lazy-user-flush.
* FLUSH[ALL|DB],SCRIPT FLUSH ASYNC: Always flushes the database in an async manner.
* FLUSH[ALL|DB],SCRIPT FLUSH SYNC: Always flushes the database in a sync manner.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approval-needed Waiting for core team approval to be merged release-notes indication that this issue needs to be mentioned in the release notes state:major-decision Requires core team consensus state:needs-doc-pr requires a PR to redis-doc repository state:to-be-merged The PR should be merged soon, even if not yet ready, this is used so that it won't be forgotten

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants