-
Notifications
You must be signed in to change notification settings - Fork 38.7k
Try to use posix_fadvise with CBufferedFile #14485
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Concept ACK, would be good to hear from @eklitzke why the original was closed though |
src/util.h
Outdated
| //! Close a file and return the result of fclose(). On POSIX systems that | ||
| //! support it, advise the kernel to remove the file contents from the page | ||
| //! cache (which can help on memory-constrained systems). | ||
| int CloseAndDiscard(FILE *file); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"discard" sounds to me as if writes to the file will be discarded, too
maybe CloseAndUncache?
|
Concept ACK though I'd really like to see benchmarks for changes like this, to know if this has any impact in practice worth making the risk/complication of the change. |
|
Probably worthing having -- rough concept ACK -- however:
|
|
Rebased and addressed @laanwj's function rename request. |
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
| There hasn't been much activity lately and the patch still needs rebase, so I am closing this for now. Please let me know when you want to continue working on this, so the pull request can be re-opened. |
|
Rebased |
vasild
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My gut feel tells me that this is good, but experience tells me that it needs some performance testing to demonstrate that it really brings a boost. Sometimes such changes have surprisingly adverse effects or no effects and end up being just code clutter.
|
Adding to the three previous requests for benchmarks: Do we have any benchmarks showing the practical gains in swiftness? :) |
This primarily affects blocks when bitcoin is launched with -reindex, as that causes the block files to be loaded as CBufferedFile objects one at a time as the reindex progresses. Co-Authored-By: Luke Dashjr <[email protected]>
|
|
|
This is about 4% difference. Could it be within noise/variance? I mean if you run 3 times Is such a workload disk bound? If it is, then does the full node (the source of the sync) compete for disk resources too (that would slow down the syncing node and bring noise)? Was the full node also running the PR? |
|
@vasild you're right, sorry about my last benchmark. I didn't lock the CPU frequency which probably caused a lot of noise. I re-ran some benchmarks with CPU frequency locked. I rebased the PR locally on master. pr: -prune=10000 -stopatheight=400000master: -prune=10000 -stopatheight=400000The results seem to be similar, so ACK 3f2c08b. |
|
🐙 This pull request conflicts with the target branch and needs rebase. |
|
There hasn't been much activity lately and the patch still needs rebase. What is the status here?
|
|
This PR does not seem to have conceptual support. Please leave a comment if you would like this to be reopened. |
This primarily affects blocks when bitcoin is launched with -reindex, as that causes the block files to be loaded as CBufferedFile objects one at a time as the reindex progresses. Co-Authored-By: Luke Dashjr <[email protected]> Github-Pull: bitcoin#14485 Rebased-From: 289e88b
Resurrecting #12491, since it seemed fine and desirable...