-
Notifications
You must be signed in to change notification settings - Fork 38.7k
Reduce number of open LevelDB files to 64 #2557
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
ACK. Reduces FD usage in my torture test from 500 to 112. However, 112 + 125 is awful close to 256, so I'm still concerned about OSX. |
Reduce number of open LevelDB files to 64
|
This seems to cause massive issues for me on Win7 x64, I often get the message that I need to reindex because of a broken database or sometimes only |
|
@Diapolo That would be bad and unexpected news. I don't see how that could cause/worsen that, but we never know of course. Can you confirm it's related to this? |
|
I sent @gmaxwell a copy of my coinstate folder, with which I had the problem. Currently I'm 73 weeks behind with a client that doesn't have this patch active. After it synced without error I'll restart with this patch applied and a bootstrap (which contains 50% of the current blocks on the net) + re-sync. |
|
No crashes without the patch... testing with it again now. |
|
Again, |
|
Perhaps we should just remove that patch for Windows, as per http://msdn.microsoft.com/en-us/library/6e3b887c.aspx "Up to 2,048 files can be open simultaneously at the lowio level (that is, opened and accessed by means of the _open, _read, _write, and so forth family of I/O functions). Up to 512 files can be open simultaneously at the stdio level (that is, opened and accessed by means of the fopen, fgetc, fputc, and so forth family of functions). The limit of 512 open files at the stdio level can be increased to a maximum of 2,048 by means of the _setmaxstdio function." |
|
@Diapolo I'd rather find the reason for this crash, as this patch should not have any difference at all during IBD. |
|
@sipa Sure, I'll take a look later... |
|
Seems the error was the same as I posted in the chat yesterday: |
|
@Diapolo Feel like retrying with the commits in https://github.com/sipa/bitcoin/commits/win32ldberr ? |
Reduce number of open LevelDB files to 64
I don't notice any significant IBD performance degradation using this.