-
Notifications
You must be signed in to change notification settings - Fork 38.7k
util: Specific GetOSRandom for Linux/FreeBSD/OpenBSD #9821
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
These are available in sandboxes without access to files or devices. Also [they are safer and more straightforward](https://en.wikipedia.org/wiki/Entropy-supplying_system_calls) to use than `/dev/urandom` as reading from a file has quite a few edge cases: - Linux: `getrandom(buf, buflen, 0)`. [getrandom(2)](http://man7.org/linux/man-pages/man2/getrandom.2.html) was introduced in version 3.17 of the Linux kernel. - OpenBSD: `getentropy(buf, buflen)`. The [getentropy(2)](http://man.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man2/getentropy.2) function appeared in OpenBSD 5.6. - FreeBSD and NetBSD: `sysctl(KERN_ARND)`. Not sure when this was added but it has existed for quite a while. Alternatives: - Linux has sysctl `CTL_KERN` / `KERN_RANDOM` / `RANDOM_UUID` which gives 16 bytes of randomness. This may be available on older kernels, however [sysctl is deprecated on Linux](https://lwn.net/Articles/605392/) and even removed in some distros so we shouldn't use it. Add tests for `GetOSRand()`: - Test that no error happens (otherwise `RandFailure()` which aborts) - Test that all 32 bytes are overwritten (initialize with zeros, try multiple times) Discussion: - When to use these? Currently they are always used when available. Another option would be to use them only when `/dev/urandom` is not available. But this would mean these code paths receive less testing, and I'm not sure there is any reason to prefer `/dev/urandom`. Closes: bitcoin#9676
|
Concept ACK. I think this should use a runtime test case at init, not just a boost testcase-- code may be built on a system where it works but run elsewhere and this is too important to mess up. :) |
|
I'm OK with adding this to the sanity checks, but in all fairness we don't check that for |
|
I think |
|
Nevermind, it seems you're using the syscall directly. |
Yes but we don't have an issue there where there are older kernels that don't have the functionality... (Also, I would be of the view that we should have a runtime test currently too-- but I didn't want to bloat up the original PR that added this.) |
|
Concept ACK. utACK on the build changes in particular. |
src/random.cpp
Outdated
| * will always return as many bytes as requested and will not be | ||
| * interrupted by signals." | ||
| */ | ||
| if (syscall(SYS_getrandom, ent32, NUM_OS_RANDOM_BYTES, 0) != NUM_OS_RANDOM_BYTES) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this not need the GRND_BLOCK flag? I think we want this blocking rather than returning /dev/random data, no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
According to the man page there is no GRND_BLOCK flag, just a GRND_NONBLOCK flag, which is only significant when passing GRND_RANDOM (which selects /dev/random) too. In that mode, it means EAGAIN can be returned if there is not enough entropy.
In /dev/urandom mode it will never block. When requesting <= 256 bytes it will not even be interruptable by a signal.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for clarifying. When I searched for the kernel implementation, I landed on https://lwn.net/Articles/605828/, which I guess was an early draft of getrandom with the flag reversed. Carry on :)
Well it still makes sense. On Linux, syscalls that are not implemented will predictably return an error w/ errno |
Move the OS random test to a sanity check function that is called every time bitcoind is initialized. Keep `src/test/random_tests.cpp` for the case that later random tests are added, and keep a rudimentary test that just calls the sanity check.
If the code was compiled with newer (>=3.17) kernel headers but executed on a system without the system call, every use of random would crash the program. Add a fallback for that case.
|
It seems I'm consistently commenting on old states of this PR. |
|
utACK 7e6dcd9 It seems none of the new methods are available on OSX (or at least in the build environment in Travis), but a new method can be added later. |
It may well be that OSX has no way to read kernel randomness besides |
|
Libressl has getentropy emulation for a number of different OS's that may be useful as a reference. |
Thanks. That source confirms that there is no other way besides using /dev/urandom in OSX. |
7e6dcd9 random: Add fallback if getrandom syscall not available (Wladimir J. van der Laan) 7cad849 sanity: Move OS random to sanity check function (Wladimir J. van der Laan) aa09ccb squashme: comment that NUM_OS_RANDOM_BYTES should not be changed lightly (Wladimir J. van der Laan) 224e6eb util: Specific GetOSRandom for Linux/FreeBSD/OpenBSD (Wladimir J. van der Laan) Tree-SHA512: 9fd408b1316c69de86674f342339b2f89192fd317c8c036b5df4320f828fa263c7966146bfc1904c51137ee4a26e4cb0f560b2cd05e18cde4d808b9b92ad15c4
While digging around I found that OSX 10.12+ does have __OSX_AVAILABLE(10.12) __IOS_AVAILABLE(10.0) __TVOS_AVAILABLE(10.0) __WATCHOS_AVAILABLE(3.0)
int getentropy(void* buffer, size_t size);Example: #include <stdlib.h>
#include <unistd.h>
#include <sys/random.h>
int main()
{
unsigned char buf[32];
if (getentropy(buf, sizeof(buf)) < 0) {
return 1;
}
write(STDOUT_FILENO, buf, sizeof(buf));
return 0;
}I'm not sure how to make autoconf detect something that may be in different headers. |
ee2d10a Check if sys/random.h is required for getentropy on OSX. (James Hilliard) Pull request description: This should check and include sys/random.h if required for osx as mentioned [here](#9821 (comment)). Tree-SHA512: e9491f67f2e8b2e6bcdbcbb8063295e844d5627daf5336e3e17b4a8027d888fa65a08e4580a745abdc35ffd8d86b4fc7434daaac172c4a06ab7566a2ed0bfb92
ee2d10a Check if sys/random.h is required for getentropy on OSX. (James Hilliard) Pull request description: This should check and include sys/random.h if required for osx as mentioned [here](bitcoin/bitcoin#9821 (comment)). Tree-SHA512: e9491f67f2e8b2e6bcdbcbb8063295e844d5627daf5336e3e17b4a8027d888fa65a08e4580a745abdc35ffd8d86b4fc7434daaac172c4a06ab7566a2ed0bfb92
7e6dcd9 random: Add fallback if getrandom syscall not available (Wladimir J. van der Laan) 7cad849 sanity: Move OS random to sanity check function (Wladimir J. van der Laan) aa09ccb squashme: comment that NUM_OS_RANDOM_BYTES should not be changed lightly (Wladimir J. van der Laan) 224e6eb util: Specific GetOSRandom for Linux/FreeBSD/OpenBSD (Wladimir J. van der Laan) Tree-SHA512: 9fd408b1316c69de86674f342339b2f89192fd317c8c036b5df4320f828fa263c7966146bfc1904c51137ee4a26e4cb0f560b2cd05e18cde4d808b9b92ad15c4
ee2d10a Check if sys/random.h is required for getentropy on OSX. (James Hilliard) Pull request description: This should check and include sys/random.h if required for osx as mentioned [here](bitcoin#9821 (comment)). Tree-SHA512: e9491f67f2e8b2e6bcdbcbb8063295e844d5627daf5336e3e17b4a8027d888fa65a08e4580a745abdc35ffd8d86b4fc7434daaac172c4a06ab7566a2ed0bfb92
ee2d10a Check if sys/random.h is required for getentropy on OSX. (James Hilliard) Pull request description: This should check and include sys/random.h if required for osx as mentioned [here](bitcoin/bitcoin#9821 (comment)). Tree-SHA512: e9491f67f2e8b2e6bcdbcbb8063295e844d5627daf5336e3e17b4a8027d888fa65a08e4580a745abdc35ffd8d86b4fc7434daaac172c4a06ab7566a2ed0bfb92
These are available in sandboxes without access to files or to devices. Also they are considered safer and more straightforward to use than
/dev/urandomas reading from a file has quite a few edge cases:getrandom(buf, buflen, 0). getrandom(2) was introduced in version 3.17 of the Linux kernel.getentropy(buf, buflen). The getentropy(2) function appeared in OpenBSD 5.6.sysctl(KERN_ARND). Not sure when this was added but it has existed for quite a while.Alternatives:
CTL_KERN/KERN_RANDOM/RANDOM_UUIDwhich gives 16 bytes of randomness. This may be available on older kernels, however sysctl is deprecated on Linux and even removed in some distros so we shouldn't use it.Add tests for
GetOSRand():RandFailure()which aborts)Discussion:
/dev/urandomis not available. But this would mean these code paths receive less testing, and I'm not sure there is any reason to prefer/dev/urandom.Closes: #9676. I've tested this on all three OS-es.