Skip to content

drbg_get_entropy: force a reseed before calling ssleay_rand_bytes() [1.0.2 only]#7259

Closed
mspncp wants to merge 2 commits intoopenssl:OpenSSL_1_0_2-stablefrom
mspncp:pr-drbg-get-entropy-force-reseed-variant
Closed

drbg_get_entropy: force a reseed before calling ssleay_rand_bytes() [1.0.2 only]#7259
mspncp wants to merge 2 commits intoopenssl:OpenSSL_1_0_2-stablefrom
mspncp:pr-drbg-get-entropy-force-reseed-variant

Conversation

@mspncp
Copy link
Contributor

@mspncp mspncp commented Sep 18, 2018

Fixes #7240
Alternative to #7247

In FIPS mode, the default FIPS DRBG uses the drbg_get_entropy()
callback to reseed itself, which is provided by the wrapping
libcrypto library. This callback in turn uses ssleay_rand_bytes()
to generate random bytes.

Now ssleay_rand_bytes() calls RAND_poll() once on first call to
seed itself, but RAND_poll() is never called again (unless the
application calls RAND_poll() explicitely). This implies that
whenever the DRBG reseeds itself (which happens every 2^14
generate requests) this happens without obtaining fresh random
data from the operating system's entropy sources.

This patch forces a reseed from system entropy sources on every
call to drbg_get_entropy(). In contrary to the automatic reseeding
of the DRBG in master, this reseeding does not break applications
running in a chroot() environment (see c7504ae), because the
SSLEAY PRNG does not maintain an error state. (It does not even
check the return value of RAND_poll() on its instantiation.)

In the worst case, if no random device is available for reseeding,
no fresh entropy will be added to the SSLEAY PRNG but it will happily
continue to generate random bytes as 'entropy' input for the DRBG's
reseeding, which is just as good (or bad) as before this patch.

Checklist
  • documentation is added or updated
  • tests are added or updated

@mspncp mspncp added the branch: 1.0.2 Applies to OpenSSL_1_0_2-stable branch (EOL) label Sep 18, 2018
@mspncp
Copy link
Contributor Author

mspncp commented Sep 18, 2018

@paulidale @kaduk please let me know which variant you prefer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I forgot to remove the counter... hang on.

Fixes openssl#7240

In FIPS mode, the default FIPS DRBG uses the drbg_get_entropy()
callback to reseed itself, which is provided by the wrapping
libcrypto library. This callback in turn uses ssleay_rand_bytes()
to generate random bytes.

Now ssleay_rand_bytes() calls RAND_poll() once on first call to
seed itself, but RAND_poll() is never called again (unless the
application calls RAND_poll() explicitely). This implies that
whenever the DRBG reseeds itself (which happens every 2^14
generate requests) this happens without obtaining fresh random
data from the operating system's entropy sources.

This patch forces a reseed from system entropy sources on every
call to drbg_get_entropy(). In contrary to the automatic reseeding
of the DRBG in master, this reseeding does not break applications
running in a chroot() environment (see c7504ae), because the
SSLEAY PRNG does not maintain an error state. (It does not even
check the return value of RAND_poll() on its instantiation.)

In the worst case, if no random device is available for reseeding,
no fresh entropy will be added to the SSLEAY PRNG but it will happily
continue to generate random bytes as 'entropy' input for the DRBG's
reseeding, which is just as good (or bad) as before this patch.
@mspncp mspncp force-pushed the pr-drbg-get-entropy-force-reseed-variant branch from c332ae4 to dd6c77e Compare September 18, 2018 18:46
@mspncp
Copy link
Contributor Author

mspncp commented Sep 18, 2018

The appveyor build was broken. I squashed the fixup and force-pushed to kick the bots anew.

Copy link
Contributor

@paulidale paulidale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the cleaner of the two but both look okay.

@mspncp
Copy link
Contributor Author

mspncp commented Sep 19, 2018

Thanks Paul. I'm slightly favouring this variant, too. I'll wait a few more hours to give @kaduk the opportunity to respond before merging.

@kaduk
Copy link
Contributor

kaduk commented Sep 19, 2018

(The "few more hours" came in overnight for me. This does look like a better approach in principle, but give me some time to eat breakfast, etc., and double-check the other uses of the 'initialized' flag (e.g., that we are not introducing any leaks.)

…ytes()

[The following lines will be added as penultimate paragraph to the commit message]

	To prevent ssleay_rand_bytes_from_system() (and hence RAND_poll())
	from being called twice during instantiation, a separate drbg_get_nonce()
	callback has been introduced, which is identical with the previous
	implementation of drbg_get_entropy().
@mspncp
Copy link
Contributor Author

mspncp commented Sep 19, 2018

No need to hurry. I did some testing and some more is coming up...

(BTW: I normally take into account the time zone of my collegues ;-) )

@mspncp
Copy link
Contributor Author

mspncp commented Sep 19, 2018

I noticed that ssleay_rand_bytes_from_system() (and hence RAND_poll()) is called twice during initialization. This is undesirable, because it changes the application behaviour at startup. The reason for the double call is that drbg_get_entropy() is used not only used as get_entropy() but also as get_nonce() callback. The latter would not require an additional reseeding, so I added a separate drbg_get_nonce() callback which behaves identical as before. Commit coming up... (to be squashed)

Here is the test setup I used:

Patch for md_rand.c

diff --git a/crypto/rand/md_rand.c b/crypto/rand/md_rand.c
index abca70f23a..1f08fdb916 100644
--- a/crypto/rand/md_rand.c
+++ b/crypto/rand/md_rand.c
@@ -563,6 +563,7 @@ int ssleay_rand_bytes(unsigned char *buf, int num, int pseudo, int lock)
 
 int ssleay_rand_bytes_from_system(unsigned char *buf, int num)
 {
+    fprintf(stderr, "ssleay_rand_bytes_from_system\n");
     initialized = 0;
     return ssleay_rand_bytes(buf, num, 0, 0);
 }

Makefile

OPENSSL_PREFIX=/opt/openssl-1.0.2-dev
CFLAGS=-g -O0 -I${OPENSSL_PREFIX}/include
LDFLAGS=-L${OPENSSL_PREFIX}/lib -lcrypto -Wl,--enable-new-dtags,-rpath,${OPENSSL_PREFIX}/lib

test: test.c

test.c

#include <stdlib.h>
#include <stdio.h>

#include <openssl/evp.h>
#include <openssl/rand.h>

int main(void)
{
    OpenSSL_add_all_algorithms();
    if (!FIPS_mode_set(1)) {
        fprintf(stderr, "FIPS mode intialization failed!\n");
        return 1;
    }

    unsigned char buf[32];

    size_t mask = (1<<20)-1;
    
    for (size_t i = 0 ; i < (1 << 26) ; ++i) {
        if ((i & mask) == 0)
            fprintf(stderr, "RAND: %zu\n", i);
        if (!RAND_bytes(buf, sizeof(buf))) {
            fprintf(stderr, "RAND_bytes() failed!\n");
            return 1;
        }
    }

    return 0;
}

gdb log

Temporary breakpoint 1, main () at test.c:8
8	{
(gdb) b ssleay_rand_bytes_from_system 
Breakpoint 2 at 0x7ffff7a74744: file md_rand.c, line 566.
(gdb) c
Continuing.

Breakpoint 2, ssleay_rand_bytes_from_system (buf=0x555555756260 "\212\003", num=60) at md_rand.c:566
566	    fprintf(stderr, "ssleay_rand_bytes_from_system\n");
(gdb) bt
#0  ssleay_rand_bytes_from_system (buf=0x555555756260 "\212\003", num=60) at md_rand.c:566
#1  0x00007ffff7a75235 in drbg_get_entropy (ctx=0x7ffff7dd3960 <ossl_dctx>, pout=0x7fffffffd558, entropy=276, min_len=60, max_len=2147483652) at rand_lib.c:201
#2  0x00007ffff79856b9 in fips_get_entropy (dctx=0x7ffff7dd3960 <ossl_dctx>, pout=0x7fffffffd5c0, entropy=256, min_len=32, max_len=2147483632) at fips_drbg_lib.c:159
#3  0x00007ffff79858f5 in FIPS_drbg_instantiate (dctx=0x7ffff7dd3960 <ossl_dctx>, pers=0x7fffffffd610 "OpenSSL DRBG2.0", perslen=32) at fips_drbg_lib.c:235
#4  0x00007ffff7a754c6 in RAND_init_fips () at rand_lib.c:294
#5  0x00007ffff7993e6b in OPENSSL_init () at o_init.c:86
#6  0x00007ffff7a7ffec in EVP_add_cipher (c=0x7ffff7dbeac0 <des_cfb64>) at names.c:72
#7  0x00007ffff7a855d2 in OpenSSL_add_all_ciphers () at c_allc.c:69
#8  0x00007ffff7a855b9 in OPENSSL_add_all_algorithms_noconf () at c_all.c:83
#9  0x0000555555554926 in main () at test.c:9
(gdb) c
Continuing.
ssleay_rand_bytes_from_system

Breakpoint 2, ssleay_rand_bytes_from_system (buf=0x555555756320 "", num=20) at md_rand.c:566
566	    fprintf(stderr, "ssleay_rand_bytes_from_system\n");
(gdb) bt
#0  ssleay_rand_bytes_from_system (buf=0x555555756320 "", num=20) at md_rand.c:566
#1  0x00007ffff7a75235 in drbg_get_entropy (ctx=0x7ffff7dd3960 <ossl_dctx>, pout=0x7fffffffd5b8, entropy=128, min_len=20, max_len=2147483632) at rand_lib.c:201
#2  0x00007ffff7985981 in FIPS_drbg_instantiate (dctx=0x7ffff7dd3960 <ossl_dctx>, pers=0x7fffffffd610 "OpenSSL DRBG2.0", perslen=32) at fips_drbg_lib.c:246
#3  0x00007ffff7a754c6 in RAND_init_fips () at rand_lib.c:294
#4  0x00007ffff7993e6b in OPENSSL_init () at o_init.c:86
#5  0x00007ffff7a7ffec in EVP_add_cipher (c=0x7ffff7dbeac0 <des_cfb64>) at names.c:72
#6  0x00007ffff7a855d2 in OpenSSL_add_all_ciphers () at c_allc.c:69
#7  0x00007ffff7a855b9 in OPENSSL_add_all_algorithms_noconf () at c_all.c:83
#8  0x0000555555554926 in main () at test.c:9

test output

msp@msppc:~/src/committer/issues/7259$ ./test 
ssleay_rand_bytes_from_system
ssleay_rand_bytes_from_system
RAND: 0
RAND: 1048576
RAND: 2097152
RAND: 3145728
RAND: 4194304
RAND: 5242880
RAND: 6291456
RAND: 7340032
RAND: 8388608
RAND: 9437184
RAND: 10485760
RAND: 11534336
RAND: 12582912
RAND: 13631488
RAND: 14680064
RAND: 15728640
ssleay_rand_bytes_from_system
RAND: 16777216
RAND: 17825792
RAND: 18874368
RAND: 19922944
RAND: 20971520
RAND: 22020096
RAND: 23068672
RAND: 24117248
RAND: 25165824
RAND: 26214400
RAND: 27262976
RAND: 28311552
RAND: 29360128
RAND: 30408704
RAND: 31457280
RAND: 32505856
ssleay_rand_bytes_from_system
RAND: 33554432

... etc. ...

@mspncp
Copy link
Contributor Author

mspncp commented Sep 19, 2018

Fix pushed (to be squashed). Please note that a6358f5 amends the commit message.

Copy link
Contributor

@kaduk kaduk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved provided that my assumption is correct...

FIPS_drbg_set_callbacks(dctx,
drbg_get_entropy, drbg_free_entropy, 20,
drbg_get_entropy, drbg_free_entropy);
drbg_get_nonce, drbg_free_entropy);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Someone is going to double-take that drbg_free_entropy is paired with drbg_get_nonce, but the behavior should be fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you think it is confusing the users, I could fix it in one of the following ways:

  • copy&paste drbg_free_entropy to drbg_free_nonce
  • rename drbg_free_entropy to drbg_free_entropy_or_nonce

Please let me know what you prefer.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think any change is needed.

return 0;
}
return min_len;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A little unfortunate to have the duplicate code, but I think it's unavoidable here.

Copy link
Contributor Author

@mspncp mspncp Sep 19, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially, I thought about basing the two callbacks on a common implementation with an extra parameter, but that didn't really make it look better. If you prefer, I can still do it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it would actually look better; no change, please.

int ssleay_rand_bytes_from_system(unsigned char *buf, int num)
{
initialized = 0;
return ssleay_rand_bytes(buf, num, 0, 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume that the locking that allows us to pass lock=0 to ssleay_rand_bytes() also serves to serialize the accesses to initialized (that is, someone calling this function already holds the CRYPTO_RAND_LOCK).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's my assumption, too, but I will double-check it in the debugger using my test program.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@mspncp mspncp changed the title drbg_get_entropy: force a reseed before calling ssleay_rand_bytes() [variant] drbg_get_entropy: force a reseed before calling ssleay_rand_bytes() [1.0.2 only] Sep 19, 2018
@mspncp
Copy link
Contributor Author

mspncp commented Sep 20, 2018

Concurrent access to ssleay_rand_bytes_from_system

TL;DR

The situation is not as simple as I thought, but it is nevertheless harmless.

The long story

First of all: concurrently clearing the initialized flag is harmless, because it would only result in RAND_poll() being called more often than intended. More problematic would be a concurrent access to the SSLEAY RNG, but this problem would have existed prior to this patch already.

For the sake of completeness, here the result of my experiments:

Instantiation of the FIPS_DRBG

The first call to ssleay_rand_bytes_from_system occurs when the DRBG is instantiated, and no lock is held. It is triggered by OPENSSL_init() in an early phase, where every sane program should still be single threaded.

In the case of my test program it happens in OPENSSL_add_all_algorithms_noconf().

#0  ssleay_rand_bytes_from_system (buf=0x555555756260 "\212\003", num=60) at md_rand.c:566
#1  0x00007ffff7a75235 in drbg_get_entropy (ctx=0x7ffff7dd3960 <ossl_dctx>, pout=0x7fffffffd568, entropy=276, min_len=60, max_len=2147483652) at rand_lib.c:201
#2  0x00007ffff79856b9 in fips_get_entropy (dctx=0x7ffff7dd3960 <ossl_dctx>, pout=0x7fffffffd5d0, entropy=256, min_len=32, max_len=2147483632) at fips_drbg_lib.c:159
#3  0x00007ffff79858f5 in FIPS_drbg_instantiate (dctx=0x7ffff7dd3960 <ossl_dctx>, pers=0x7fffffffd620 "OpenSSL DRBG2.0", perslen=32) at fips_drbg_lib.c:235
#4  0x00007ffff7a7558a in RAND_init_fips () at rand_lib.c:310
#5  0x00007ffff7993e6b in OPENSSL_init () at o_init.c:86
#6  0x00007ffff7a800b0 in EVP_add_cipher (c=0x7ffff7dbeac0 <des_cfb64>) at names.c:72
#7  0x00007ffff7a85696 in OpenSSL_add_all_ciphers () at c_allc.c:69
#8  0x00007ffff7a8567d in OPENSSL_add_all_algorithms_noconf () at c_all.c:83
#9  0x000055555555495e in main () at test.c:14

Probably the FIPS_mode_set() call should have gone first in my test program, so I swapped the two functions and obtained the following callstack:

#0  ssleay_rand_bytes_from_system (buf=0x555555756260 "\212\003", num=60) at md_rand.c:566
#1  0x00007ffff7a75235 in drbg_get_entropy (ctx=0x7ffff7dd3960 <ossl_dctx>, pout=0x7fffffffd598, entropy=276, min_len=60, max_len=2147483652) at rand_lib.c:201
#2  0x00007ffff79856b9 in fips_get_entropy (dctx=0x7ffff7dd3960 <ossl_dctx>, pout=0x7fffffffd600, entropy=256, min_len=32, max_len=2147483632) at fips_drbg_lib.c:159
#3  0x00007ffff79858f5 in FIPS_drbg_instantiate (dctx=0x7ffff7dd3960 <ossl_dctx>, pers=0x7fffffffd650 "OpenSSL DRBG2.0", perslen=32) at fips_drbg_lib.c:235
#4  0x00007ffff7a7558a in RAND_init_fips () at rand_lib.c:310
#5  0x00007ffff7993e6b in OPENSSL_init () at o_init.c:86
#6  0x00007ffff7993dc1 in FIPS_mode_set (r=1) at o_fips.c:78
#7  0x000055555555492b in main () at test.c:9

Reseeding of the FIPS_DRBG

The FIPS object module only has a single FIPS_DRBG instance, which is wired to the RAND api when FIPS mode is enabled. So reseeding only occurs via RAND_bytes(), and the CRYPTO_LOCK_RAND lock is taken by fips_drbg_bytes in frame #5*.

#0  ssleay_rand_bytes_from_system (buf=0x55555575ac10 "\300\252uUUU", num=60) at md_rand.c:566
#1  0x00007ffff7a75235 in drbg_get_entropy (ctx=0x7ffff7dd3960 <ossl_dctx>, pout=0x7fffffffd568, entropy=276, min_len=60, max_len=2147483652) at rand_lib.c:201
#2  0x00007ffff79856b9 in fips_get_entropy (dctx=0x7ffff7dd3960 <ossl_dctx>, pout=0x7fffffffd5c8, entropy=256, min_len=32, max_len=2147483632) at fips_drbg_lib.c:159
#3  0x00007ffff7985bee in drbg_reseed (dctx=0x7ffff7dd3960 <ossl_dctx>, adin=0x7ffff7dd4730 <buf> "IY\243[\314\020\005", adinlen=16, hcheck=1) at fips_drbg_lib.c:332
#4  0x00007ffff7985f2e in FIPS_drbg_generate (dctx=0x7ffff7dd3960 <ossl_dctx>, out=0x7fffffffd6d0 "\212\230G\213\Exotic border cases205\016\002P\b", outlen=32, 
    prediction_resistance=0, adin=0x7ffff7dd4730 <buf> "IY\243[\314\020\005", adinlen=16) at fips_drbg_lib.c:430
#5* 0x00007ffff798a4d3 in fips_drbg_bytes (out=0x7fffffffd6d0 "\212\230G\213\205\016\002P\b", count=32) at fips_drbg_rand.c:99
#6  0x00007ffff7a75118 in RAND_bytes (buf=0x7fffffffd6d0 "\212\230G\213\205\016\002P\b", num=32) at rand_lib.c:159
#7  0x00005555555549ad in main () at test.c:23

Exotic border cases

There are hypothetical cases where ssleay_rand_bytes_from_system could be accessed concurrently without holding the lock, for example if one would instantiate and use per-thread instances of the FIPS_DRBG. But the libcrypto wrapper doesn't do things like this, it only has one global instance which is accessible exclusively via the RAND api. The FIPS_DRBG API of the FOM is not exported by libcrypto in 1.0.2.

@mspncp
Copy link
Contributor Author

mspncp commented Sep 20, 2018

Conclusion: after rechecking the facts and having obtained two approvals, I'm assured this is ready to be merged. I'll do it this evening (CEST).

@mspncp mspncp added the approval: done This pull request has the required number of approvals label Sep 20, 2018
@mspncp mspncp self-assigned this Sep 20, 2018
levitte pushed a commit that referenced this pull request Sep 20, 2018
Fixes #7240

In FIPS mode, the default FIPS DRBG uses the drbg_get_entropy()
callback to reseed itself, which is provided by the wrapping
libcrypto library. This callback in turn uses ssleay_rand_bytes()
to generate random bytes.

Now ssleay_rand_bytes() calls RAND_poll() once on first call to
seed itself, but RAND_poll() is never called again (unless the
application calls RAND_poll() explicitely). This implies that
whenever the DRBG reseeds itself (which happens every 2^14
generate requests) this happens without obtaining fresh random
data from the operating system's entropy sources.

This patch forces a reseed from system entropy sources on every
call to drbg_get_entropy(). In contrary to the automatic reseeding
of the DRBG in master, this reseeding does not break applications
running in a chroot() environment (see c7504ae), because the
SSLEAY PRNG does not maintain an error state. (It does not even
check the return value of RAND_poll() on its instantiation.)

In the worst case, if no random device is available for reseeding,
no fresh entropy will be added to the SSLEAY PRNG but it will happily
continue to generate random bytes as 'entropy' input for the DRBG's
reseeding, which is just as good (or bad) as before this patch.

To prevent ssleay_rand_bytes_from_system() (and hence RAND_poll())
from being called twice during instantiation, a separate
drbg_get_nonce() callback has been introduced, which is identical
with the previous implementation of drbg_get_entropy().

Reviewed-by: Paul Dale <[email protected]>
Reviewed-by: Ben Kaduk <[email protected]>
(Merged from #7259)
@mspncp
Copy link
Contributor Author

mspncp commented Sep 20, 2018

Merged. Thanks for reviewing!

@mspncp mspncp closed this Sep 20, 2018
@mspncp mspncp deleted the pr-drbg-get-entropy-force-reseed-variant branch September 20, 2018 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approval: done This pull request has the required number of approvals branch: 1.0.2 Applies to OpenSSL_1_0_2-stable branch (EOL)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants