Repository navigation
Provide more background information / documentation about need and options regarding high quality entropy #172
Description
I surely have a build switch to use the system expat lib but I also
include expat in my project for convenience and I'm feeling
uncomfortable (just because of lack of knowledge) with xmlparse.c,
lines 96-107:
Your options include: \
* Linux + glibc >=2.25 (getrandom): HAVE_GETRANDOM, \
* Linux + glibc <2.25 (syscall SYS_getrandom): HAVE_SYSCALL_GETRANDOM, \
* BSD / macOS >=10.7 (arc4random_buf): HAVE_ARC4RANDOM_BUF, \
* BSD / macOS <10.7 (arc4random): HAVE_ARC4RANDOM, \
* libbsd (arc4random_buf): HAVE_ARC4RANDOM_BUF + HAVE_LIBBSD, \
* libbsd (arc4random): HAVE_ARC4RANDOM + HAVE_LIBBSD, \
* Linux / BSD / macOS (/dev/urandom): XML_DEV_URANDOM \
* Windows (RtlGenRandom): _WIN32. \
\
If insist on not using any of these, bypass this error by defining \
XML_POOR_ENTROPY; you have been warned. \
Especially binary distributions (not only of my project but also of
projects, that include my project) seem problematic to me - if the
build uses a modern, fancy source of entropy then the resulting
binaries won't work on older systems, that doesn't provide them, no?
There are decisions to made: provide different (binary) downloads or
drop support for older systems or pick the entropy source that is most
widely available, despite its possible drawbacks (XML_DEV_URANDOM is a
tempting candidate) or just leave everything as it was for years and
years (XML_POOR_ENTROPY).
I don't feel well-informed enough to decide this for me and even
lesser to advice the user of my code what they should do.
The author(s) of this code change of expat must have this knowledge, I
suspect. Therefor a short write-up that covers at least questions
like
-
what is the problem that this code tries to solve in the first place
(defend against DoS attacks, I think, but some more detailed info
(may it be just a link) would be helpful) -
what are the costs (mainly in terms of runtime and compared against
the "null" solution (using the old thing with XML_POOR_ENTROPY)) and
the benefits (how well does it help against the problem) of the
several options above -
does some of the options fail or block for some time under certain
circumstances (because of lack of available entropy)
would be super helpful.
Activity
hartwork commentedon Nov 19, 2017
Hi!
I'm rather short on time, please excuse that I only cover some of your questions in this reply:
Please note that if your project does need to support Windows, bundling Expat (like any library) is not recommended for security reasons, because you and your users would all need to update their copy to stay up to date with security fixes and few people do that at all or in time. See also:
High quality entropy is used for DoS protection in Expat, hash collisions to be precise.
The "null" solution uses poor entropy, and does not protect against DoS well.
If you need to provide binaries supporting non-recent distributions of Linux, I'd recommend use of both
HAVE_SYSCALL_GETRANDOMandXML_DEV_URANDOMbecause then you:getrandomavailablegetrandomsyscall for high quality entropy where available/dev/urandomif former fails (including case of absence)Best, Sebastian
Let's try with HAVE_SYSCALL_GETRANDOM
Let's try with HAVE_SYSCALL_GETRANDOM
kkkunche commentedon Aug 8, 2019
What needs to be done here other than the issue
#239
kkkunche commentedon Aug 8, 2019
Does the pull request #301 suffice this issue, please check.
Please let me know if any additional things need to be done.
hartwork commentedon Aug 10, 2019
#301 gives more power to users but does not educate them about their options and the need for high quality entropy. This ticket is about a "short write-up", about educating users.
[-]Sources of entropy - missing background information[/-][+]Provide more background information / documentation about need and options regarding high quality entropy[/+]kkkunche commentedon Sep 14, 2019
The short write-up where you would like to update ?.
These are the options which we have added recently.
--with-getrandom enforce the use of getrandom function in the system
[default=check]
--without-getrandom skip auto detect of getrandom [default=check]
--with-sys-getrandom enforce the use of syscall SYS_getrandom function in
the system [default=check]
--without-sys-getrandom skip auto detect of syscall SYS_getrandom
[default=check]
kkkunche commentedon Sep 30, 2019
Hartwork, any update on this. Let us update the required documentation and close it.
xmlparse.c: Impove warning regarding lack of high quality entropy (#172)
xmlparse.c: Improve warning regarding lack of high quality entropy (#172
hartwork commentedon Oct 26, 2019
There was no reply from the original reporter for quite a while. I have improved the in-code warning text a bit at 515f5d0 now and will try to provide another round of answers to the concrete questions from the report, and then close the ticket as fixed until a re-open is requested.
Correct. If a single binary needs to support stone age glibc and Linux, then
XML_DEV_URANDOMis all you have. For a bit more recent environments,HAVE_SYSCALL_GETRANDOMmight be a good compromise.The last is no option, the others have different pros and cons that would need to be evaluated on a case-by-case basis.
It protects against hash flooding where crafted XML can be abused to keep the CPU busy for an unhealthy long time so it can be used for denial of service.
The runtime cost of extracting the entropy is negligible to my best knowledge.
The benefit an increase of entropy to a point where the attacker will not be able to craft XML matching the hash seed.
I am not aware of any of them blocking after #92 was merged for Expat >=2.2.3.
Closing, please request a re-open as needed.
Update documentation on use of XML_POOR_ENTOPY on SOlaris
Update documentation on use of XML_POOR_ENTOPY on Solaris
Update documentation on use of XML_POOR_ENTOPY on Solaris
noloader commentedon Feb 26, 2022
@hartwork,
A quick comment on this. It is merely food for thought. I am not suggesting you should revisit this or make changes or open another bug report.
SIpHash might be a good fallback choice. It is designed for this type of use case. The idea is you fetch a seed, and then key SipHash. Then you hash the data and get a tag that is uniformly distributed. SipHash offers 64-bit and 128-bit tags to use for your buckets.
SipHash also runs fast. SipHash (2,4) runs at 1.44 cycles per byte (cpb) on modern x86_64. SipHash (4,8) runs at 2.55 cpb on modern x86_64. ARM A-32 and Aaarch64 run at relatively similar speeds (ARM is a little slower).
SipHash is based on AES. AES is used to process the seed into a SipHash key. Once SipHash is keyed it does not use AES any longer. SipHash is secure as long as AES is secure. Attackers should not be able to craft collisions as long as the seed is secret.
Update documentation on use of XML_POOR_ENTOPY on Solaris