Skip to content

Comments

Fix some undefined behaviour in the Curve448 code (2nd attempt)#6838

Closed
mspncp wants to merge 1 commit intoopenssl:masterfrom
mspncp:pr-add-sha3-update-length-check
Closed

Fix some undefined behaviour in the Curve448 code (2nd attempt)#6838
mspncp wants to merge 1 commit intoopenssl:masterfrom
mspncp:pr-add-sha3-update-length-check

Conversation

@mspncp
Copy link
Contributor

@mspncp mspncp commented Aug 1, 2018

[extended tests]

Fixes #6800 (Alternative to #6819)
Replaces #5418

This commit reverts commit 5b56dd1 and moves the check for a zero-length input down the callstack into sha3_update().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check is analogous to md32_common.h, L135.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, looking at the implemenation of sha3_update(), I can't see any undefined behaviour even in the absence of this check, since memcpy(dest, src, 0) is well behaved and a no-op, provided dest and src are valid pointers. What's your opinion @dot-asm?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sad. Because it's still a non-problem. Indeed, what does language standard say about memcpy? "The memcpy function copies n characters from the object pointed to by s2 into the object pointed to by s1. If copying takes place between objects that overlap, the behavior is undefined." Full stop. This means that passing zero length is in fact a no-op(*). Moreover, what does sanitizer documentation say? "While violating nullability does not have undefined behavior, it is often unintentional, so UBSan offers to catch it." Again, "does not"! This is just sad. It's sad to become a hostage of somebody else's idea about how bad programmers are. I mean it's offered with rationale that people would appreciate it, right? But program would crash anyway, so what [real] value does it actually add? ... Anyway, this means that whole premise that it's about undefined behaviour is essentially wrong. It's not about undefined behavour, but about sanitizer being overzealous. And with this in mind wouldn't it be more appropriate to just pass empty string as argument to first test in curve448_internal_test.c?

(*) One might be able to argue that memcpy(NULL,NULL,0) would be undefined, because NULL overlaps NULL. Even if one can, it's not case here, only one argument is NULL.

Copy link
Contributor Author

@mspncp mspncp Aug 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One might be able to argue that memcpy(NULL,NULL,0) would be undefined, because NULL overlaps NULL.

I object! If one thing is for sure then the fact that two empty ranges never overlap regardless of what src and dest pointers are, because the intersection of two empty sets is always empty ;-)

But seriously: I almost anticipated that you would dislike this check, that's why I asked you for your opinion. I wasn't worrying about undefined behaviour because of the overlap, but because of the invalid memory references. My understanding was - admittedly taken from second-hand sources like this one - that the no-op guarantee was given only in the case where src and dest point to valid memory:

Yes, the C standard explicitly addresses this in §7.24.1/2, which applies to memcpy and all other functions from string.h

Where an argument declared as size_t n specifies the length of the array for a function, n can have the value zero on a call to that function. Unless explicitly stated otherwise in the description of a particular function in this subclause, pointer arguments on such a call shall still have valid values, as described in 7.1.4. On such a call, a function that locates a character finds no occurrence, a function that compares two character sequences returns zero, and a function that copies characters copies zero characters

It seems however that they are referring to a newer version of the standard and that ISO C90 is not mentioning the fact of invalid memory references at all, as you already wrote, citing section 7.11.2.1 The memcpy function of the standard.

So what do you suggest? Revert only Matt's commit and don't add the extra check to m_sha3.c? Then how do we deal with this specific ubsan warning? Can it be disabled selectively? Wouldn't it be better to just keep the check in order to reassure all worried programmers and stop the need to have endless discussions?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I almost anticipated that you would dislike this check,

It's rather that I dislike things that are done for wrong reason. But of course, if memcpy(0,0,0) is legitimately(*) undefined, then among all suggested solutions, the one in this request is most appropriate. At least you can observe the check and potentially problematic call on same page [nor does it affect general-purpose proxy interface]. For completeness sake it's possible to disable it selectively with

#ifdef __clang__
__attribute__((no_sanitize("undefined")))
#endif
static int sha3_update(...

But I'd say it would be too much compiler specifics. I mean code preferably should have minimum possible compiler dependencies.

(*) It should be noted that standard can have bugs too ;-)

@mspncp
Copy link
Contributor Author

mspncp commented Aug 1, 2018

@mattcaswell in #5418 you wrote that the problem was reported by ubsan.

Ubsan found some issues in the Curve448 code. It also broke no-ec. This fixes both issues.

please note that the following travis jobs with enable-ubsan succeeded.

https://travis-ci.org/openssl/openssl/jobs/410977657
https://travis-ci.org/openssl/openssl/jobs/410977655

@mspncp
Copy link
Contributor Author

mspncp commented Aug 1, 2018

Erratum: my commit message refers to the wrong commit id:

This commit reverts commit 5b56dd1

The correct formulation is:

This commit reverts commit 7876dbf

the former is the original commit from the pull request, the latter the final commit after merging. It will be changed on the next force-push.

Fixes openssl#6800
Replaces openssl#5418

This commit reverts commit 7876dbf and moves the check for a
zero-length input down the callstack into sha3_update().
@mattcaswell
Copy link
Member

For reference this is the job that failed originally that initiated the first attempt to fix this:
https://travis-ci.org/openssl/openssl/jobs/343818027

@mattcaswell mattcaswell added this to the 1.1.1 milestone Aug 2, 2018
Copy link
Member

@mattcaswell mattcaswell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved assuming there is no -1 from @dot-asm.

@mattcaswell mattcaswell added the approval: done This pull request has the required number of approvals label Aug 2, 2018
@mspncp
Copy link
Contributor Author

mspncp commented Aug 2, 2018

Thanks for your approval. Since there has been a lot of discussion and since there is a competing pull request which has also been approved, I will wait at least another 24 hours for feedback from @paulidale e.a., before merging.

I will however force push a new commit in a minute which contains no changes except for the aforementioned correction of the commit message.

@mspncp mspncp force-pushed the pr-add-sha3-update-length-check branch from 6f5b08a to 93f3247 Compare August 2, 2018 11:44
Copy link
Contributor

@paulidale paulidale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with this.

I still wonder if the length check wouldn't better be in EVP_DigestUpdate though. We'd be able to remove the same check from all the digests on the next API change revision.

@dot-asm dot-asm mentioned this pull request Aug 3, 2018
levitte pushed a commit that referenced this pull request Aug 3, 2018
Fixes #6800
Replaces #5418

This commit reverts commit 7876dbf and moves the check for a
zero-length input down the callstack into sha3_update().

Reviewed-by: Matt Caswell <[email protected]>
Reviewed-by: Andy Polyakov <[email protected]>
Reviewed-by: Paul Dale <[email protected]>
(Merged from #6838)
@mspncp
Copy link
Contributor Author

mspncp commented Aug 3, 2018

Ok, merged. Thanks, everybody!

@mspncp mspncp closed this Aug 3, 2018
mspncp added a commit to mspncp/openssl that referenced this pull request Aug 4, 2018
The majority of callers does not check for an empty buffer (buffer_length == 0) before calling
EVP_Digest*Update(), because all implementations handle this case correctly as no-op.
(See discussions in openssl#6800, openssl#6817, and openssl#6838.)
@mspncp mspncp deleted the pr-add-sha3-update-length-check branch October 19, 2018 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approval: done This pull request has the required number of approvals

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants