The return value of RSA_*_{en,de}crypt() is signed#29323
The return value of RSA_*_{en,de}crypt() is signed#29323vdukhovni wants to merge 1 commit intoopenssl:masterfrom
Conversation
The functions RSA_(public|private)_(en|de)crypt() return a signed result, in particular `-1` may be returned on error, so the caller MUST treat the value as signed.
vdukhovni
force-pushed
the
fix-rsa-result-sign
branch
from
13aa6f1 to
eee2273
Compare
github-actions
bot
added
the
severity: fips change
Sashan
added
the
approval: review pending
mattcaswell
left a comment
mattcaswell
left a comment
There was a problem hiding this comment.
LGTM.
Side note: plaintext_len is changed to a signed type here to match the return value of RSA_size. AFAICT, that function never returns a negative value, although the documentation is silent about what happens on error. If it ever were to return a negative value, then the subsequent OPENSSL_calloc in this code would go wrong - attempting to allocate a very large positive value. But its probably fair to assume RSA_size will only return a value >=0.
mattcaswell
added
approval: done
|
What branches are we targeting this change for? |
I hadn't looked into backports, but the code that introduced the unsigned variables to handle the signed returned values was added in 3.4 as part of #23832 So we could backport to 3.4 if that seems appropriate. |
vdukhovni
added
branch: 3.4
t8m
added
triaged: bug
|
The change looks fine. |
|
24 hours has passed since 'approval: done' was set, but as this PR has been updated in that time the label 'approval: ready to merge' is not being automatically set. Please review the updates and set the label manually. |
t8m
added
approval: ready to merge
The functions RSA_(public|private)_(en|de)crypt() return a signed result, in particular `-1` may be returned on error, so the caller MUST treat the value as signed. Reviewed-by: Saša Nedvědický <[email protected]> Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from #29323)
|
Merged to the master, 3.6, 3.5 and 3.4 branches. Thank you. |
The functions RSA_(public|private)_(en|de)crypt() return a signed result, in particular `-1` may be returned on error, so the caller MUST treat the value as signed. Reviewed-by: Saša Nedvědický <[email protected]> Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from #29323) (cherry picked from commit f247d36)
The functions RSA_(public|private)_(en|de)crypt() return a signed result, in particular `-1` may be returned on error, so the caller MUST treat the value as signed. Reviewed-by: Saša Nedvědický <[email protected]> Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from #29323) (cherry picked from commit f247d36)
The functions RSA_(public|private)_(en|de)crypt() return a signed result, in particular `-1` may be returned on error, so the caller MUST treat the value as signed. Reviewed-by: Saša Nedvědický <[email protected]> Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from #29323) (cherry picked from commit f247d36)
The functions RSA_(public|private)_(en|de)crypt() return a signed result, in particular `-1` may be returned on error, so the caller MUST treat the value as signed. Reviewed-by: Saša Nedvědický <[email protected]> Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#29323)
6 participants
The functions RSA_(public|private)_(en|de)crypt() return a signed result, in particular
-1may be returned on error, so the caller MUST treat the value as signed.Checklist