Tentative fix for the async engine#16734
Conversation
|
The "unsupported padding mode" error is actually caused by #16738. |
This is because daysnc's custom EVP_PKEY_METHOD does not have the diff --git a/engines/e_dasync.c b/engines/e_dasync.c
index e2e587d839..d7f45a046c 100644
--- a/engines/e_dasync.c
+++ b/engines/e_dasync.c
@@ -211,7 +211,8 @@ static int bind_dasync(ENGINE *e)
/* Setup RSA */
;
if ((dasync_rsa_orig = EVP_PKEY_meth_find(EVP_PKEY_RSA)) == NULL
- || (dasync_rsa = EVP_PKEY_meth_new(EVP_PKEY_RSA, 0)) == NULL)
+ || (dasync_rsa = EVP_PKEY_meth_new(EVP_PKEY_RSA,
+ EVP_PKEY_FLAG_AUTOARGLEN)) == NULL)
return 0;
EVP_PKEY_meth_set_init(dasync_rsa, dasync_rsa_init);
EVP_PKEY_meth_set_cleanup(dasync_rsa, dasync_rsa_cleanup); |
Actually I think the fixes in e_dasync.c are fine - although I would extend the same change to all the pkey init functions. |
Will do. Should I revert the rsa_pmeth changes? |
Yes. With the suggested |
|
While you are making changes to e_dasync.c you might also want to address #16735. See my comments there as to the cause. Something like this should do it: static void destroy_pkey(void)
{
- EVP_PKEY_meth_free(dasync_rsa);
+ /*
+ * We don't actually need to free the dasync_rsa method since this is
+ * automatically freed for us by libcrypto.
+ */
dasync_rsa_orig = NULL;
dasync_rsa = NULL;
} |
|
Will do later. Thanks! |
engines/e_dasync.c
Outdated
There was a problem hiding this comment.
return pparamgen_init != NULL ? pparamgen_init(ctx) : 1;There was a problem hiding this comment.
Did you push your fixes? I don't see them?
Fixes: openssl#16724 Fixes: openssl#16735
|
This pull request is ready to merge |
Fixes: #16724 Fixes: #16735 Reviewed-by: Matt Caswell <[email protected]> (Merged from #16734)
Fixes: #16724 Fixes: #16735 Reviewed-by: Matt Caswell <[email protected]> (Merged from #16734) (cherry picked from commit 59cd0bc)
|
Merged both to master and 3.0. Thanks! |
Related to #16724
It fixes the crashes, but I think it's incorrect fix
As a result of the proposed changes, s_server fails with output
BTW, the problem is we've lost the pattern 'call
EVP_PKEY_encryptwith NULL out ptr - get enclen - allocate memory and callEVP_PKEY_encryptagain':openssl/ssl/statem/statem_clnt.c
Lines 2860 to 2870 in 19e277d
If we don't change
pkey_rsa_encryptand just fix the dasync engine, we get a SEGFAULT with the following stack trace: