Skip to content

Fix miscellaneous engine problems#16846

Closed
mattcaswell wants to merge 10 commits intoopenssl:masterfrom
mattcaswell:fix-engine-problems
Closed

Fix miscellaneous engine problems#16846
mattcaswell wants to merge 10 commits intoopenssl:masterfrom
mattcaswell:fix-engine-problems

Conversation

@mattcaswell
Copy link
Member

This PR fixes 3 different problems:

  • engine reference mishandling in pkey_set_type
  • engine reference mishandling in provider_util.c
  • dasync bug when using EVP_CIPHER_CTX_copy()

Fixes #16757
Fixes #16845
Fixes #16844

This PR is targeting master and 3.0. A backport will need to be done for 1.1.1 (not all issues apply, and some tweaks will be required).

Note: investigating these issues highlighted that we have a behaviour change between 1.1.1 and 3.0 wrt CMAC keys. In 1.1.1 if you attempt to create a CMAC key with EVP_PKEY_new_CMAC_key(), but supply key data that is too short, then we notice this right away. However in 3.0 we don't notice it until you actually try and use the key. This means we'll need to tweak my test during backporting as well.

Ciphers in the daysnc engine were failing to copy their context properly
in the event of EVP_CIPHER_CTX_copy() because they did not define the
flag EVP_CIPH_CUSTOM_FLAG

Fixes openssl#16844
pkey_set_type should not consume the ENGINE references that may be
passed to it.

Fixes openssl#16757
provider_util.c failed to free ENGINE references when clearing a cipher
or a digest. Additionally ciphers and digests were not copied correctly,
which would lead to double-frees if it were not for the previously
mentioned leaks.

Fixes openssl#16845
Add some tests which would have caught the issues fixed in the previous
3 commits related to engine handling.
@mattcaswell mattcaswell added branch: master Applies to master branch approval: review pending This pull request needs review by a committer branch: 3.0 Applies to openssl-3.0 branch labels Oct 15, 2021
@mattcaswell
Copy link
Member Author

CIs for this are now green. Please take a look.

Copy link
Member

@t8m t8m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few nits. Otherwise LGTM.

@mattcaswell
Copy link
Member Author

All nits fixed and fixups pushed. Please take another look.

@t8m t8m added approval: done This pull request has the required number of approvals and removed approval: review pending This pull request needs review by a committer labels Oct 18, 2021
@openssl-machine openssl-machine added approval: ready to merge The 24 hour grace period has passed, ready to merge and removed approval: done This pull request has the required number of approvals labels Oct 19, 2021
@openssl-machine
Copy link
Collaborator

This pull request is ready to merge

openssl-machine pushed a commit that referenced this pull request Oct 19, 2021
Ciphers in the daysnc engine were failing to copy their context properly
in the event of EVP_CIPHER_CTX_copy() because they did not define the
flag EVP_CIPH_CUSTOM_FLAG

Fixes #16844

Reviewed-by: Tomas Mraz <[email protected]>
(Merged from #16846)
openssl-machine pushed a commit that referenced this pull request Oct 19, 2021
pkey_set_type should not consume the ENGINE references that may be
passed to it.

Fixes #16757

Reviewed-by: Tomas Mraz <[email protected]>
(Merged from #16846)
openssl-machine pushed a commit that referenced this pull request Oct 19, 2021
provider_util.c failed to free ENGINE references when clearing a cipher
or a digest. Additionally ciphers and digests were not copied correctly,
which would lead to double-frees if it were not for the previously
mentioned leaks.

Fixes #16845

Reviewed-by: Tomas Mraz <[email protected]>
(Merged from #16846)
openssl-machine pushed a commit that referenced this pull request Oct 19, 2021
Ciphers in the daysnc engine were failing to copy their context properly
in the event of EVP_CIPHER_CTX_copy() because they did not define the
flag EVP_CIPH_CUSTOM_FLAG

Fixes #16844

Reviewed-by: Tomas Mraz <[email protected]>
(Merged from #16846)

(cherry picked from commit a0cbc2d)
openssl-machine pushed a commit that referenced this pull request Oct 19, 2021
Add some tests which would have caught the issues fixed in the previous
3 commits related to engine handling.

Reviewed-by: Tomas Mraz <[email protected]>
(Merged from #16846)
openssl-machine pushed a commit that referenced this pull request Oct 19, 2021
pkey_set_type should not consume the ENGINE references that may be
passed to it.

Fixes #16757

Reviewed-by: Tomas Mraz <[email protected]>
(Merged from #16846)

(cherry picked from commit f7d6868)
openssl-machine pushed a commit that referenced this pull request Oct 19, 2021
provider_util.c failed to free ENGINE references when clearing a cipher
or a digest. Additionally ciphers and digests were not copied correctly,
which would lead to double-frees if it were not for the previously
mentioned leaks.

Fixes #16845

Reviewed-by: Tomas Mraz <[email protected]>
(Merged from #16846)

(cherry picked from commit 86c15ba)
openssl-machine pushed a commit that referenced this pull request Oct 19, 2021
Add some tests which would have caught the issues fixed in the previous
3 commits related to engine handling.

Reviewed-by: Tomas Mraz <[email protected]>
(Merged from #16846)

(cherry picked from commit 0299094)
@mattcaswell
Copy link
Member Author

Pushed to master and 3.0. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approval: ready to merge The 24 hour grace period has passed, ready to merge branch: master Applies to master branch branch: 3.0 Applies to openssl-3.0 branch

Projects

None yet

3 participants