Don't keep the store open in by_store_ctrl_ex#28198
Don't keep the store open in by_store_ctrl_ex#28198mattcaswell wants to merge 3 commits intoopenssl:masterfrom
Conversation
|
I guess it also fixes #28065. |
|
I assume this will revert the performance improvement reported in #27529 (comment) though. Unfortunately I am afraid there is no way around that. Will you take it out of draft @mattcaswell ? |
|
Unfortunately, there's the risk that there are errors when reopening the URI in Mind you, I do understand the reasons for this PR, but it seems we're weighing one bad outcome against another. |
|
Can you clarify why cache_objects can't report errors in any sensible way? |
Sure. It goes back to #27461, which was about STORE URIs not being reported on when giving them (when So |
|
And there is no cache per location? |
|
Not sure I understand the question... |
That risk existed even in #27529. In #27529 it only reused the currently open store once. After that it reopened it again anyway (because it closed the store at the end of |
I think there is more work to be done. I'd like to confirm the performance impact (which I hope is negligible) - and I'd like to write a test. Unfortunately that won't happen until next week at the earliest. |
Oh, good point |
That performance improvement was seen in the x509storeissuer performance test. I've run this test on my local machine with 16 threads (the default case you see the performance graphs) both before and after this patch. In each case I ran it three times: Before this patch: After this patch: Any performance impact appears negligible. |
Previously openssl#27529 made a change to `by_store_ctrl_ex` in order to open the OSSL_STORE early. The reason given in that PR is: "This way, we can call OSSL_STORE_open_ex() in by_store_ctrl_ex(), and get to see possible errors when the URI is loaded" That PR then kept the store open until cache_objects is called and then reused it. Unfortunately by the time cache_objects() is called we could be in a multi-threaded scenario where the X509_STORE is being shared by multiple threads. We then get a race condition where multiple threads are all using (and ultimately closing) the same `OSSL_STORE_CTX`. The purpose of keeping the `OSSL_STORE` object between by_store_ctrl_ex() and `cache_objects` is presumably an optimisation to avoid having to open the store twice. But this does not work because of the above issue. We just take the hit and open it again. Fixes openssl#28171
aa9c77a to
bdaa78c
Compare
Check we don't have any threading issues when accessing an X509_STORE simultaneously
bdaa78c to
238a29e
Compare
|
I've now added a test for this and taken it out of draft. Please review. Ping @openssl/committers. |
When looking in the stack of objects in the store we need to ensure we are holding a read lock for the store. Issue detected via thread sanitizer after the test from the previous commit was added. Reviewed-by: Saša Nedvědický <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from #28198) (cherry picked from commit 07f65e1)
|
Merged to the master, 3.5, 3.4, 3.3 and 3.2 branches (fixing trivial conflicts). The test case will need bigger adjustments for 3.0. @mattcaswell please submit a backport to 3.0 as a separate PR. Thank you. |
|
Back for 3.0 in #28385 |
CHANGES.md: * openssl#28198 * openssl#28398 * openssl#28411 * openssl#28447 * openssl#28449 NEWS.md: * openssl#28447 Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]>
CHANGES.md: * openssl#28198 * openssl#28398 * openssl#28411 * openssl#28447 * openssl#28449 NEWS.md: * openssl#28447 Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]>
CHANGES.md: * #28198 * #28398 * #28411 * #28447 * #28449 NEWS.md: * #28447 Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from #28558)
3.0.18 CHANGES.md includes the following: * openssl#28098 * openssl#28198 * openssl#28398 * openssl#28411 * openssl#28449 * openssl#28504 * openssl#28535 * openssl#28591 * openssl#28624 Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]>
3.2.6 CHANGES.md includes the following: * openssl#28098 * openssl#28198 * openssl#28398 * openssl#28411 * openssl#28449 * openssl#28504 * openssl#28535 * openssl#28591 * openssl#28603 * openssl#28624 * openssl#28642 3.2.6 NEWS.md do not have any updates. Updated the changes and news in the previous branches. Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]>
3.3.5 CHANGES.md includes the following: * openssl#28098 * openssl#28198 * openssl#28398 * openssl#28411 * openssl#28449 * openssl#28504 * openssl#28535 * openssl#28591 * openssl#28603 * openssl#28624 * openssl#28642 3.3.5 NEWS.md do not have any updates. Updated the changes and news in the previous branches. Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]>
3.4.3 CHANGES.md includes the following: * openssl#28098 * openssl#28198 * openssl#28398 * openssl#28411 * openssl#28415 * openssl#28449 * openssl#28504 * openssl#28535 * openssl#28591 * openssl#28603 * openssl#28624 * openssl#28642 3.4.3 NEWS.md do not have any updates. Updated the changes and news in the previous branches. Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]>
3.3.5 CHANGES.md includes the following: * openssl#28098 * openssl#28198 * openssl#28398 * openssl#28411 * openssl#28449 * openssl#28504 * openssl#28535 * openssl#28591 * openssl#28603 * openssl#28624 * openssl#28642 3.3.5 NEWS.md do not have any updates. Updated the changes and news in the previous branches. Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]>
3.4.3 CHANGES.md includes the following: * openssl#28098 * openssl#28198 * openssl#28398 * openssl#28411 * openssl#28415 * openssl#28449 * openssl#28504 * openssl#28535 * openssl#28591 * openssl#28603 * openssl#28624 * openssl#28642 3.4.3 NEWS.md do not have any updates. Updated the changes and news in the previous branches. Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]>
3.2.6 CHANGES.md includes the following: * openssl#28098 * openssl#28198 * openssl#28398 * openssl#28411 * openssl#28449 * openssl#28504 * openssl#28535 * openssl#28591 * openssl#28603 * openssl#28624 * openssl#28642 3.2.6 NEWS.md do not have any updates. Updated the changes and news in the previous branches. Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]>
3.0.18 CHANGES.md includes the following: * openssl#28098 * openssl#28198 * openssl#28398 * openssl#28411 * openssl#28449 * openssl#28504 * openssl#28535 * openssl#28591 * openssl#28624 Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]>
3.4.3 CHANGES.md includes the following: * #28198 * #28398 * #28411 * #28415 * #28449 Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Tomas Mraz <[email protected]>
3.3.5 CHANGES.md includes the following: * #28198 * #28398 * #28411 * #28449 Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Tomas Mraz <[email protected]>
3.2.6 CHANGES.md includes the following: * #28198 * #28398 * #28411 * #28449 Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Tomas Mraz <[email protected]>
3.0.18 CHANGES.md includes the following: * #28198 * #28398 * #28411 * #28449 Release: Yes Signed-off-by: Eugene Syromiatnikov <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Tomas Mraz <[email protected]>
3.2.6 has fixed 3.2.5 regression which broke python3 ptests so we can upgrade now. We can also drop CVE-2025-27587 patch which was taken instead of 3.2.5 upgrade under: openssl/openssl#28198 Release information: https://github.com/openssl/openssl/blob/openssl-3.0/NEWS.md#major-changes-between-openssl-3017-and-openssl-3018-30-sep-2025 OpenSSL 3.2.6 is a security patch release. The most severe CVE fixed in this release is Moderate. This release incorporates the following bug fixes and mitigations: * Fix Out-of-bounds read & write in RFC 3211 KEK Unwrap. (CVE-2025-9230) * Fix Timing side-channel in SM2 algorithm on 64 bit ARM. (CVE-2025-9231) * Fix Out-of-bounds read in HTTP client no_proxy handling. (CVE-2025-9232) Release information: https://github.com/openssl/openssl/blob/openssl-3.2/NEWS.md#major-changes-between-openssl-324-and-openssl-325-1-jul-2025 OpenSSL 3.2.5 is a bug fix release. This release incorporates the following bug fixes and mitigations: * Miscellaneous minor bug fixes. (From OE-Core rev: 64c05c878253978980edf74903873b3d3f860b93) Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Steve Sakoman <[email protected]>
3.2.6 has fixed 3.2.5 regression which broke python3 ptests so we can upgrade now. We can also drop CVE-2025-27587 patch which was taken instead of 3.2.5 upgrade under: openssl/openssl#28198 Release information: https://github.com/openssl/openssl/blob/openssl-3.0/NEWS.md#major-changes-between-openssl-3017-and-openssl-3018-30-sep-2025 OpenSSL 3.2.6 is a security patch release. The most severe CVE fixed in this release is Moderate. This release incorporates the following bug fixes and mitigations: * Fix Out-of-bounds read & write in RFC 3211 KEK Unwrap. (CVE-2025-9230) * Fix Timing side-channel in SM2 algorithm on 64 bit ARM. (CVE-2025-9231) * Fix Out-of-bounds read in HTTP client no_proxy handling. (CVE-2025-9232) Release information: https://github.com/openssl/openssl/blob/openssl-3.2/NEWS.md#major-changes-between-openssl-324-and-openssl-325-1-jul-2025 OpenSSL 3.2.5 is a bug fix release. This release incorporates the following bug fixes and mitigations: * Miscellaneous minor bug fixes. (From OE-Core rev: f6db0fad3b9dc4649501da3545f7292400702444) Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Steve Sakoman <[email protected]>
3.2.6 has fixed 3.2.5 regression which broke python3 ptests so we can upgrade now. We can also drop CVE-2025-27587 patch which was taken instead of 3.2.5 upgrade under: openssl/openssl#28198 Release information: https://github.com/openssl/openssl/blob/openssl-3.0/NEWS.md#major-changes-between-openssl-3017-and-openssl-3018-30-sep-2025 OpenSSL 3.2.6 is a security patch release. The most severe CVE fixed in this release is Moderate. This release incorporates the following bug fixes and mitigations: * Fix Out-of-bounds read & write in RFC 3211 KEK Unwrap. (CVE-2025-9230) * Fix Timing side-channel in SM2 algorithm on 64 bit ARM. (CVE-2025-9231) * Fix Out-of-bounds read in HTTP client no_proxy handling. (CVE-2025-9232) Release information: https://github.com/openssl/openssl/blob/openssl-3.2/NEWS.md#major-changes-between-openssl-324-and-openssl-325-1-jul-2025 OpenSSL 3.2.5 is a bug fix release. This release incorporates the following bug fixes and mitigations: * Miscellaneous minor bug fixes. (From OE-Core rev: f6db0fad3b9dc4649501da3545f7292400702444) Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Steve Sakoman <[email protected]>
3.2.6 has fixed 3.2.5 regression which broke python3 ptests so we can upgrade now. We can also drop CVE-2025-27587 patch which was taken instead of 3.2.5 upgrade under: openssl/openssl#28198 Release information: https://github.com/openssl/openssl/blob/openssl-3.0/NEWS.md#major-changes-between-openssl-3017-and-openssl-3018-30-sep-2025 OpenSSL 3.2.6 is a security patch release. The most severe CVE fixed in this release is Moderate. This release incorporates the following bug fixes and mitigations: * Fix Out-of-bounds read & write in RFC 3211 KEK Unwrap. (CVE-2025-9230) * Fix Timing side-channel in SM2 algorithm on 64 bit ARM. (CVE-2025-9231) * Fix Out-of-bounds read in HTTP client no_proxy handling. (CVE-2025-9232) Release information: https://github.com/openssl/openssl/blob/openssl-3.2/NEWS.md#major-changes-between-openssl-324-and-openssl-325-1-jul-2025 OpenSSL 3.2.5 is a bug fix release. This release incorporates the following bug fixes and mitigations: * Miscellaneous minor bug fixes. (From OE-Core rev: f8a7f941c9ac1dbcaeb8cddec2b1e7388c6eded0) Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Steve Sakoman <[email protected]>
3.2.6 has fixed 3.2.5 regression which broke python3 ptests so we can upgrade now. We can also drop CVE-2025-27587 patch which was taken instead of 3.2.5 upgrade under: openssl/openssl#28198 Release information: https://github.com/openssl/openssl/blob/openssl-3.0/NEWS.md#major-changes-between-openssl-3017-and-openssl-3018-30-sep-2025 OpenSSL 3.2.6 is a security patch release. The most severe CVE fixed in this release is Moderate. This release incorporates the following bug fixes and mitigations: * Fix Out-of-bounds read & write in RFC 3211 KEK Unwrap. (CVE-2025-9230) * Fix Timing side-channel in SM2 algorithm on 64 bit ARM. (CVE-2025-9231) * Fix Out-of-bounds read in HTTP client no_proxy handling. (CVE-2025-9232) Release information: https://github.com/openssl/openssl/blob/openssl-3.2/NEWS.md#major-changes-between-openssl-324-and-openssl-325-1-jul-2025 OpenSSL 3.2.5 is a bug fix release. This release incorporates the following bug fixes and mitigations: * Miscellaneous minor bug fixes. Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Steve Sakoman <[email protected]>
3.2.6 has fixed 3.2.5 regression which broke python3 ptests so we can upgrade now. We can also drop CVE-2025-27587 patch which was taken instead of 3.2.5 upgrade under: openssl/openssl#28198 Release information: https://github.com/openssl/openssl/blob/openssl-3.0/NEWS.md#major-changes-between-openssl-3017-and-openssl-3018-30-sep-2025 OpenSSL 3.2.6 is a security patch release. The most severe CVE fixed in this release is Moderate. This release incorporates the following bug fixes and mitigations: * Fix Out-of-bounds read & write in RFC 3211 KEK Unwrap. (CVE-2025-9230) * Fix Timing side-channel in SM2 algorithm on 64 bit ARM. (CVE-2025-9231) * Fix Out-of-bounds read in HTTP client no_proxy handling. (CVE-2025-9232) Release information: https://github.com/openssl/openssl/blob/openssl-3.2/NEWS.md#major-changes-between-openssl-324-and-openssl-325-1-jul-2025 OpenSSL 3.2.5 is a bug fix release. This release incorporates the following bug fixes and mitigations: * Miscellaneous minor bug fixes. (From OE-Core rev: ef6bbf39c10ff7bd8ad36d5d2f59ddd0756e0141) Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Steve Sakoman <[email protected]>
Source: poky MR: 204763, 296373 Type: Security Fix Disposition: Merged from poky ChangeID: 2f0df03 Description: 3.2.6 has fixed 3.2.5 regression which broke python3 ptests so we can upgrade now. We can also drop CVE-2025-27587 patch which was taken instead of 3.2.5 upgrade under: openssl/openssl#28198 Release information: https://github.com/openssl/openssl/blob/openssl-3.0/NEWS.md#major-changes-between-openssl-3017-and-openssl-3018-30-sep-2025 OpenSSL 3.2.6 is a security patch release. The most severe CVE fixed in this release is Moderate. This release incorporates the following bug fixes and mitigations: * Fix Out-of-bounds read & write in RFC 3211 KEK Unwrap. (CVE-2025-9230) * Fix Timing side-channel in SM2 algorithm on 64 bit ARM. (CVE-2025-9231) * Fix Out-of-bounds read in HTTP client no_proxy handling. (CVE-2025-9232) Release information: https://github.com/openssl/openssl/blob/openssl-3.2/NEWS.md#major-changes-between-openssl-324-and-openssl-325-1-jul-2025 OpenSSL 3.2.5 is a bug fix release. This release incorporates the following bug fixes and mitigations: * Miscellaneous minor bug fixes. (From OE-Core rev: ef6bbf39c10ff7bd8ad36d5d2f59ddd0756e0141) Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Steve Sakoman <[email protected]> Signed-off-by: Jeremy A. Puhlman <[email protected]>
Previously #27529 made a change to
by_store_ctrl_exin order to open the OSSL_STORE early. The reason given in that PR is:"This way, we can call OSSL_STORE_open_ex() in by_store_ctrl_ex(), and get to see possible errors when the URI is loaded"
That PR then kept the store open until cache_objects is called and then reused it. Unfortunately by the time cache_objects() is called we could be in a multi-threaded scenario where the X509_STORE is being shared by multiple threads. We then get a race condition where multiple threads are all using (and ultimately closing) the same
OSSL_STORE_CTX.The purpose of keeping the
OSSL_STOREobject between by_store_ctrl_ex() andcache_objectsis presumably an optimisation to avoid having to open the store twice. But this does not work because of the above issue.We just take the hit and open it again.
Fixes #28171