Convert ossltest engine to a provider by nhorman · Pull Request #28461 · openssl/openssl

nhorman · 2025-09-06T14:45:03Z

Replace the ossltest engine with a provider

As part of our effort to remove support for ENGINES in openssl, we have to remove the ossltest engine. This engine is the core of several of our unit tests (the TLSProxy code relies on its, and in turn most/all of our tls unit tests rely on TLSProxy).

As such, to preserve the ability to run those tests we need to re-create all the algorithms that the ossltest engine implements as a provider. Note all of these algorithms are dummy versions of algorithms in the default provider that instead return known/predictable data for the purposes of testing (hence our need to re-implement them rather than just use those in the default provider)

This PR does the following:

Creates a provider that implements:
a) AES-128-CBC, AES-128-GCM and AES-128-CBC-HMAC-SHA1 ciphers
b) MD5, SHA1, SHA256, SHA384 and SHA512 message digests
c) The CTR-DRBG random number generator
Replaces the use of the ossltest engine with the provider in (1) in TLSProxy.pm
Updates all the dependent tests to set OPENSSL_MODULES to the test dir so that the provider can be found
Removes the gating of the tests in (3) such that the tests are no longer skipped if engines aren't enabled
Updates the test_rand and test_dgst methods to directly use this provider rather than the ossltest engine
Removes tests that rely on the loader_attic (which is typically used with ossltest) as, based on input from @levitte we have a default provider which implement the file:store loader, making these test effectively redundant.

Checklist

tests are added or updated

test/p_ossltest.c

slontis

crypto/evp contains p_ files.. Maybe dont propagate this name..

ossltest_prov or prov_ossltest would be fine

test/p_ossltest.c

test/recipes/05-test_rand.t

util/perl/TLSProxy/Proxy.pm

test/recipes/90-test_store.t

nhorman · 2025-09-10T15:29:34Z

@bob-beck

test/recipes/20-test_dgst.t

test/recipes/70-test_renegotiation.t

test/recipes/70-test_sslcertstatus.t

crypto/evp/mac_lib.c

paulidale · 2025-09-17T00:23:10Z

Merged to master. Thanks for the improvement.

Part of the effort to remove engines creates a problem for our test suite, in that we have a large number of tests that rely on the use of a test engine (ossltest), which implements the aes-128-cbc, aes-128-gcm, aes-128-cbc-hmac-sha1 ciphers, several digests and a random number generator to produce predictable outputs for the purposes of doing testing against known values. Since we're getting rid of engines, these tests need to be updated to use a provider that presents the same functionality. This commit implements that provider. Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from #28461)

Replace ossltest engine with ossltest provider in test_rand Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from #28461)

Use the new ossltest provider rather than the ossltest engine Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from #28461)

replace use of ossltest engine with provider in TLSProxy and update all dependent tests Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from #28461)

@levitte

With the removal of engines we need to handle the loader_attic test that will fail with said removal based on the advice of @levitte, given that we have a file: loader in the default provider already, theres no need to test an engine thats going away, so just remove it. Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from #28461)

This engine is going away (in fact they all are), so just remove the test cases referencing this engine Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from #28461)

We have several composite alg usages (i.e. MAC/KDF) which pick the right digest implementation when using an engine, but fail to get the right one when using a provider because we don't pass the propquery in a parameter to their instantiation. Fix them up by constructing the appropriate parameters Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from #28461)

This function fails to construct a param list that includes the passed in property query string in the param lists when allocating subordonate algorithms. Make sure we allow callers to pass a param list (so that providers for subordonate algorithms can be selected), and merge those into the param list that this function builds on its own. Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Shane Lontis <[email protected]> Reviewed-by: Paul Dale <[email protected]> (Merged from #28461)

nhorman · 2025-09-17T00:29:02Z

master!? This was targeted to go to the feature/engineremoval feature branch. I guess we can just rebase the feature branch, but it would be better if we could merge it to the proper location

t8m · 2025-09-17T05:53:50Z

Given this work is useful regardless of whether the engine removal will be completed or not, I think it is better now just to rebase the feature branch than revert this on master and merge it to the feature branch. But yeah, @openssl/committers please be more careful when merging!

mattcaswell · 2025-09-17T07:11:02Z

Should we have a "feature: branch" label to make it clearer what the intent is? Personally I always try to remember to check the labels when merging to know where something was intended to go.

nhorman · 2025-09-17T12:21:20Z

@mattcaswell another alternative (or additional) measure, might be to alter ghmerge to not just target master by default, but rather query the PR being merged via the github REST api to fetch the target branch for a PR and use that as a default.

It wouldn't catch everything, but it would prevent casual misses on branch selection.

mattcaswell · 2025-09-17T12:31:49Z

That would help, but isn't foolproof. (This particular fool doesn't use ghmerge).

We're removing the engine, so we don't need to test this anymore. NOTE: This also removes the engine skip check from the test, and this breaks testing until such time as PR openssl#28461 is merged (which replaces the remaining engine test with a provider).

We're removing the engine, so we don't need to test this anymore. NOTE: This also removes the engine skip check from the test, and this breaks testing until such time as PR #28461 is merged (which replaces the remaining engine test with a provider). Reviewed-by: Richard Levitte <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from #28525)

We're removing the engine, so we don't need to test this anymore. NOTE: This also removes the engine skip check from the test, and this breaks testing until such time as PR openssl#28461 is merged (which replaces the remaining engine test with a provider). Reviewed-by: Richard Levitte <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#28525)

We're removing the engine, so we don't need to test this anymore. NOTE: This also removes the engine skip check from the test, and this breaks testing until such time as PR #28461 is merged (which replaces the remaining engine test with a provider). Reviewed-by: Richard Levitte <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Saša Nedvědický <[email protected]> Reviewed-by: Eugene Syromiatnikov <[email protected]> Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Norbert Pocs <[email protected]> (Merged from #29305)

We're removing the engine, so we don't need to test this anymore. NOTE: This also removes the engine skip check from the test, and this breaks testing until such time as PR openssl#28461 is merged (which replaces the remaining engine test with a provider). Reviewed-by: Richard Levitte <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Saša Nedvědický <[email protected]> Reviewed-by: Eugene Syromiatnikov <[email protected]> Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Norbert Pocs <[email protected]> (Merged from openssl#29305)

nhorman self-assigned this Sep 6, 2025

nhorman added the triaged: feature The issue/pr requests/adds a feature label Sep 6, 2025

nhorman marked this pull request as draft September 6, 2025 14:45

nhorman requested review from Sashan, andrewkdinh, esyr, jogme, levitte, mattcaswell, t-j-h and t8m September 6, 2025 14:45

nhorman linked an issue Sep 6, 2025 that may be closed by this pull request

translate ossltest engine to a provider openssl/project#1336

Closed

nhorman force-pushed the osstesttoprovider branch from 9346d61 to 24790e6 Compare September 7, 2025 15:25