Very early WIP, TLS tests are completely broken.

In travis it doesnt get past

make update
Due to s390 related code..

Failure in ./test/evp_test ./test/recipes/30-test_evp_data/evppkey.txt is in fips mode as these are not in the fips module.
You may need to seperate them out into another .txt file and add into the .t file.

It looks like you broke it good :)

Pauli just having a cursory look at a failure in ssl_test_old:

I think it generates one of your new keys and then runs..

static int add_key_share(SSL *s, WPACKET pkt, unsigned int curve_id)
{
....
/ Encode the public key. */
encodedlen = EVP_PKEY_get1_tls_encodedpoint(key_share_key,
&encoded_point);

which then calls rv = evp_pkey_asn1_ctrl(pkey, ASN1_PKEY_CTRL_GET1_TLS_ENCPT, 0, ppt);

which probably then calls legacy_asn1_ctrl_to_param which will not handle ASN1_PKEY_CTRL_GET1_TLS_ENCPT and then return -2;

@mattcaswell you may already be aware of this?

I added these in FIPS mode. I suspect there are things that have been missed so far.
Or I've missed thing that I shouldn't have.

@mattcaswell you may already be aware of this?

Yes - I made the same note here wrt ec keymgmt:

#11328 (comment)

Failing in the TLS tests, but finally passing the EVP ones.

This seems to be close now, the TLS tests are rubbish but otherwise it seems to work. After building this branch:

$ ./util/opensslwrap.sh genpkey -algorithm ED25519 -out ed.pem
$ ./util/opensslwrap.sh pkeyutl -sign -rawin -in /tmp/f -inkey ed.pem -out sig

Then on a fresh 3.0 (master) build:

$ ./util/opensslwrap.sh pkeyutl -verify -rawin -in /tmp/f -inkey ed.pem -sigfile sig

I get:

Signature Verified Successfully

Where /tmp/f is a trivial test file being verified.

Can this be rebased? Will the various fixes that @levitte put in help with the TLS failures now?

Just needs a rebase (to address the merge conflicts) and this fix, and all the tests will start passing:

diff --git a/crypto/evp/pmeth_gn.c b/crypto/evp/pmeth_gn.c
index 67800282de..9a7cdcf591 100644
--- a/crypto/evp/pmeth_gn.c
+++ b/crypto/evp/pmeth_gn.c
@@ -155,8 +155,9 @@ int EVP_PKEY_gen(EVP_PKEY_CTX *ctx, EVP_PKEY **ppkey)
 
         if (keydata == NULL)
             goto not_supported;
-        ret = evp_keymgmt_gen_set_template(ctx->keymgmt,
-                                           ctx->op.keymgmt.genctx, keydata);
+        if (ctx->keymgmt->gen_set_template != NULL)
+            ret = evp_keymgmt_gen_set_template(ctx->keymgmt,
+                                            ctx->op.keymgmt.genctx, keydata);
     }
 
     /*

I've removed the commit. The CIs are expected to fail at this point.

Once #11550 is merged, I'll rebase and confirm the CIs pass.

24 hours has passed since 'approval: done' was set, but as this PR has been updated in that time the label 'approval: ready to merge' is not being automatically set. Please review the updates and set the label manually.

Travis is happy, merged. Thanks for all the input and assistance.