THIS IS A VERY EARLY DRAFT!

It is incomplete and doesn't even build right. I just wanted to expose the idea of being able to switch the keytype of the EC key structure.

Heh, it turns out that this must wait on #11248, since there are tests that generate an SM2 key in this PR, and then have signature failing because that hasn't been implemented yet.

Heh, it turns out that this must wait on #11248, since there are tests that generate an SM2 key in this PR, and then have signature failing because that hasn't been implemented yet.

I expect #11248 will remove the WIP soon

I think we should discuss the approach of mixing SM2 and EC keygen.

• Why should the provider care about the legacy quirkiness of libcrypto in handling SM2 and EC as if they were the same? Wouldn't it be better if the provider implementation was simpler (at the cost of small repetition in non critical code)?
  • if framework asks for an SM2 key generation the SM2 key generation is performed
  • if framework asks for an EC key generation (even on the curve named SM2) an EC key generation is performed
  • push the "you asked for an EC key on SM2, take instead this SM2 key" logic inside libcrypto
• The current implementation of keygen for SM2 in 1.1.1 (and in this PR) is slightly incorrect: the ranges for the random scalar differ (see SM2 private key cannot produce valid signatures #8435). In 1.1.1 we got stuck because EC and SM2 share too much code about keygen, here in the provider keeping SM2 and EC keygen more separate could be better.

I think we should discuss the approach of mixing SM2 and EC keygen.
@romen spoke thusly:

• Why should the provider care about the legacy quirkiness of libcrypto in handling SM2 and EC as if they were the same? Wouldn't it be better if the provider implementation was simpler (at the cost of small repetition in non critical code)?
  • if framework asks for an SM2 key generation the SM2 key generation is performed
  • if framework asks for an EC key generation (even on the curve named SM2) an EC key generation is performed
  • push the "you asked for an EC key on SM2, take instead this SM2 key" logic inside libcrypto

Generally speaking, it's a good question, and it basically comes down to "no surprise". Essentially, the way the new legacy code works is by detection, i.e. change the key type based on curve name, currently only for SM2, but hang on, it'll be more interesting as I continue 😉.

For the "push" that you mention, i.e. check at generation if the curve name was "SM2", and in that case, switch over to ask for an SM2 keymgmt instead of the EC keymgmt... this involves much more, and I think that this also ties into what you've talked about elsewhere, to be able to have keytypes that contain a bit more information, for example to be able to specify, say, the keytype "EC-P239v1".
So basically, you're saying that EVP_PKEY_gen() should:

• detect that it's handling a key type that has possible sub-categories. Right now, that would be detecting that the key type is "EC"
• figure out the sub-category. Right now and if the key type is "EC", that involves getting the curve name, create a new key type composed of the key type "EC", a dash, and the curve name. If the curve name is "P239v1", the resulting string is "EC-P239v1"

So question is, when happens with the next key type that has similar properties, but is otherwise different? On the vid call today, you mentioned RSA (I supposed you meant things like RSASSA-PSS and RSAES-OAEP), so we'd have to hack in special knowledge about those. And then there was something about DSA...

Do you see where I'm going with this, and how things could become quite "interesting" indeed?

• The current implementation of keygen for SM2 in 1.1.1 (and in this PR) is slightly incorrect: the ranges for the random scalar differ (see SM2 private key cannot produce valid signatures #8435). In 1.1.1 we got stuck because EC and SM2 share too much code about keygen, here in the provider keeping SM2 and EC keygen more separate could be better.

I'll have to study that issue. I wonder if that could even be the issue to kill EVP_PKEY_set_alias_type() (I thoroughly dislike that function, it's not well thought out)...

For SM2 my argument is slightly different than for ECDH/ECDSA over different curves.

• SM2 is its own cryptosystem, it has its specific keygen and its specific sign/verify algorithms.
• ECDSA is its own cryptosystem with its keygen and sign/verify algorithms.
• Both SM2 and ECDSA (and ECDH) are based on elliptic curve cryptography, so they use the EC module, the same way RSA and DSA use the BN module, but they are strictly independent from an operational point of view just as RSA and DSA.
• Both SM2 and ECDSA have as a parameter the elliptic curve on which to run the algorithms: for EC many different curves are standardized and commonly in use, for SM2 at the moment there is only one curve that the standard recommends for use (and it is what we informally refer to as the "SM2 curve"), but nothing prevents the definition of new SM2 recommended curves in the future (e.g. to increase the level of security one could expect the definition of a >512 bits curve, as the current one is around 256 bits), and already now the SM2 standard allows for arbitrary prime and binary curves with the full set of parameters we have for traditional ECDSA/ECDH (even though it is not clear to me how such keys would be serialized), nor we should prevent applications from running ECDSA/ECDH over the "SM2 curve".
• The only real point of contact between SM2 and ECDSA(/DH) is that SM2 does not specify its own distinct serialization format for keys
  • Instead implementations are reusing the SECG serialization used for ECDSA/ECDH keys, but instead of filling the "named curve" identifier member with a curve OID, OID 1.2.156.10197.1.301 for "SM2" elliptic curve cryptography is used
  • This OID is not properly a curve OID, and at the moment I don't even know how an SM2 key on another curve would be serialized: actually 1.2.156.10197.1.301 it's even weirder because usually it's leaf nodes in the "OID tree" that are used as identifiers, but in this case 301 represents a class as it has child nodes!
  • An EC curve over the "SM2 curve" at the moment could only be serialized using explicit parameters, because among the many SM2 related OIDs none is officially identifying the curve itself

So, originally, the issue of coercing "EC(DSA/DH) over SM2 ECC" serialized objects into "SM2" memory objects, could have been an issue to be handled only on deserialization, but the way we handled SM2 in 1.1.1 has created a problem that we have to deal with:

• SM2 was treated as a subtype of ECDSA/ECDH: programmatically one created an EC_KEY on the "SM2 curve" and magically/hackishly the application ended up with a peculiar kind of EC_KEY object that instead of running ECDSA/ECDH like any other EC_KEY ran SM2 as the signing algorithm.
  • There might be applications out there that would expect this behavior of the API to keep working in 3.0
  • My belief is that providing compatibility with legacy somewhat quirky behaviour is a job for the framework and not for the providers that should register EC and/or SM2 keymgmt algorithms.
  • IMO we should avoid mixing again EC and SM2 keys and keep the provider interface simple, so if the framework asks an EC keymgmt for a key it should get back an EC key object, similarly an SM2 keymgmt should be able to generate only an SM2 key object: there shouldn't be the notion of EC keymgmt returning an SM2 object.

Relevant standards

Using https://github.com/alipay/tls13-sm-spec and https://github.com/metanorma/gmt-0009-2012 for English translations.
More details about different standardization bodies and the relationship between different standards can be found at https://github.com/alipay/tls13-sm-spec#chinese-algorithm-standards

Last one shows some ingredients for encoding private and public keys, but is not complete.
I couldn't find the sources for the registration of the various OIDs related to SM2, or at least not in English.

ECDSA is its own cryptosystem with its keygen ...

Wait what?

IMO we should avoid mixing again EC and SM2 keys and keep the provider interface simple, so if the framework asks an EC keymgmt for a key it should get back an EC key object, similarly an SM2 keymgmt should be able to generate only an SM2 key object: there shouldn't be the notion of EC keymgmt returning an SM2 object.

I wholeheartedly agree with this. Upon a closer look, that won't be very much of a breaking change either. Basically, EVP_PKEY_set_alias_type() will have to stop working. All other SM2 things are in master only, so there's quite ample space to get things right.

ECDSA is its own cryptosystem with its keygen ...

Wait what?

Well, technically it is not false, it happens that both ECDH and ECDSA were defined to share the key generation algorithm.

I was oversimplifying, and focusing on the digital signature side of things, as we can see from the table above also SM2 can be declined not only in a digital signature algorithm, but also in key exchange/derivation (a la ECDH) and for public key encryption.

I said:

Upon a closer look, that won't be very much of a breaking change either. Basically, EVP_PKEY_set_alias_type() will have to stop working. All other SM2 things are in master only, so there's quite ample space to get things right.

I didn't look close enough, it's a bit more than that.

Having looked around at the code in crypto/ec, I've decided to slowly and quietly back out from legacy SM2 stuff, and do what @romen proposes and keep that strictly separate in provider code.
(in other words, I'm removing the SM2 keygen from this PR and reworking it in another... as to the legacy SM2 implementation, I frankly have zero desire to dive into that rabbit hole, especially in view of our schedule)

In the same vein, I'm move away the keytype change feature from this PR, even though I see uses in the future (if we do decide to "upgrade" EC keys to composite "EC-{grp}" key types)...

I reworked this to really only include provider side EC key generation, and let `EVP_PKEY_gen() detect the SM2 curve, and downgrade it to a legacy key. This is as close to emulating what legacy EC key generation currently does as I can imagine.

This is not a draft any more. Comments welcome

This is not a draft any more. Comments welcome

Is it still pending on other PRs? Is the WIP label still appropriate?

Is it still pending on other PRs?

At least #11358, which quickly fixes the issues with EVP_PKEY_get0_EC
I've removed the dependency on #11248, though

Is the WIP label still appropriate?

I want to try this PR merged with #11358 to see that they want to play well together. I'll remove the WIP marker when I've been able to do so.

Still a couple of travis failures remaining (issues noted above)

I could have sworn I had pushed exactly those fixes, @mattcaswell. Now done...

This pull request is ready to merge

Merged

2b9add6 KEYMGMT: Add functions to get param/key generation parameters
1f185f5 PROV: Implement EC param / key generation
10d756a EC: Refactor EVP_PKEY_CTX curve setting macros for param generation
813d317 EVP: Add a temporary SM2 hack to key generation
49276c3 EVP: fix memleak in evp_pkey_downgrade()