Skip to content

Comments

SERIALIZER: New API for serialization of objects through providers#10394

Closed
levitte wants to merge 59 commits intoopenssl:masterfrom
levitte:serialize-me
Closed

SERIALIZER: New API for serialization of objects through providers#10394
levitte wants to merge 59 commits intoopenssl:masterfrom
levitte:serialize-me

Conversation

@levitte
Copy link
Member

@levitte levitte commented Nov 9, 2019

Serialization is needed to be able to take a provider object (such as
the provider side key data) and output it in PEM form, DER form, text
form (for display), and possibly other future forms (XML? JSON? JWK?)

The idea is that a serializer should be able to handle objects it has
intimate knowledge of, as well as object data in OSSL_PARAM form. The
latter will allow libcrypto to serialize some object with a different
provider than the one holding the data, if exporting of that data is
allowed and there is a serializer that can handle it.

We will provide serializers for the types of objects we know about,
which should be useful together with any other provider that provides
implementations of the same type of object.

Serializers are selected by method name and a couple of additional
properties:

  • format
    used to tell what format the output should be in.
    Possibilities could include "format=text",
    "format=pem", "format=der", "format=pem-pkcs1"
    (traditional), "format=der-pkcs1" (traditional)
  • type
    used to tell exactly what type of data should be
    output, for example "type=public" (the public part of
    a key), "type=private" (the private part of a key),
    "type=domainparams" (domain parameters).
Checklist
  • documentation is added or updated
  • tests are added or updated

@levitte levitte changed the title [WIP] SERIALIZER: New API for serialization of objects through providers [WIP, Pending on #10390, #10391] SERIALIZER: New API for serialization of objects through providers Nov 9, 2019
@levitte
Copy link
Member Author

levitte commented Nov 9, 2019

Background

when implementing key and domparam generation (#10289), I realised that we couldn't do much with the generated keys, apart from signing, verifying, that sort of thing. But no way whatsoever to save them.

Some kind of solution was needed, and after a conversation with @mattcaswell, we came to the conclusion that the smoothest would be to provide serializers, and to allow the serializers to handle both provider side objects directly (they have to come from the same provider as the keymgmt that manages the key itself) as well as key data in OSSL_PARAM array form (which allows a serializer to handle anything it recognises coming from any other provider, as long as that provider allows export of said data).

Why not an EVP API?

Serializers may be extended some day to handle any type of data, such as X.509, which isn't an EVP layer object. Therefore, serializers need to be a higher level class.

@levitte
Copy link
Member Author

levitte commented Nov 9, 2019

If anyone wonders, the mechanism this offers would allow us to extract the data from a key made in the FIPS provider module, and serialize it using serializers in the default provider. This is by design.

@levitte levitte changed the title [WIP, Pending on #10390, #10391] SERIALIZER: New API for serialization of objects through providers [Pending on #10390, #10391] SERIALIZER: New API for serialization of objects through providers Nov 11, 2019
@levitte
Copy link
Member Author

levitte commented Nov 11, 2019

This is no longer WIP, so can take reviews

@levitte
Copy link
Member Author

levitte commented Nov 11, 2019

I just realised that I do have DER serializers, but haven't hooked them in anywhere. So er, not as done as I thought...

@levitte levitte changed the title [Pending on #10390, #10391] SERIALIZER: New API for serialization of objects through providers [WIP, Pending on #10390, #10391] SERIALIZER: New API for serialization of objects through providers Nov 11, 2019
@levitte
Copy link
Member Author

levitte commented Nov 11, 2019

(don't let that stop anyone from looking at what's already here, though)

@levitte levitte changed the title [WIP, Pending on #10390, #10391] SERIALIZER: New API for serialization of objects through providers [WIP, Pending on #10412, #10414] SERIALIZER: New API for serialization of objects through providers Nov 12, 2019
@levitte
Copy link
Member Author

levitte commented Nov 12, 2019

I've updated this branch to depend on #10412 and #10414

openssl-machine pushed a commit that referenced this pull request Nov 29, 2019
openssl-machine pushed a commit that referenced this pull request Nov 29, 2019
The BIO_vprintf() will allow the provider to print any text, given a
BIO supplied by libcrypto.

Additionally, we add a provider library with functions to collect all
the currently supplied BIO upcalls, as well as wrappers around those
upcalls.

Reviewed-by: Matt Caswell <[email protected]>
(Merged from #10394)
openssl-machine pushed a commit that referenced this pull request Nov 29, 2019
To support generic output of public keys wrapped in a X509_PUBKEY,
additional PEM and i2d/d2i routines are added for that type.

Reviewed-by: Matt Caswell <[email protected]>
(Merged from #10394)
openssl-machine pushed a commit that referenced this pull request Nov 29, 2019
This also adds the missing accessor RSA_get0_pss_params(), so those
parameters can be included in the PKCS#8 data structure without
needing to know the inside of the RSA structure.

Reviewed-by: Matt Caswell <[email protected]>
(Merged from #10394)
openssl-machine pushed a commit that referenced this pull request Nov 29, 2019
openssl-machine pushed a commit that referenced this pull request Nov 29, 2019
openssl-machine pushed a commit that referenced this pull request Nov 29, 2019
This renames test/evp_fromdata_test.c to test/evp_pkey_provided_test.c,
to encourage additional testing of provider made keys.

Reviewed-by: Matt Caswell <[email protected]>
(Merged from #10394)
openssl-machine pushed a commit that referenced this pull request Nov 29, 2019
openssl-machine pushed a commit that referenced this pull request Nov 29, 2019
The prototype checks shouldn't be performed on SYNOPSIS lines that
aren't function prototypes.

Reviewed-by: Matt Caswell <[email protected]>
(Merged from #10394)
openssl-machine pushed a commit that referenced this pull request Nov 29, 2019
These are helper macros alongside the IMPLEMENT_ and DECLARE_ macros.

Reviewed-by: Matt Caswell <[email protected]>
(Merged from #10394)
@levitte
Copy link
Member Author

levitte commented Nov 29, 2019

Merged.

3d83c73 CORE: ossl_namemap_add_names(): new function to add multiple names
36fa4d8 CORE: pass the full algorithm definition to the method constructor
0d003c5 SERIALIZER: New API for serialization of objects through providers
742496f SERIALIZER: add functions for serialization to file
1793d27 CORE: expose the property parsers and checker to the rest of the libraries
866234a SERIALIZER: add support for serializing EVP_PKEYs
f864a93 SERIALIZER: add hooks in PEM_write_bio_ and PEM_write_fp_ routines
54c1711 SERIALIZER: add hooks in EVP_PKEY_print_ routines
63665ff PROV BIO: add a BIO_vprintf() upcall, and a provider BIO library
cb58d81 PROV SERIALIZER: add common functionality to serialize keys
677add3 PROV SERIALIZER: add support for writing RSA keys
045e51c PROV SERIALIZER: add support for writing DH keys and parameters
264b789 PROV SERIALIZER: add support for writing DSA keys and parameters
6ae5543 TEST: add tests of text and PEM printout of a provider made key
c48e2d1 Add NEWS and CHANGES entries about OSSL_SERIALIZER
31d3a75 util/find-doc-nits: limit the prototype check
14ee781 util/find-doc-nits: ignore macros ending in _fnsig

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approval: ready to merge The 24 hour grace period has passed, ready to merge branch: master Applies to master branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants