Skip to content

Comments

PROV: DER writer#11450

Closed
levitte wants to merge 11 commits intoopenssl:masterfrom
levitte:derlib
Closed

PROV: DER writer#11450
levitte wants to merge 11 commits intoopenssl:masterfrom
levitte:derlib

Conversation

@levitte
Copy link
Member

@levitte levitte commented Mar 31, 2020

When thinking about adding a lot more OID and AID generation, the thought of hand crafting more OID DER encoding made me cringe enough to bring in an OID reader/encoder that I'd already written and perl template support so we could get those encodings as C arrays.

The result is a ASN.1 reader that knows enough to find all OBJECT IDENTIFIER definitions in a given file and have that result in relevant C arrays. That way, we only need to copy OBJECT IDENTIFIER definitions from diverse standards and fire away.

Furthermore, this adds a small, quick and easily extensible DER writing library.

crypto/rsa/rsa_aid.c, crypto/dsa/dsa_aid.c, crypto/ec/ecdsa_aid.c are replaced by this solution.

@levitte levitte added branch: master Applies to master branch approval: review pending This pull request needs review by a committer labels Mar 31, 2020
@levitte levitte added this to the 3.0.0 milestone Mar 31, 2020
@levitte
Copy link
Member Author

levitte commented Mar 31, 2020

I fully intend to use this DER writer in #11422

@richsalz
Copy link
Contributor

Looks kinda neat. Karma points for removing the "aid" files.

Can we instead augment objects.txt somehow, or have a file that points into objects.txt so that we only have to write OID's in one place?

I assume that while you sometimes you shorthands "{ pkcs1 12 }" and other times you write the full OID is because that's what the source documents do.

Copy link
Member

@mattcaswell mattcaswell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran out of time reviewing this - but here's my comments so far

@beldmit
Copy link
Member

beldmit commented Mar 31, 2020

Will look a tomorrow

@levitte
Copy link
Member Author

levitte commented Mar 31, 2020

Can we instead augment objects.txt somehow, or have a file that points into objects.txt so that we only have to write OID's in one place?

I'd rather not. The objects.txt format is a despicable pile of hacks on top of a pile of hacks, and I'm just as culpable for getting it there as anyone. Having to rewrite RFC input to that format is a waste of cycles when it wasn't too hard to put together something that can take OBJECT IDENTIFIER definitions directly. I'd say it's better to start over fresh.

Besides, this is an internal thing for our providers, entirely separate from everything that has to do with objects.txt, and needed because we have banned OBJ and ASN1 routines from the FIPS module.

@richsalz
Copy link
Contributor

I guess that given your affinity for perl, you're okay with more than one way to do it :)

Bummer.

@levitte
Copy link
Member Author

levitte commented Mar 31, 2020

I guess that given your affinity for perl, you're okay with more than one way to do it :)

If it were entirely up to me, I'd replace objects.txt and the Perl 4 flavored code that supports it.

@richsalz
Copy link
Contributor

If it were entirely up to me, I'd replace objects.txt and the Perl 4 flavored code that supports it.

!!! please

@levitte
Copy link
Member Author

levitte commented Mar 31, 2020

!!! please

That's further in the future, mind...

@levitte
Copy link
Member Author

levitte commented Mar 31, 2020

I assume that while you sometimes you shorthands "{ pkcs1 12 }" and other times you write the full OID is because that's what the source documents do.

Yup. Straight copy, I only needed remove page footer and header when what i copied crossed from one RFC page to the next.

Copy link
Member

@slontis slontis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea - It is probably less error prone doing it this way.
But maybe a bit more cryptic for someone to set up.
Any time new DER functions are created it is a bit of a worry that some edge case is going to cause issues.

Copy link
Contributor

@paulidale paulidale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a good approach.

Travis looks relevant.

@levitte
Copy link
Member Author

levitte commented Apr 1, 2020

Any time new DER functions are created it is a bit of a worry that some edge case is going to cause issues.

There's that risk whichever way we turn... It shouldn't be too hard to write a test, though...

@levitte
Copy link
Member Author

levitte commented Apr 1, 2020

Travis is drunk and needs to catch up. Close/open kick happening now

@levitte levitte closed this Apr 1, 2020
@levitte levitte reopened this Apr 1, 2020
@levitte levitte mentioned this pull request Apr 1, 2020
@levitte
Copy link
Member Author

levitte commented Apr 1, 2020

Travis failure is a timeout

Copy link
Member

@mattcaswell mattcaswell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had more time to look at the rest of this. Further comments below.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure the shifting is quite right here. For example, if n == BN_BYTES then we shift by 0 bits. Surely in that case we should shift by 8 * (BN_BYTES - 1) bits.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. n == BN_BYTES is the lowest byte in word [1], not the highest byte in word [0].

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wait, now I see what you said...

@beldmit
Copy link
Member

beldmit commented Apr 1, 2020

So, did I correctly got that you've introduced provider-level set of OIDs and when we don't need corresponding NIDs for any purposes, most of, e.g. GOST OIDs can be removed from objects.txt and processed separately on provider-level?

@levitte
Copy link
Member Author

levitte commented Apr 1, 2020

So, did I correctly got that you've introduced provider-level set of OIDs and when we don't need corresponding NIDs for any purposes, most of, e.g. GOST OIDs can be removed from objects.txt and processed separately on provider-level?

One of the objectives with the new core effort is that there should be no reason whatsoever for a provider author to have any specific information "registered" with OpenSSL. We're not entirely there yet, libssl needs some deep mods to stop depending on registered NIDs.

@kaduk
Copy link
Contributor

kaduk commented Apr 5, 2020

I tried to look at the (internal) man pages, and for the 'cont' arguments, "context number of the element being handled" didn't really give me a clear picture. Is it supposed to be the ASN.1 tag value when explicit tags are used?

@levitte
Copy link
Member Author

levitte commented Apr 5, 2020

I tried to look at the (internal) man pages, and for the 'cont' arguments, "context number of the element being handled" didn't really give me a clear picture. Is it supposed to be the ASN.1 tag value when explicit tags are used?

Yes. I took the name because it's the context-specific tag, describe like this in X.690 (italics mine):

31.2.4 The new type is isomorphic with the old type, but has a tag with class "Class" and number "ClassNumber", except when "Class" is "empty", in which case the tag is context-specific class and number is "ClassNumber".

Feel free to propose a better text regarding this. As a matter of fact, I'd be grateful if you did.

@levitte
Copy link
Member Author

levitte commented Apr 5, 2020

Woot! CIs are happy at least 😄

@richsalz
Copy link
Contributor

richsalz commented Apr 5, 2020

ctag or similar for context tag? or asn1tag?

@kaduk
Copy link
Contributor

kaduk commented Apr 5, 2020

Feel free to propose a better text regarding this. As a matter of fact, I'd be grateful if you did.

I put some words up at levitte#2 . Okay, and some code, too.
I tried to keep things in separate commits so you can reject the more-invasive stuff if you want.

@levitte
Copy link
Member Author

levitte commented Apr 6, 2020

I put some words up at levitte#2 . Okay, and some code, too.

Taken!

I tried to keep things in separate commits so you can reject the more-invasive stuff if you want.

You're too modest, I didn't find any of it particularly invasive. It was all straight to the point. Thank you.

Copy link
Member

@mattcaswell mattcaswell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reconfirm

@levitte
Copy link
Member Author

levitte commented Apr 7, 2020

Merged.

77de6bb Add perl support to parse and DER encode ASN.1 OID specs
1d39620 PROV: Add the beginning of a DER writing library
6f5837d PROV: Add DERlib support for RSA
8c55580 PROV: Add DERlib support for DSA
2d956b3 PROV: Add DERlib support for ECDSA and EC keys

@levitte levitte closed this Apr 7, 2020
openssl-machine pushed a commit that referenced this pull request Apr 7, 2020
We have an old OID database that's not as readable as would be
desired, and we have spots with hand coded DER for well known OIDs.

The perl modules added here give enough support that we can parse
OBJECT IDENTIFIER definitions and encode them as DER.

OpenSSL::OID is a general OID parsing and encoding of ASN.1
definitions, and supports enough of the X.680 syntax to understand
what we find in RFCs and similar documents and produce the DER
encoding for them.

oids_to_c is a specialized module to convert the DER encoding from
OpenSSL::OID to C code.  This is primarily useful in file templates
that are processed with util/dofile.pl.

Reviewed-by: Matt Caswell <[email protected]>
(Merged from #11450)
openssl-machine pushed a commit that referenced this pull request Apr 7, 2020
This library is meant to be small and quick.  It's based on WPACKET,
which was extended to support DER writing.  The way it's used is a
bit unusual, as it's used to write the structures backward into a
given buffer.  A typical quick call looks like this:

    /*
     * Fill in this structure:
     *
     * something ::= SEQUENCE {
     *     id OBJECT IDENTIFIER,
     *     x [0] INTEGER OPTIONAL,
     *     y [1] BOOLEAN OPTIONAL,
     *     n INTEGER
     * }
     */
    unsigned char buf[nnnn], *p = NULL;
    size_t encoded_len = 0;
    WPACKET pkt;
    int ok;

    ok =   WPACKET_init_der(&pkt, buf, sizeof(buf)
        && DER_w_start_sequence(&pkt, -1)
        && DER_w_bn(&pkt, -1, bn)
        && DER_w_boolean(&pkt, 1, bool)
        && DER_w_precompiled(&pkt, -1, OID, sizeof(OID))
        && DER_w_end_sequence(&pkt, -1)
        && WPACKET_finish(&pkt)
        && WPACKET_get_total_written(&pkt, &encoded_len)
        && (p = WPACKET_get_curr(&pkt)) != NULL;

Reviewed-by: Matt Caswell <[email protected]>
(Merged from #11450)
openssl-machine pushed a commit that referenced this pull request Apr 7, 2020
This replaces crypto/rsa/rsa_aid.c with new code and generated OIDs

Reviewed-by: Matt Caswell <[email protected]>
(Merged from #11450)
openssl-machine pushed a commit that referenced this pull request Apr 7, 2020
This replaces crypto/dsa/dsa_aid.c with new code and generated OIDs

Reviewed-by: Matt Caswell <[email protected]>
(Merged from #11450)
openssl-machine pushed a commit that referenced this pull request Apr 7, 2020
This replaces crypto/ec/ecdsa_aid.c with new code and generated OIDs

Reviewed-by: Matt Caswell <[email protected]>
(Merged from #11450)
@levitte levitte deleted the derlib branch April 7, 2020 09:22
@richsalz
Copy link
Contributor

richsalz commented Apr 7, 2020

I think this was a very nice example of collaboration.

@levitte
Copy link
Member Author

levitte commented Apr 7, 2020

Yup, I thoroughly enjoyed that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approval: done This pull request has the required number of approvals branch: master Applies to master branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants