Skip to content

Too optimistic heuristics when auto detecting the private key format #15022

@beldmit

Description

@beldmit

d2i_AutoPrivateKey function that is used for the auto-detection of the key type looks for the number of elements in the SEQUENCE of the DER key, using the number of elements as a detector of key type.

{
STACK_OF(ASN1_TYPE) *inkey;
const unsigned char *p;
int keytype;
p = *pp;
/*
* Dirty trick: read in the ASN1 data into a STACK_OF(ASN1_TYPE): by
* analyzing it we can determine the passed structure: this assumes the
* input is surrounded by an ASN1 SEQUENCE.
*/
inkey = d2i_ASN1_SEQUENCE_ANY(NULL, &p, length);
p = *pp;
/*
* Since we only need to discern "traditional format" RSA and DSA keys we
* can just count the elements.
*/
if (sk_ASN1_TYPE_num(inkey) == 6)
keytype = EVP_PKEY_DSA;
else if (sk_ASN1_TYPE_num(inkey) == 4)
keytype = EVP_PKEY_EC;
else if (sk_ASN1_TYPE_num(inkey) == 3) { /* This seems to be PKCS8, not
* traditional format */
PKCS8_PRIV_KEY_INFO *p8 = d2i_PKCS8_PRIV_KEY_INFO(NULL, &p, length);
EVP_PKEY *ret;
sk_ASN1_TYPE_pop_free(inkey, ASN1_TYPE_free);
if (!p8) {
ASN1err(ASN1_F_D2I_AUTOPRIVATEKEY,
ASN1_R_UNSUPPORTED_PUBLIC_KEY_TYPE);
return NULL;
}
ret = EVP_PKCS82PKEY(p8);
PKCS8_PRIV_KEY_INFO_free(p8);
if (ret == NULL)
return NULL;
*pp = p;
if (a) {
*a = ret;
}
return ret;
} else
keytype = EVP_PKEY_RSA;
sk_ASN1_TYPE_pop_free(inkey, ASN1_TYPE_free);
return d2i_PrivateKey(keytype, a, pp, length);
}

At least one of the Russian GOST commercial implementations adds an extra proprietary element to the SEQUENCE, so 3 elements turn into 4, and the GOST key is started being parsed as EC key, unsuccessfully.

When the PEM format is used, the PRIVATE KEY header indicates the format, and no heuristics is used, so the PKCS#8 PEM parser is used.

Don't know when this behavior was introduced. It seems that it was done last 6 months, though I can be mistaken.

Metadata

Metadata

Assignees

No one assigned

    Labels

    branch: 1.1.1Applies to OpenSSL_1_1_1-stable branch (EOL)triaged: bugThe issue/pr is/fixes a bug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions