identity: Proxy fails to validate certificates with name constraints

Linkerd cannot be deployed when its CA certificate(s) use name constraints. Proxies fail to validate issued certificates.

## Repro/Investigation

1. Generate a self-signed certificate with name constraints limiting issued
   certificates to `foo.bar`:

bar.tpl:
```json
{
    "subject": {{ toJson .Subject }},
    "issuer": {{ toJson .Subject }},
    "keyUsage": ["certSign"],
    "basicConstraints": {
        "isCA": true,
        "maxPathLen": 1
    },
    "nameConstraints": {
        "critical": true,
        "permittedDNSDomains": ["foo.bar"]
    }
}
```

```sh
:; step certificate create foo.bar cert.pem key.pem --template=bar.tpl --not-after=24h --no-password --insecure
```
```
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 223894858428966857916253664437917250458 (0xa87094cb1d1a378f60aa362777a4f39a)
    Signature Algorithm: ECDSA-SHA256
        Issuer: CN=foo.bar
        Validity
            Not Before: Aug 30 20:07:42 2022 UTC
            Not After : Aug 31 20:07:42 2022 UTC
        Subject: CN=foo.bar
        Subject Public Key Info:
            Public Key Algorithm: ECDSA
                Public-Key: (256 bit)
                X:
                    10:f9:57:fa:30:30:93:4d:9b:e1:64:20:54:a1:1a:
                    20:76:28:5e:31:c4:6f:19:f6:62:9d:fe:9b:68:e8:
                    14:ee
                Y:
                    58:aa:27:73:08:24:78:a9:57:f1:35:fe:53:d9:2d:
                    83:01:d4:fd:c3:b6:44:26:16:06:a0:fb:6f:d8:4b:
                    6b:c8
                Curve: P-256
        X509v3 extensions:
            X509v3 Key Usage: critical
                Certificate Sign
            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:1
            X509v3 Subject Key Identifier:
                A3:E8:20:63:F8:F4:35:D0:0D:47:79:B7:AD:F3:CD:3B:EF:2C:74:AE
            X509v3 Name Constraints: critical
                Permitted:
                  DNS: foo.bar
    Signature Algorithm: ECDSA-SHA256
         30:45:02:20:60:b2:5c:96:44:30:0c:5b:fb:63:98:90:7d:60:
         7e:75:ce:88:81:70:8a:01:97:e8:f5:e4:29:30:bf:87:04:ad:
         02:21:00:ed:fa:f6:71:15:c0:26:99:ec:c7:9a:a3:4d:e6:5f:
         e0:e4:d0:fb:9e:3a:3e:30:26:0f:90:25:ba:0a:17:6f:52
```

2. Install Linkerd using this certificate:

```
:; linkerd install --crds |k apply -f -
```
```
:; linkerd install \
    --set identityTrustDomain="foo.bar" \
    --set-file identity.issuer.tls.crtPEM=cert.pem \
    --set-file identity.issuer.tls.keyPEM=key.pem \
    --set-file identityTrustAnchorsPEM=cert.pem \
    |k apply -f -
```

3. The control plane does not start.

The identity controller becomes 'running' and it appears to issue a certificate
to its local proxy:

```console
:; k logs -n linkerd linkerd-identity-5c9d8bbb4c-n8n8n
Defaulted container "identity" out of: identity, linkerd-proxy, linkerd-init (init)
time="2022-08-30T20:13:40Z" level=info msg="running version stable-2.12.0"
time="2022-08-30T20:13:40Z" level=info msg="starting admin server on :9990"
time="2022-08-30T20:13:40Z" level=info msg="starting gRPC server on :8080"
time="2022-08-30T20:13:46Z" level=info msg="issued certificate for linkerd-identity.linkerd.serviceaccount.identity.linkerd.foo.bar until 2022-08-31 20:07:42 +0000 UTC: 4a28173a08c2e9629eeb982e8b2699b6c6007eef8b2377b31eeb2a6149bb371f"
```

But the proxy fails to validate the certificate:

```console
:; k logs -n linkerd linkerd-identity-5c9d8bbb4c-n8n8n -c linkerd-proxy
...
[     0.011665s] ERROR ThreadId(02) identity: linkerd_proxy_identity_client::certify: Failed to obtain identity error=invalid peer certificate contents: invalid peer certificate: UnknownIssuer
```

The rest of the control plane fails to start because the identity controller never becomes ready.

---

This process works when name constraints are omitted from the certificate:

```json
{
    "subject": {{ toJson .Subject }},
    "issuer": {{ toJson .Subject }},
    "keyUsage": ["certSign"],
    "basicConstraints": {
        "isCA": true,
        "maxPathLen": 1
    }
}
```

```
:; step certificate create foo.bar cert.pem key.pem --template=bar-unconstrained.tpl --not-after=24h --no-password --insecure
```
```
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 22516031807147259098312418109158923000 (0x10f06e13bbe38643d7ce95ec4cf91af8)
    Signature Algorithm: ECDSA-SHA256
        Issuer: CN=foo.bar
        Validity
            Not Before: Aug 30 20:33:12 2022 UTC
            Not After : Aug 31 20:33:12 2022 UTC
        Subject: CN=foo.bar
        Subject Public Key Info:
            Public Key Algorithm: ECDSA
                Public-Key: (256 bit)
                X:
                    74:e5:86:db:e6:68:1d:a5:c7:1d:fd:21:29:2d:69:
                    de:f1:0a:57:7c:30:35:f1:9d:c4:c2:d3:61:cf:42:
                    09:19
                Y:
                    ab:00:91:94:2c:df:e5:7f:4b:13:8e:40:e5:04:07:
                    e9:0e:db:c9:03:83:a2:fb:42:80:37:cb:35:df:f4:
                    17:7d
                Curve: P-256
        X509v3 extensions:
            X509v3 Key Usage: critical
                Certificate Sign
            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:1
            X509v3 Subject Key Identifier:
                60:42:74:A2:23:17:6A:BD:94:1E:0B:87:4C:A8:70:7B:B8:BD:AD:55
    Signature Algorithm: ECDSA-SHA256
         30:45:02:20:63:24:33:92:8a:71:65:d3:9f:fd:da:44:ab:40:
         1e:74:6e:0c:58:f5:8d:93:98:31:76:be:e4:96:7a:97:a5:e2:
         02:21:00:a1:a0:93:4d:0e:bc:96:0c:af:99:4a:34:0a:40:5f:
         b6:d1:72:d5:ed:16:85:8a:3a:c4:e1:5e:90:43:80:e0:15
```

```
:; linkerd identity -n linkerd linkerd-identity-75f7d4df68-wd8km

POD linkerd-identity-75f7d4df68-wd8km (1 of 1)

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1 (0x1)
    Signature Algorithm: ECDSA-SHA256
        Issuer: CN=foo.bar
        Validity
            Not Before: Aug 30 20:38:17 2022 UTC
            Not After : Aug 31 20:38:27 2022 UTC
        Subject: CN=linkerd-identity.linkerd.serviceaccount.identity.linkerd.foo.bar
        Subject Public Key Info:
            Public Key Algorithm: ECDSA
                Public-Key: (256 bit)
                X:
                    bb:43:be:97:a7:39:83:6c:e5:71:e1:17:71:c2:ef:
                    fc:64:fd:1e:cb:6f:5c:25:fe:8b:ce:1f:91:93:24:
                    40:bf
                Y:
                    07:89:d1:56:5d:8e:b7:b7:3c:9a:8a:a6:12:e3:19:
                    30:63:63:25:49:4b:8a:61:be:1f:5b:ed:f2:88:6c:
                    b8:79
                Curve: P-256
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Authority Key Identifier:
                keyid:5E:A4:79:4B:A0:EA:D3:8A:07:9F:14:8F:2A:77:E8:0E:0B:1A:4B:D7
            X509v3 Subject Alternative Name:
                DNS:linkerd-identity.linkerd.serviceaccount.identity.linkerd.foo.bar

    Signature Algorithm: ECDSA-SHA256
         30:45:02:20:4d:85:c6:af:4b:04:aa:eb:89:8f:c8:64:f3:49:
         42:60:b0:66:6d:1a:22:5d:e9:41:3c:40:9d:54:8f:e0:d8:1c:
         02:21:00:d5:ee:41:06:a1:ca:fb:1c:40:91:e4:35:75:68:ae:
         23:9e:f3:7b:24:b2:4d:14:a6:dd:66:0d:43:7c:5f:79:04
```

---

Note that we can manually create an end-entity certificate (as used by a proxy) with:

```sh
:; step certificate create linkerd-identity.linkerd.serviceaccount.identity.linkerd.foo.bar \
    ee-cert.pem ee-key.pem \
    --san=linkerd-identity.linkerd.serviceaccount.identity.linkerd.foo.bar \
    --ca-key=key.pem --ca=cert.pem \
    --not-after=24h \
    --no-password --insecure
```
```
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 55110646295972328840139681094716422534 (0x2975eca5308fa2fde788e681d188f986)
    Signature Algorithm: ECDSA-SHA256
        Issuer: CN=foo.bar
        Validity
            Not Before: Aug 30 20:53:31 2022 UTC
            Not After : Aug 31 20:53:31 2022 UTC
        Subject: CN=linkerd-identity.linkerd.serviceaccount.identity.linkerd.foo.bar
        Subject Public Key Info:
            Public Key Algorithm: ECDSA
                Public-Key: (256 bit)
                X:
                    52:b2:ad:f7:19:87:ef:f0:0c:e3:f4:4a:5c:3f:a1:
                    67:7b:b1:75:31:32:14:ba:42:56:f7:4a:58:b7:af:
                    fe:8c
                Y:
                    ed:fa:80:45:fb:30:51:8a:3e:86:bd:dd:0f:b8:75:
                    01:47:50:d7:c6:0a:24:ca:42:65:02:26:0a:36:db:
                    3d:4f
                Curve: P-256
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature
            X509v3 Extended Key Usage:
                Server Authentication, Client Authentication
            X509v3 Subject Key Identifier:
                0B:1B:29:6D:CF:76:77:A7:3E:40:7F:10:E6:0A:BD:69:69:BE:70:7B
            X509v3 Authority Key Identifier:
                keyid:5E:A4:79:4B:A0:EA:D3:8A:07:9F:14:8F:2A:77:E8:0E:0B:1A:4B:D7
            X509v3 Subject Alternative Name:
                DNS:linkerd-identity.linkerd.serviceaccount.identity.linkerd.foo.bar
    Signature Algorithm: ECDSA-SHA256
         30:44:02:20:3d:0a:2b:dd:49:21:a9:95:3c:49:48:17:54:d0:
         99:c9:f7:c7:7b:73:da:39:86:1b:27:8a:e6:9b:5c:c8:68:eb:
         02:20:3e:16:16:bd:ff:82:99:16:c8:f6:60:bb:e5:55:5f:f0:
         de:6e:02:73:37:eb:d8:3a:b4:37:96:31:4e:a8:ce:29
```

`step` happily verifies that the end-entity certificate is signed by the CA, even
when name constraints are present:

```sh
:; step certificate verify ee-cert.pem \
    --roots=cert.pem \
    --host=linkerd-identity.linkerd.serviceaccount.identity.linkerd.foo.bar
```

If we generate an end entity certificate for an alternate suffix (`idk.lol`),
`step` properly fails to validate the certificate.

```console
:; step certificate create linkerd-identity.linkerd.serviceaccount.identity.linkerd.idk.lol ee-cert.pem ee-key.pem --san=linkerd-identity.linkerd.serviceaccount.identity.linkerd.idk.lol --ca-key=key.pem --ca=cert.pem --not-after=24h --no-password --insecure
...
:; step certificate verify ee-cert.pem \
    --roots=cert.pem \
    --host=linkerd-identity.linkerd.serviceaccount.identity.linkerd.foo.bah
failed to verify certificate: x509: a root or intermediate certificate is not authorized to sign for this name: DNS name "linkerd-identity.linkerd.serviceaccount.identity.linkerd.foo.bah" is not permitted by any constraint
```

## Suggested next steps

Why doesn't the proxy validate its end entity certificates when name constraints are used?

To figure this out, I'd probably try to create a test (or repo) that attempts to
use `webpki` to validate these certificates. If this works in a standalone repo,
then we can try to figure out how Linkerd's usage differs from a simpler example
setup.

Note that it may be required to alter the format of the private key with, e.g.:

```sh
:; openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in key.pem -out pkcs8.key  
```

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

identity: Proxy fails to validate certificates with name constraints #9299

Repro/Investigation

Suggested next steps

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

identity: Proxy fails to validate certificates with name constraints #9299

Description

Repro/Investigation

Suggested next steps

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions