Background

crypto/x509 comes with a pretty good chain builder and verifier, but it doesn’t ship a root store, the set of root CA certificates that leaf certificates need to chain to. Instead, x509.Verify either takes an explicit CertPool, or uses “the system roots”.

“The system roots” are mostly a lie (#39540). The idea is that the platform should be in charge of trust decisions, and that you’d want a Go CLI tool to work like the browser on the same machine (including trusting any “enterprise” or development roots). In practice no modern root store can be captured by a static list of roots: many have additional rules such as “this root can only sign certificates that end in .fr” or “this root can only sign this specific set of certificates” or “this root can only sign certificates logged to CT”. Moreover, platforms also have space for CA certificates that are not trusted, but can be used to build chains.

Our current API leaks the idea that system roots are just a list of trusted CAs: x509.SystemCertPool returns a CertPool that can be used with Verify like a CertPool full of custom certificates. It has no visible space for custom logic or untrusted intermediates.

On macOS, we go to great lengths to apply the complex root store logic to extract the list of certificates that are unconditionally trusted, by invoking the Security.framework APIs. This is fickle and complicated logic (golang.org/cl/227037, #38888, #42414), prone to security-relevant errors, slow (#19561), memory intensive (#26731), with no support for reloading (#41888, #35887), and with no support for untrusted intermediates (#35631).

On iOS, we don’t have access to the root store, so we hardcode a copy of it, which we do our best to keep in sync (#38843), but which has even harder to avoid memory usage issues.

On Windows, x509.SystemCertPool is not supported (#16736) and a Verify with a nil Roots invokes the platform verifier through the platform APIs.

On UNIX, there isn’t really a consistent platform root store story, and roots are mostly a list of certificates you can find in one of many somewhat arbitrary paths. Our logic for finding roots kind of grew organically and can be improved (#38869), but this is the platform for which the API is a better fit. There is no platform verifier.

Proposal

We’d like to move to using the platform verifier where available, so on macOS, iOS, and Windows. When Verify is invoked with a nil Roots, we will pass the rest of the VerifyOptions to the platform, and return the chain the platform built, if any. If Roots is set to a custom CertPool, we will keep using our chain builder.

In order not to break existing code that invokes x509.SystemCertPool and then adds extra custom certificates to it, we’ll have Verify perform two verifications in that case, one with the platform and one with the custom roots, and return chains from both.

This will resolve the loading time issue, the memory issue, the risk of mistakes in root extraction code, the need for reloading the root store, the lack of support for untrusted intermediates, and will more correctly apply the custom platform logic.

Open questions

Alignment

There are a few verification rules that we want to apply regardless of the platform:

1. Name Constraints are applied to all names on the certificate. This is critical because VerifyHostname can only see the leaf, so it needs to be able to trust that if Verify returned it, the certificate is authorized to claim all those names.

2. Extended Key Usage values are enforced nested down a chain.

3. IP addresses and certain invalid DNS names are supported, like in VerifyHostname.

If the platform verifier does not align with these rules, it might be hard to compensate for it, because these rules influence chain building itself.

SystemCertPool().Subjects()

The CertPool returned by SystemCertPool() will be just a placeholder that says “use the platform verifier”. That doesn’t work with the Subjects method, though, because the list of roots is not in fact available.

We have two options: 1) we keep all the complicated, slow, inaccurate, and out-of-sync macOS and iOS code to extract roots and only use it to populate Subjects, and leave SystemCertPool unsupported on Windows; or 2) we deprecate Subjects, making CertPool fully opaque.

(2) lets us easily implement Equal (#46057) and Copy (#35044), but not Contains (#39179).

/cc @rolandshoemaker @golang/security