What version of Go are you using (go version)?

$ go version
go version go1.14 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

$ go env
GOARCH="amd64"
GOHOSTOS="darwin"

What did you do?

1. Run on a mac with a large but not unreasonable number of certificates.
2. Try to connect to https endpoint with a binary compiled with CGO_ENABLED=0
3. Set a connection timeout of a few seconds
4. Watch the connection time out

Even without a timeout, the initial response can take many seconds. The default mac has ~22 certificates and it can take 3-4 seconds. Any corporate system with more certs can take many multiples.

What did you expect to see?

Reasonable return times

What did you see instead?

Long waits and timeouts.

Lots More Detail

I spent quite some time tracking it down. When compiling with CGO_ENABLED=1, which is fine, although may require more tools installed and, critically, makes cross-compiling and some CI nearly impossible, the lookup time is roughly well below O(n). When compiling with CGO_ENABLED=0, which is desired when possible, every cert in the system increases the time to verify the server cert.

The core code issue is in root_darwin.go, and specifically execSecurityRoots().

It does the following (simplified):

1. gets the root certs via exec'ing /usr/bin/security with appropriate args
2. checks each of the root certs by exec'ing /usr/bin/security on it with appropriate args

The above is, essentially, O(n), and rises linearly with the number of certs. The comments here indicate that it should be optimized to reasonably close to with CGO, but I haven't found that to be the case. I have had several runs where 160 certs and a timeout of 5 seconds would, well, time out.

I created a repo with a program which recreates it. It also includes directions and output from my mac. In my case, it was 244ms for CGO and 1.7s for no CGO. I worked with someone who had ~160 certs (not unreasonable); it took well over 5 seconds.

repo here