This repository was archived by the owner on Jul 31, 2025. It is now read-only.
This repository was archived by the owner on Jul 31, 2025. It is now read-only.
Reduce package init costs #3717
Description
Is your feature request related to a problem? Please describe.
While testing the new init cost debugging output in go 1.16, I found that aws/endpoints has the highest package-level memory cost out of Hugo's many dependencies.
$ grep aws-sdk-go go.mod
github.com/aws/aws-sdk-go v1.35.0
$ GODEBUG=inittrace=1 go run . version 2>&1 | rg aws-sdk-go | tee time.txt
init github.com/aws/aws-sdk-go/internal/ini @30 ms, 0.008 ms clock, 2304 bytes, 22 allocs
init github.com/aws/aws-sdk-go/aws/credentials @30 ms, 0.002 ms clock, 496 bytes, 8 allocs
init github.com/aws/aws-sdk-go/aws/endpoints @30 ms, 0.98 ms clock, 852984 bytes, 5396 allocs
init github.com/aws/aws-sdk-go/aws @31 ms, 0.004 ms clock, 128 bytes, 2 allocs
init github.com/aws/aws-sdk-go/aws/awsutil @31 ms, 0.009 ms clock, 3416 bytes, 31 allocs
init github.com/aws/aws-sdk-go/aws/request @31 ms, 0.003 ms clock, 976 bytes, 9 allocs
init github.com/aws/aws-sdk-go/internal/sdkrand @31 ms, 0.044 ms clock, 5448 bytes, 3 allocs
init github.com/aws/aws-sdk-go/private/protocol @32 ms, 0 ms clock, 0 bytes, 0 allocs
init github.com/aws/aws-sdk-go/private/protocol/rest @32 ms, 0.003 ms clock, 56 bytes, 3 allocs
init github.com/aws/aws-sdk-go/aws/signer/v4 @32 ms, 0.011 ms clock, 1392 bytes, 4 allocs
init github.com/aws/aws-sdk-go/service/sts @32 ms, 0.002 ms clock, 0 bytes, 0 allocs
init github.com/aws/aws-sdk-go/aws/corehandlers @32 ms, 0.017 ms clock, 2184 bytes, 31 allocs
init github.com/aws/aws-sdk-go/private/protocol/json/jsonutil @32 ms, 0.002 ms clock, 112 bytes, 4 allocs
init github.com/aws/aws-sdk-go/aws/session @32 ms, 0.003 ms clock, 320 bytes, 5 allocs
init github.com/aws/aws-sdk-go/private/protocol/eventstream @40 ms, 0 ms clock, 0 bytes, 0 allocs
init github.com/aws/aws-sdk-go/service/s3 @40 ms, 0.057 ms clock, 51584 bytes, 346 allocs
init github.com/aws/aws-sdk-go/service/s3/s3manager @40 ms, 0.004 ms clock, 48 bytes, 2 allocs
$ cat time.txt | awk '$5 > 0 {print $5}' | paste -sd+ | bc
1.149
$ cat time.txt | awk '$5 > 0 {print $8}' | paste -sd+ | bc
921448
$ cat time.txt | awk '$5 > 0 {print $10}' | paste -sd+ | bc
5866
The last 3 commands are summing all packages: 1.149 ms clock, 921448 bytes, and 5866 allocs.
Describe the solution you'd like
Reduce package init costs.
Additional context
I'd recommend using sync.Once to defer creating awsPartition and friends. The Go team uses this technique to defer building a large HTML entities map until it's needed.