Skip to content

Updates for go1.21 rand changes#28

Merged
mroth merged 4 commits intomainfrom
go1.21
Aug 13, 2023
Merged

Updates for go1.21 rand changes#28
mroth merged 4 commits intomainfrom
go1.21

Conversation

@mroth
Copy link
Copy Markdown
Owner

@mroth mroth commented Aug 7, 2023 •
edited
Loading

Work in progress

Fixes #26

@mroth
bench: separate benchmarks for Pick and PickSource
0ee7b80 
As discussed in #26, the changes in go1.21 will remove rand contention
when the (now deprecated) rand.Seed() is not called.

In order to check if this brings performance parity, start to benchmark
both Pick and PickSource functions in parallel, and no longer rand.Seed
in test init.
@mroth
Copy link
Copy Markdown
Owner Author

mroth commented Aug 7, 2023 •
edited
Loading

Benchmarks using above commit, showing PickSource still quite necessary on go1.20

$ go version
go version go1.20.7 darwin/arm64
$ go test -bench=Parallel$ -cpu 8 -count 10 > go120.txt
$ benchstat -col=".name" go120.txt
goos: darwin
goarch: arm64
pkg: github.com/mroth/weightedrand/v2
            │ PickParallel  │          PickSourceParallel          │
            │    sec/op     │    sec/op     vs base                │
*/10-8        117.550n ± 1%   2.479n ±  8%  -97.89% (p=0.000 n=10)
*/100-8       125.250n ± 2%   4.143n ±  4%  -96.69% (p=0.000 n=10)
*/1000-8      131.000n ± 3%   5.942n ±  2%  -95.46% (p=0.000 n=10)
*/10000-8     154.100n ± 3%   8.178n ± 18%  -94.69% (p=0.000 n=10)
*/100000-8     180.90n ± 1%   11.18n ±  0%  -93.82% (p=0.000 n=10)
*/1000000-8    221.20n ± 1%   14.51n ±  0%  -93.44% (p=0.000 n=10)
geomean         151.1n        6.578n        -95.65%

On go1.21, the difference mostly negligible (some minor noise in the tests here due to running on laptop):

$ go1.21.0 version
go version go1.21.0 darwin/arm64
go1.21.0 test -bench=Parallel$ -cpu 8 -count 10 > go121.txt
$ benchstat -col=".name" go121.txt
goos: darwin
goarch: arm64
pkg: github.com/mroth/weightedrand/v2
            │ PickParallel │         PickSourceParallel         │
            │    sec/op    │   sec/op     vs base               │
*/10-8         2.620n ± 5%   2.449n ± 2%  -6.55% (p=0.002 n=10)
*/100-8        4.227n ± 1%   4.142n ± 1%  -2.01% (p=0.000 n=10)
*/1000-8       5.900n ± 0%   5.878n ± 0%  -0.36% (p=0.001 n=10)
*/10000-8      7.784n ± 0%   7.986n ± 0%  +2.60% (p=0.000 n=10)
*/100000-8     10.99n ± 0%   11.14n ± 0%  +1.36% (p=0.000 n=10)
*/1000000-8    14.43n ± 1%   14.52n ± 0%  +0.59% (p=0.013 n=10)
geomean        6.573n        6.522n       -0.77%

Direct comparison of go1.20 to go1.21:

$ benchstat go120.txt go121.txt
goos: darwin
goarch: arm64
pkg: github.com/mroth/weightedrand/v2
                             │   go120.txt    │              go121.txt              │
                             │     sec/op     │   sec/op     vs base                │
PickParallel/10-8              117.550n ±  1%   2.620n ± 5%  -97.77% (p=0.000 n=10)
PickParallel/100-8             125.250n ±  2%   4.227n ± 1%  -96.63% (p=0.000 n=10)
PickParallel/1000-8            131.000n ±  3%   5.900n ± 0%  -95.50% (p=0.000 n=10)
PickParallel/10000-8           154.100n ±  3%   7.784n ± 0%  -94.95% (p=0.000 n=10)
PickParallel/100000-8           180.90n ±  1%   10.99n ± 0%  -93.92% (p=0.000 n=10)
PickParallel/1000000-8          221.20n ±  1%   14.43n ± 1%  -93.48% (p=0.000 n=10)
PickSourceParallel/10-8          2.479n ±  8%   2.449n ± 2%        ~ (p=0.190 n=10)
PickSourceParallel/100-8         4.143n ±  4%   4.142n ± 1%        ~ (p=0.684 n=10)
PickSourceParallel/1000-8        5.942n ±  2%   5.878n ± 0%   -1.08% (p=0.000 n=10)
PickSourceParallel/10000-8       8.178n ± 18%   7.986n ± 0%   -2.34% (p=0.007 n=10)
PickSourceParallel/100000-8      11.18n ±  0%   11.14n ± 0%   -0.36% (p=0.005 n=10)
PickSourceParallel/1000000-8     14.51n ±  0%   14.52n ± 0%        ~ (p=0.806 n=10)
geomean                          31.52n         6.548n       -79.23%

@mroth
Copy link
Copy Markdown
Owner Author

mroth commented Aug 12, 2023

A search of the dependents for this module reveal that many who use PickSource are using it incorrectly, so in order to simplify the surface area going to just go ahead and mark as deprecated as of go1.21 rather than waiting another cycle for go1.22.

There is still a potential valid use case for wanting to provide a custom randomness source, but would rather get it out of the API for now, and revisit if it can be done a more elegant way with the proposed math/rand/v2 in discussion.

mroth added 2 commits August 13, 2023 11:49
@mroth
docs: update examples/compbench for go1.21
60f6145 
results on my Apple M2 Pro laptop:

$ make analysis
benchstat -filter=".name:Multiple" -col="/lib" -row="/size" -table="/concurrency" results.txt
/concurrency: single
        │    randutil     │             weightedrand             │
        │     sec/op      │   sec/op     vs base                 │
1e1          455.70n ± 0%   19.16n ± 2%   -95.80% (p=0.000 n=10)
1e2          438.20n ± 0%   32.81n ± 0%   -92.51% (p=0.000 n=10)
1e3         1179.00n ± 0%   46.00n ± 0%   -96.10% (p=0.000 n=10)
1e4         6591.00n ± 1%   60.34n ± 0%   -99.08% (p=0.000 n=10)
1e5        61188.00n ± 0%   84.83n ± 0%   -99.86% (p=0.000 n=10)
1e6        631928.5n ± 1%   112.6n ± 2%   -99.98% (p=0.000 n=10)
1e7       6402254.0n ± 1%   240.7n ± 0%  -100.00% (p=0.000 n=10)
geomean       16.84µ        63.16n        -99.62%

/concurrency: parallel
        │     randutil     │             weightedrand             │
        │      sec/op      │   sec/op     vs base                 │
1e1          589.300n ± 0%   2.026n ± 1%   -99.66% (p=0.000 n=10)
1e2          490.000n ± 0%   3.337n ± 1%   -99.32% (p=0.000 n=10)
1e3          766.500n ± 0%   4.518n ± 1%   -99.41% (p=0.000 n=10)
1e4          808.700n ± 0%   5.951n ± 2%   -99.26% (p=0.000 n=10)
1e5         6438.500n ± 1%   8.184n ± 1%   -99.87% (p=0.000 n=10)
1e6        108285.50n ± 1%   12.00n ± 1%   -99.99% (p=0.000 n=10)
1e7       2011263.00n ± 2%   28.95n ± 1%  -100.00% (p=0.000 n=10)
geomean        5.907µ        6.548n        -99.89%
@mroth
port benchstat format from examples/compbench
b306b18 
key-value format is nicer for later filtering in benchstat
@mroth mroth marked this pull request as ready for review August 13, 2023 16:36
@mroth mroth merged commit 38f771d into main Aug 13, 2023
@mroth mroth deleted the go1.21 branch August 13, 2023 16:37
@mroth mroth mentioned this pull request Aug 13, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

update docs regarding PickSource for go1.21

1 participant

@mroth