depends: make osx output deterministic #9891
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
theuni
force-pushed
the
fix-osx-link-determinism
branch
from
4cbbe5b to
9e4d842
Compare
ld64 is threaded, and uses a worker for each CPU to parse input files. But there's a bug in the parser causing dependencies to be calculated differently based on which files have already been parsed. As a result, builders with more CPUs are more likely to see non-determinism. This looks to have been fixed in a newer version of ld64, so just disable threading for now. There's no noticible slowdown.
Contributor
|
@theuni Brilliant investigation! Thank you! For more info, the unpacked code is available at https://github.com/froydnj/ld64/blob/master/src/ld/InputFiles.cpp#L716 utACK 9e4d842 |
Contributor
Member
|
Wow, this is a really sneaky bug. Thanks for catching it. utACK 9e4d842 |
laanwj
added a commit
that referenced
this pull request
Mar 1, 2017
9e4d842 depends: make osx output deterministic (Cory Fields) Tree-SHA512: 997a671832a28e600bc3c9cd1340c6c1fa08f050d7cf676a5d51b38f08161ac418621520459a9a53dd87102c13fb69cb2e58ec3cc00f738e204eb787d6f0cad6
laanwj
pushed a commit
that referenced
this pull request
Mar 1, 2017
ld64 is threaded, and uses a worker for each CPU to parse input files. But there's a bug in the parser causing dependencies to be calculated differently based on which files have already been parsed. As a result, builders with more CPUs are more likely to see non-determinism. This looks to have been fixed in a newer version of ld64, so just disable threading for now. There's no noticible slowdown. Github-Pull: #9891 Rebased-From: 9e4d842
Member
|
Post merge ACK. Tested by running 50 gitian builds with this patch applied and the hashes of each build match. |
codablock
pushed a commit
to codablock/dash
that referenced
this pull request
Jan 26, 2018
9e4d842 depends: make osx output deterministic (Cory Fields) Tree-SHA512: 997a671832a28e600bc3c9cd1340c6c1fa08f050d7cf676a5d51b38f08161ac418621520459a9a53dd87102c13fb69cb2e58ec3cc00f738e204eb787d6f0cad6
CryptoCentric
pushed a commit
to absolute-community/absolute
that referenced
this pull request
Feb 27, 2019
9e4d842 depends: make osx output deterministic (Cory Fields) Tree-SHA512: 997a671832a28e600bc3c9cd1340c6c1fa08f050d7cf676a5d51b38f08161ac418621520459a9a53dd87102c13fb69cb2e58ec3cc00f738e204eb787d6f0cad6
zkbot
added a commit
to zcash/zcash
that referenced
this pull request
Dec 11, 2019
Enable macOS cross-compilation Includes code cherry-picked from the following upstream PRs: - bitcoin/bitcoin#7809 - The `native_cctools` fix. - bitcoin/bitcoin#8210 - The macOS toolchain bump. - bitcoin/bitcoin#9891 - bitcoin/bitcoin#15581 - The `tar` change. - bitcoin/bitcoin#16049 - The `native_cctools` change. Build instructions: - Fetch `MacOSX10.11.sdk` from e.g. https://github.com/phracker/MacOSX-SDKs/releases - Extract it into `depends/SDKs` (creating that folder first) - `HOST=x86_64-apple-darwin11 ./zcutil/build.sh`
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ld64 is threaded, and uses a worker for each CPU to parse input files. But
there's a bug in the parser causing dependencies to be calculated differently
based on which files have already been parsed.
As a result, builders with more CPUs are more likely to see non-determinism.
This looks to have been fixed in a newer version of ld64, so just disable
threading for now. There's no noticible slowdown.