Skip to content

Conversation

@dongcarl
Copy link
Contributor

When building nsis, if VERSION is not specified, it defaults to
cvs_version which is non-deterministic as it includes the current date.

This patches nsis to default to SOURCE_DATE_EPOCH if it exists so that
nsis is reproducible.

Upstream change: https://github.com/kichik/nsis/pull/13

Sidenote: also a good demonstration of how Guix allows us to flexibly patch our tools!

Note to reviewers: if you want to compare hashes, please build after Jan 16th 2021 without my substitute server enabled!

@DrahtBot
Copy link
Contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@fanquake
Copy link
Member

Nice. Concept ACK.

Copy link
Contributor

@practicalswift practicalswift left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

@dongcarl dongcarl force-pushed the 2021-01-nsis-sde-reproducibility branch from 954341c to ec06efc Compare January 15, 2021 18:17
@dongcarl
Copy link
Contributor Author

Pushed 954341cefc355d68226ea5cde7c9bd8024e6740c -> ec06efc69684aa08a7f5c7be7d8a683b24ef4f8c

When building nsis, if VERSION is not specified, it defaults to
cvs_version which is non-deterministic as it includes the current date.

This patches nsis to default to SOURCE_DATE_EPOCH if it exists so that
nsis is reproducible.

Upstream change: NSIS-Dev/nsis#13
@dongcarl dongcarl force-pushed the 2021-01-nsis-sde-reproducibility branch from ec06efc to 1fca981 Compare January 17, 2021 23:44
@fanquake
Copy link
Member

Thanks for rebasing.

please build after Jan 16th 2021 without my substitute server enabled!

# guix describe
Generation 5	Jan 18 2021 00:11:42	(current)
  guix 1b2c32d
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 1b2c32dc919ec331bbd3219c586f46fe66e06346

# git rev-parse HEAD
1fca9811e1331ac5dae8188f6178cc37da4929a7

# find output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
9300fae60f64636c7224376d388a5ec582c60541b844b9d62072bc91bdb731b4  output/bitcoin-1fca9811e133-win-unsigned.tar.gz
1d0d9c713f9ffa2437d5ce29d4bc61abc6866cbe17e0935b0794873e509700cf  output/bitcoin-1fca9811e133-win64-debug.zip
6dddccd0af5f820ccaa57a1790c5a589ea43864d542e22bb0d770e5cd562d75d  output/bitcoin-1fca9811e133-win64-setup-unsigned.exe
9b52e4fddedf229ce29930baaf6e139cf385cc4a38c0ca50122df4af020a96bd  output/bitcoin-1fca9811e133-win64.zip
207f33df4ceb953da33a334604e5dec5f048ee53e69cfad8af1afd52ed0cc98f  output/src/bitcoin-1fca9811e133.tar.gz

@DrahtBot
Copy link
Contributor

Guix builds

File commit 7acda55
(master)
commit c0eeb13161f4f4f2b111e039b3c93474b790bea8
(master and this pull)
*-aarch64-linux-gnu-debug.tar.gz 3cb5d38bef3ce7ff... 0bf445d3d4a25f57...
*-aarch64-linux-gnu.tar.gz 921f0ea1765e9d3a... 6313d6e9a20150aa...
*-arm-linux-gnueabihf-debug.tar.gz 292b29d4055e60a7... 6ca3046fcbd26a8c...
*-arm-linux-gnueabihf.tar.gz 0d62afadd03f613a... 264deec0506390b0...
*-x86_64-linux-gnu-debug.tar.gz a984b708b2687740... 12748774b0fe8e5d...
*-x86_64-linux-gnu.tar.gz d008154dc9bdb68e... f94bf7d1d79e793b...
*.tar.gz dff681d0d36b7bfa... b4a5001eb1ab05a7...
guix_build.log 70436d6db6777cd5... 3d12f77c61f929ca...
guix_build.log.diff dc043ea0a1e7f0aa...

@dongcarl
Copy link
Contributor Author

dongcarl commented Jan 19, 2021

Thanks fanquake!

Here are my hashes, they appear to be matching!

# find output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
9300fae60f64636c7224376d388a5ec582c60541b844b9d62072bc91bdb731b4  output/bitcoin-1fca9811e133-win-unsigned.tar.gz
1d0d9c713f9ffa2437d5ce29d4bc61abc6866cbe17e0935b0794873e509700cf  output/bitcoin-1fca9811e133-win64-debug.zip
6dddccd0af5f820ccaa57a1790c5a589ea43864d542e22bb0d770e5cd562d75d  output/bitcoin-1fca9811e133-win64-setup-unsigned.exe
9b52e4fddedf229ce29930baaf6e139cf385cc4a38c0ca50122df4af020a96bd  output/bitcoin-1fca9811e133-win64.zip
207f33df4ceb953da33a334604e5dec5f048ee53e69cfad8af1afd52ed0cc98f  output/src/bitcoin-1fca9811e133.tar.gz

Would you care to supply your stat /gnu/store/n4jdwhwip7wir0gzl92smvsdpwwh2bz9-nsis-x86_64-3.05? Just so we make sure we built nsis on differing dates so we'd potentially trigger the non-determinism.

# stat /gnu/store/n4jdwhwip7wir0gzl92smvsdpwwh2bz9-nsis-x86_64-3.05
  File: /gnu/store/n4jdwhwip7wir0gzl92smvsdpwwh2bz9-nsis-x86_64-3.05
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: 901h/2305d      Inode: 23744146    Links: 5
Access: (0555/dr-xr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-01-14 18:52:51.143425311 -0500
Modify: 1969-12-31 19:00:01.000000000 -0500
Change: 2021-01-14 18:52:51.136758471 -0500
 Birth: 2021-01-14 18:52:37.683076782 -0500

@DrahtBot
Copy link
Contributor

Gitian builds

File commit 43f3ada
(master)
commit 39d896e2b0cfc7d8821dd94e9f5020cf6adad994
(master and this pull)
*-aarch64-linux-gnu-debug.tar.gz 559dd36d19c18766... c0b4ebb543b5cd2d...
*-aarch64-linux-gnu.tar.gz 2327131dc0b85b16... 12b69b194c0a1d64...
*-arm-linux-gnueabihf-debug.tar.gz a344291979c13f62... 92e32512ed89e936...
*-arm-linux-gnueabihf.tar.gz 3fa15aad12aa0b4d... d5412a5c78b4df9c...
*-osx-unsigned.dmg daa2cf679fb82613... 4cd56da99cd06ccf...
*-osx64.tar.gz 72f5f659ccc515d2... 68a8e5fdf5b5971a...
*-riscv64-linux-gnu-debug.tar.gz 7203c3f42c26dd51... 978740cb5d86a043...
*-riscv64-linux-gnu.tar.gz 42b2f7b5e3adba32... 39d9254fa1848c5c...
*-win64-debug.zip c9d2083970c06a74... 4551a2d0665e211e...
*-win64-setup-unsigned.exe 0d56ad727cd6854c... fd2f4bcc5db1f6cc...
*-win64.zip 04a512476cc79e35... b2b2c7eb75485bc2...
*-x86_64-linux-gnu-debug.tar.gz 86cb4df751bb46dc... cb5959561bb24f7a...
*-x86_64-linux-gnu.tar.gz 1d287b0c59bfb059... a11eb676ea7f3baf...
*.tar.gz a1ee1dac4c106342... 8de12e3db14a73b9...
bitcoin-core-linux-22-res.yml c4a09a6900e447e4... 20a6236a1282092e...
bitcoin-core-osx-22-res.yml 55b7f77fae35f92a... 8767999e8488a7ee...
bitcoin-core-win-22-res.yml 77d6843038373494... 9d19d00dcad01b66...
linux-build.log 35bd4b0a51da2e8f... fa9aac00c63cd3bc...
osx-build.log 5aff87f4cdceb992... fb386df075f9af83...
win-build.log 93ca97a3a1aa62e6... 822fdcb8cf58acc6...
bitcoin-core-linux-22-res.yml.diff cab486abdd8d1c97...
bitcoin-core-osx-22-res.yml.diff 67ff86a91635d5f9...
bitcoin-core-win-22-res.yml.diff 8cbbcfb2d845de76...
linux-build.log.diff 134948ae3da35227...
osx-build.log.diff 62d658f9a523469d...
win-build.log.diff 4e789d017641ee00...

@fanquake
Copy link
Member

Would you care to supply your stat /gnu/store/n4jdwhwip7wir0gzl92smvsdpwwh2bz9-nsis-x86_64-3.05?

# stat /gnu/store/n4jdwhwip7wir0gzl92smvsdpwwh2bz9-nsis-x86_64-3.05/
  File: /gnu/store/n4jdwhwip7wir0gzl92smvsdpwwh2bz9-nsis-x86_64-3.05/
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
Device: 35h/53d	Inode: 4096562     Links: 5
Access: (0555/dr-xr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-01-17 06:37:19.000000000
Modify: 1970-01-01 00:00:01.000000000
Change: 2021-01-17 06:37:19.000000000

@dongcarl
Copy link
Contributor Author

Lovely! Looks like we fixed the reproducibility problem! This should be good to merge now

Copy link
Member

@fanquake fanquake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 1fca981

@fanquake fanquake merged commit 977bec1 into bitcoin:master Jan 19, 2021
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Jan 20, 2021
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Aug 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

guix: Potential NSIS non-reproducibility

5 participants