gitian: Use custom MacOS code signing tool #20880
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Concept ACK |
Member
|
Concept ACK, nice! |
Contributor
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
Contributor
|
Impressive work! Well done @achow101. |
Member
|
Concept ACK -- nice work! |
Contributor
Contributor
|
🕵️ @sipa @practicalswift have been requested to review this pull request as specified in the REVIEWERS file. |
Member
|
Code review ACK 2c40327 Also cursory code review ACK on the signer tool achow101/signapple@c7e73aa—I can't vouch it does exactly the same as Apple's tool, but it definitely follows a similar structure as their code, and modifies the same parts of the binary. It most notably doesn't change any code so it won't introduce a back door itself.
+24 −67 |
Contributor
Gitian builds
|
Member
|
Does this need backport? |
sidhujag
pushed a commit
to syscoin/syscoin
that referenced
this pull request
Jan 20, 2021
achow101
added a commit
to achow101/bitcoin
that referenced
this pull request
Jun 8, 2021
Github-Pull: bitcoin#20880 Rebased-From: 42bb1ea
achow101
added a commit
to achow101/bitcoin
that referenced
this pull request
Jun 8, 2021
Github-Pull: bitcoin#20880 Rebased-From: 95b06d2
achow101
added a commit
to achow101/bitcoin
that referenced
this pull request
Jun 8, 2021
Github-Pull: bitcoin#20880 Rebased-From: f55eed2
achow101
added a commit
to achow101/bitcoin
that referenced
this pull request
Jun 8, 2021
Github-Pull: bitcoin#20880 Rebased-From: 2c40327
achow101
added a commit
to achow101/bitcoin
that referenced
this pull request
Jun 8, 2021
Github-Pull: bitcoin#20880 Rebased-From: 42bb1ea
achow101
added a commit
to achow101/bitcoin
that referenced
this pull request
Jun 8, 2021
Github-Pull: bitcoin#20880 Rebased-From: 95b06d2
achow101
added a commit
to achow101/bitcoin
that referenced
this pull request
Jun 8, 2021
Github-Pull: bitcoin#20880 Rebased-From: f55eed2
achow101
added a commit
to achow101/bitcoin
that referenced
this pull request
Jun 8, 2021
Github-Pull: bitcoin#20880 Rebased-From: 2c40327
This was referenced Jun 8, 2021
maflcko
pushed a commit
that referenced
this pull request
Jun 19, 2021
0fe60a8 Use latest signapple commit (Andrew Chow) 5313d6a gitian: Remove codesign_allocate and pagestuff from MacOS build (Andrew Chow) 27d691b gitian: use signapple to create the MacOS code signature (Andrew Chow) 2f33e33 gitian: use signapple to apply the MacOS code signature (Andrew Chow) 65ce833 gitian: install signapple in gitian-osx-signer.yml (Andrew Chow) Pull request description: Backport of #20880 and #22190 ACKs for top commit: MarcoFalke: cherry-pick-only ACK 0fe60a8 🍀 Tree-SHA512: e864048fab02a1857161602dd53abba552ca3f859c133a47a5e62c28d3e4de9cd099bce86123a1b5892042b09f51cc1ddd2ed1b0c71bfba162710eaee3f5bf91
maflcko
pushed a commit
that referenced
this pull request
Jun 19, 2021
890397c Use latest signapple commit (Andrew Chow) a17041e gitian: Remove codesign_allocate and pagestuff from MacOS build (Andrew Chow) 9c7c0e6 gitian: use signapple to create the MacOS code signature (Andrew Chow) f834485 gitian: use signapple to apply the MacOS code signature (Andrew Chow) a2650f6 gitian: install signapple in gitian-osx-signer.yml (Andrew Chow) Pull request description: Backport of #20880 and #22190 ACKs for top commit: MarcoFalke: cherry-pick-only ACK 890397c 💢 Tree-SHA512: 7467f664f1673152118c19f6cc31dabf05a34f37d7ea0e687ff5f740f9dfa5aab57a4b90d24c0cbb919830e6ab2dd131af147789576fcd9d3d444e7d61bd14ea
hebasto
reviewed
Aug 16, 2021
| UNSIGNED=bitcoin-osx-unsigned.tar.gz | ||
| # Install signapple | ||
| cd signapple | ||
| python3 -m pip install -U pip setuptools |
Member
There was a problem hiding this comment.
When using LXC (LXC 4.0.6 on Ubuntu Focal) this line fails with the following error:
...
+ python3 -m pip install -U pip setuptools
Collecting pip
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f82adc3a860>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/pip/
Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f82adc3a978>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/pip/
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f82adc3aa58>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/pip/
Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f82adc3ab38>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/pip/
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f82adc3abe0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/pip/
Could not find a version that satisfies the requirement pip (from versions: )
No matching distribution found for pip
...
UdjinM6
pushed a commit
to UdjinM6/dash
that referenced
this pull request
Aug 17, 2022
2c40327 gitian: Remove codesign_allocate and pagestuff from MacOS build (Andrew Chow) f55eed2 gitian: use signapple to create the MacOS code signature (Andrew Chow) 95b06d2 gitian: use signapple to apply the MacOS code signature (Andrew Chow) 42bb1ea gitian: install signapple in gitian-osx-signer.yml (Andrew Chow) Pull request description: The MacOS code signing issues that were encountered during the 0.21.0 release cycle have shown that it is necessary for us to use a code signing tool for which the source code is available and modifiable by us. Given that there appears to not be such a tool available, I have written such a tool, [signapple](https://github.com/achow101/signapple), that we can use. This tool is able to create a valid MacOS code signature, detach it in a way that we were doing previously, and attach it to the unsigned binary. This tool can also verify that the signature is correct. This PR implements the usage of that tool in the gitian build for the code signed MacOS binary. The code signer will use this tool to create the detached signature. Gitian builders will use this tool to apply the detached signature. The `gitian-osx-signer.yml` descriptor has been modified to install this tool so that the `detached-sig-apply.sh` script can use it. Additionally, the `codesign_allocate` and `pagestuff` tools are no longer necessary so they are no longer added to the tarball used in code signing. Lastly, both the `detached-sig-create.sh` and `detached-sig-apply.sh` scripts are made to be significantly less complex and to not do unexpected things such as unpacking an already unpacked tarball. The detached code signature that signapple creates is almost identical to that which we were previously creating. The only difference is that the cpu architecture name is included in the extension (e.g. we have `bitcoin-qt.x86_64sign` instead of `bitcoin-qt.sign`). This was done in order to support signing universal binaries which we may want to do in the future. However signapple can still apply existing code signatures as it will accept the `.sign` extension. If it is desired, it can be modified to produce signatures with just the `.sign` extension. However I do not think it is necessary to maintain compatibility with the old process. ACKs for top commit: laanwj: Code review ACK 2c40327 Tree-SHA512: 2a0e01e9133f8859b9de26e7e8fe1d2610d2cbdee2845e6008b12c083c7e3622cbb2d9b83c50a269e2c3074ab95914a8225d3cd4108017f58b77a62bf10951e0
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
8 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The MacOS code signing issues that were encountered during the 0.21.0 release cycle have shown that it is necessary for us to use a code signing tool for which the source code is available and modifiable by us. Given that there appears to not be such a tool available, I have written such a tool, signapple, that we can use. This tool is able to create a valid MacOS code signature, detach it in a way that we were doing previously, and attach it to the unsigned binary. This tool can also verify that the signature is correct.
This PR implements the usage of that tool in the gitian build for the code signed MacOS binary. The code signer will use this tool to create the detached signature. Gitian builders will use this tool to apply the detached signature. The
gitian-osx-signer.ymldescriptor has been modified to install this tool so that thedetached-sig-apply.shscript can use it. Additionally, thecodesign_allocateandpagestufftools are no longer necessary so they are no longer added to the tarball used in code signing. Lastly, both thedetached-sig-create.shanddetached-sig-apply.shscripts are made to be significantly less complex and to not do unexpected things such as unpacking an already unpacked tarball.The detached code signature that signapple creates is almost identical to that which we were previously creating. The only difference is that the cpu architecture name is included in the extension (e.g. we have
bitcoin-qt.x86_64signinstead ofbitcoin-qt.sign). This was done in order to support signing universal binaries which we may want to do in the future. However signapple can still apply existing code signatures as it will accept the.signextension. If it is desired, it can be modified to produce signatures with just the.signextension. However I do not think it is necessary to maintain compatibility with the old process.