[22.x] guix: ignore additional failing certvalidator test #24215

fanquake · 2022-01-31T13:10:33Z

Backports 8588591 from #24057 so that from-scratch Guix builds for the Darwin host aren't broken due to a (very recently) expired certificate causing one of the python-certvalidator tests to fail. Kept separate from #23276 because that hasn't gotten review attention, and I don't think we should leave 22.x Darwin Guix builds broken for any longer than we have to.

Fixes #24110.

======================================================================
ERROR: test_revocation_mode_soft (tests.test_validate.ValidateTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/guix-build-python-certvalidator-0.1-1.e5bdb4b.drv-0/source/tests/test_validate.py", line 85, in test_revocation_mode_soft
    validate_path(context, path)
  File "/tmp/guix-build-python-certvalidator-0.1-1.e5bdb4b.drv-0/source/tests/../certvalidator/validate.py", line 50, in validate_path
    return _validate_path(validation_context, path)
  File "/tmp/guix-build-python-certvalidator-0.1-1.e5bdb4b.drv-0/source/tests/../certvalidator/validate.py", line 358, in _validate_path
    raise PathValidationError(pretty_message(
certvalidator.errors.PathValidationError: The path could not be validated because the end-entity certificate expired 2022-01-14 12:00:00Z

Guix Build:

bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
359755bffecc64b4c005c5cdee3824190f6b1759dbc6c20034476dcc06413959  guix-build-b7ecef1ddf0c/output/dist-archive/bitcoin-b7ecef1ddf0c.tar.gz
0c6700270ec75991d70a97cad77e22cc00553f812edb56c1bac5ef6421f963e1  guix-build-b7ecef1ddf0c/output/x86_64-apple-darwin/SHA256SUMS.part
87d4637a87959a304422550edf87feda3953d7305894154a6a2d413cc0dd2034  guix-build-b7ecef1ddf0c/output/x86_64-apple-darwin/bitcoin-b7ecef1ddf0c-osx-unsigned.dmg
9cabae32689bd5f93e7faaaf341827f1c4069a63ab6f74276564e47819343b6c  guix-build-b7ecef1ddf0c/output/x86_64-apple-darwin/bitcoin-b7ecef1ddf0c-osx-unsigned.tar.gz
bb5fb113bc022a305e49783d0ba48be90aca61e4a942beeb45206dbc5b91ca6e  guix-build-b7ecef1ddf0c/output/x86_64-apple-darwin/bitcoin-b7ecef1ddf0c-x86_64-apple-darwin.tar.gz

====================================================================== ERROR: test_revocation_mode_soft (tests.test_validate.ValidateTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/tmp/guix-build-python-certvalidator-0.1-1.e5bdb4b.drv-0/source/tests/test_validate.py", line 85, in test_revocation_mode_soft validate_path(context, path) File "/tmp/guix-build-python-certvalidator-0.1-1.e5bdb4b.drv-0/source/tests/../certvalidator/validate.py", line 50, in validate_path return _validate_path(validation_context, path) File "/tmp/guix-build-python-certvalidator-0.1-1.e5bdb4b.drv-0/source/tests/../certvalidator/validate.py", line 358, in _validate_path raise PathValidationError(pretty_message( certvalidator.errors.PathValidationError: The path could not be validated because the end-entity certificate expired 2022-01-14 12:00:00Z Github-Pull: bitcoin#24057 Rebased-From: 8588591

Sjors · 2022-01-31T18:26:32Z

I did a a Guix clean followed by:

env HOSTS='x86_64-apple-darwin' ./contrib/guix/guix-build

This succeeds, but code-sign doesn't:

env HOSTS='x86_64-apple-darwin' ./contrib/guix/guix-codesign 
Checking that we can connect to the guix-daemon...

Hint: If this hangs, you may want to try turning your guix-daemon off and on
      again.

INFO: Codesigning b7ecef1ddf0c for platform triple x86_64-apple-darwin:
      ...using reference timestamp: 1642652187
      ...from worktree directory: '/home/guix/bitcoin'
          ...bind-mounted in container to: '/bitcoin'
      ...in build directory: '/home/guix/bitcoin/guix-build-b7ecef1ddf0c/distsrc-b7ecef1ddf0c-x86_64-apple-darwin-codesigned'
          ...bind-mounted in container to: '/distsrc-base/distsrc-b7ecef1ddf0c-x86_64-apple-darwin-codesigned'
      ...outputting in: '/home/guix/bitcoin/guix-build-b7ecef1ddf0c/output/x86_64-apple-darwin-codesigned'
          ...bind-mounted in container to: '/outdir-base/x86_64-apple-darwin-codesigned'
      ...using detached signatures in: '/home/guix/bitcoin-detached-sigs'
          ...bind-mounted in container to: '/detached-sigs'
Required environment variables as seen inside the container:
    UNSIGNED_TARBALL: /outdir-base/x86_64-apple-darwin/bitcoin-b7ecef1ddf0c-osx-unsigned.tar.gz
    DETACHED_SIGS_REPO: /detached-sigs
    DIST_ARCHIVE_BASE: /outdir-base/dist-archive
    DISTNAME: bitcoin-b7ecef1ddf0c
    HOST: x86_64-apple-darwin
    SOURCE_DATE_EPOCH: 1642652187
    DISTSRC: /distsrc-base/distsrc-b7ecef1ddf0c-x86_64-apple-darwin-codesigned
    OUTDIR: /outdir-base/x86_64-apple-darwin-codesigned
/gnu/store/q3y2bpd61bvb7d0g9ils1zi6pax5yvb1-python-elfesteem-0.1-1.87bbd79/lib/python3.8/site-packages/elfesteem/cstruct.py:412: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if name is not '' and not name in table: table[name] = {}
/gnu/store/q3y2bpd61bvb7d0g9ils1zi6pax5yvb1-python-elfesteem-0.1-1.87bbd79/lib/python3.8/site-packages/elfesteem/cstruct.py:415: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if name is not '':
Code signature applied
Traceback (most recent call last):
  File "/gnu/store/vsi3743mm41cfdv0zmap6p2qfsqylcmx-python-signapple-0.1-1.b084cbb/bin/.signapple-real", line 11, in <module>
    load_entry_point('signapple==0.1.0', 'console_scripts', 'signapple')()
  File "/gnu/store/vsi3743mm41cfdv0zmap6p2qfsqylcmx-python-signapple-0.1-1.b084cbb/lib/python3.8/site-packages/signapple/__init__.py", line 112, in main
    args.func(args)
  File "/gnu/store/vsi3743mm41cfdv0zmap6p2qfsqylcmx-python-signapple-0.1-1.b084cbb/lib/python3.8/site-packages/signapple/__init__.py", line 36, in apply
    verify(args)
  File "/gnu/store/vsi3743mm41cfdv0zmap6p2qfsqylcmx-python-signapple-0.1-1.b084cbb/lib/python3.8/site-packages/signapple/__init__.py", line 10, in verify
    verify_mach_o_signature(args.filename)
  File "/gnu/store/vsi3743mm41cfdv0zmap6p2qfsqylcmx-python-signapple-0.1-1.b084cbb/lib/python3.8/site-packages/signapple/verify.py", line 227, in verify_mach_o_signature
    _verify_single(filepath, header)
  File "/gnu/store/vsi3743mm41cfdv0zmap6p2qfsqylcmx-python-signapple-0.1-1.b084cbb/lib/python3.8/site-packages/signapple/verify.py", line 161, in _verify_single
    _validate_code_hashes(f, sig_superblob.code_dir_blob)
  File "/gnu/store/vsi3743mm41cfdv0zmap6p2qfsqylcmx-python-signapple-0.1-1.b084cbb/lib/python3.8/site-packages/signapple/verify.py", line 54, in _validate_code_hashes
    raise Exception(
Exception: Code slot hash mismatch. Expected 8cdb98ee7dbd9c1a5b021603c4b0ef933a31717f57884bc3a72536257e4d53dd, Calculated 95bcde7c12f864fcf3de59c87142b9fde1b728328a48cd923023381b730b4eab

Some hashes:

8f70852feb39078e02182563517d17bdfc4a12904cf1bdabbae95594d9a1e473  guix-build-b7ecef1ddf0c/output/dist-archive/bitcoin-b7ecef1ddf0c-codesignatures-22.0.tar.gz
359755bffecc64b4c005c5cdee3824190f6b1759dbc6c20034476dcc06413959  guix-build-b7ecef1ddf0c/output/dist-archive/bitcoin-b7ecef1ddf0c.tar.gz
0c6700270ec75991d70a97cad77e22cc00553f812edb56c1bac5ef6421f963e1  guix-build-b7ecef1ddf0c/output/x86_64-apple-darwin/SHA256SUMS.part
87d4637a87959a304422550edf87feda3953d7305894154a6a2d413cc0dd2034  guix-build-b7ecef1ddf0c/output/x86_64-apple-darwin/bitcoin-b7ecef1ddf0c-osx-unsigned.dmg
9cabae32689bd5f93e7faaaf341827f1c4069a63ab6f74276564e47819343b6c  guix-build-b7ecef1ddf0c/output/x86_64-apple-darwin/bitcoin-b7ecef1ddf0c-osx-unsigned.tar.gz
bb5fb113bc022a305e49783d0ba48be90aca61e4a942beeb45206dbc5b91ca6e  guix-build-b7ecef1ddf0c/output/x86_64-apple-darwin/bitcoin-b7ecef1ddf0c-x86_64-apple-darwin.tar.gz

Sjors · 2022-01-31T18:27:25Z

cc @dongcarl

achow101 · 2022-01-31T18:44:35Z

I've also updated the certvalidator fork we use to have passing tests.

hebasto · 2022-01-31T18:48:23Z

@Sjors

This succeeds, but code-sign doesn't:

Maybe #21851 (comment) and #22546 ?

Sjors · 2022-01-31T20:22:42Z

@hebasto I'm running Guix on an Ubuntu machine. Are you saying I should copy it over to my Mac and code-sign it? That seems odd. Or is that a workaround because we don't have detached signatures for this commit?

@achow101 certvalidator?

dongcarl · 2022-01-31T22:54:39Z

Wondering if we should just switch to achow's achow101/certvalidator@e5bdb4b instead?

fanquake · 2022-01-31T23:00:22Z

Wondering if we should just switch to achow's achow101/certvalidator@e5bdb4b instead?

Isn't that the version we are already using?

achow101 · 2022-01-31T23:07:38Z

Wondering if we should just switch to achow's achow101/certvalidator@e5bdb4b instead?

I think you mean achow101/certvalidator@a145bf2

@hebasto I'm running Guix on an Ubuntu machine. Are you saying I should copy it over to my Mac and code-sign it? That seems odd. Or is that a workaround because we don't have detached signatures for this commit?

The error you get there is because the detached sig it is using is for a different release. signapple does not know that the detached sig is not for the binary you have built.

@achow101 certvalidator?

The tool that this patch is fixing.

fanquake · 2022-02-03T06:44:37Z

I think we could switch to the newer certvalidator branch in master, and then re-enable tests. However for 22.x I'd rather fix the build by backporting this change. I'll add some Guix hashes to the PR description shortly.

Sjors · 2022-02-03T10:15:30Z

My hashes match those in the PR description, but those don't include the signed DMG. I guess I'm confused about what certificate related problem this PR is solving.

fanquake · 2022-02-04T01:58:41Z

I guess I'm confused about what certificate related problem this PR is solving.

If you Guix build the current 22.x branch, from scratch (so that python-certvalidator is built instead of a cached version being used), it will fail. This PR is making it not fail. The python-certvalidator package has a test that has recently started failing, due to a certificate used in it's test suite expiring. All this change does is ignore the failing test, similar to how we already ignore some of its tests, so that the Guix build will work.

but those don't include the signed DMG.

They don't need to. You don't need to run the code-sign step at all to verify this fixing what it's supposed to be fixing.

Sjors · 2022-02-04T09:09:46Z

What is python-certvalidator used for roughly?

fanquake · 2022-02-04T09:16:53Z

What is python-certvalidator used for roughly?

https://github.com/wbond/certvalidator: "A Python library for validating X.509 certificates or paths. ". We use a fork maintained by achow. It's a dependency of sign-apple.

Sjors · 2022-02-04T10:36:48Z

from scratch

I did a guix clean and then built the commit before PR. Which works fine. I'll try a more thorough nuke of Guix files...

maflcko · 2022-02-09T09:25:08Z

Concept ACK b7ecef1

Seems good to make it possible to compile the branch again

fanquake · 2022-02-09T09:38:12Z

I did a guix clean and then built the commit before PR. Which works fine.

You need to Guix build such that the python-certvalidator package is actually built, and the tests run.

I am going to merge this now to un-break from-scratch builds.

fanquake added the Backport label Jan 31, 2022

fanquake linked an issue Jan 31, 2022 that may be closed by this pull request

22.x: darwin guix build currently failing #24110

Closed

fanquake merged commit a06915d into bitcoin:22.x Feb 9, 2022

fanquake deleted the backport_guix_darwin_22_x branch February 9, 2022 09:38

This was referenced Feb 9, 2022

[22.x] build: Bump depends packages (zmq, libXau) #24256

Merged

22.x: darwin guix build currently failing #24110

Closed

justanwar mentioned this pull request Mar 6, 2022

Guix for Darwin failing firoorg/firo#1150

Closed

bitcoin locked and limited conversation to collaborators Feb 9, 2023

[22.x] guix: ignore additional failing certvalidator test #24215

[22.x] guix: ignore additional failing certvalidator test #24215

Uh oh!

Conversation

fanquake commented Jan 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Sjors commented Jan 31, 2022

Uh oh!

Sjors commented Jan 31, 2022

Uh oh!

achow101 commented Jan 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hebasto commented Jan 31, 2022

Uh oh!

Sjors commented Jan 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dongcarl commented Jan 31, 2022

Uh oh!

fanquake commented Jan 31, 2022

Uh oh!

achow101 commented Jan 31, 2022

Uh oh!

fanquake commented Feb 3, 2022

Uh oh!

Sjors commented Feb 3, 2022

Uh oh!

fanquake commented Feb 4, 2022

Uh oh!

Sjors commented Feb 4, 2022

Uh oh!

fanquake commented Feb 4, 2022

Uh oh!

Sjors commented Feb 4, 2022

Uh oh!

maflcko commented Feb 9, 2022

Uh oh!

fanquake commented Feb 9, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

fanquake commented Jan 31, 2022 •

edited

Loading

achow101 commented Jan 31, 2022 •

edited

Loading

Sjors commented Jan 31, 2022 •

edited

Loading