Automate handling of Distribution certificates and profiles by bjornoleh · Pull Request #453 · nightscout/Trio

bjornoleh · 2025-01-15T14:24:29Z

Summary:

This pull request introduces automation for handling Distribution certificates and provisioning profiles within GitHub Actions workflows. It simplifies certificate management by detecting invalid or expired certificates and automatically creating or renewing them when needed. Additionally, it ensures a seamless process for annual certificate renewals by integrating these steps into the build process.

Key Changes:

Workflow Updates:
- Added check_certs job to .github/workflows/build_trio.yml for verifying the validity of Distribution certificates.
- Introduced a conditional nuke_certs job to recreate certificates if they are expired or missing.
- Enhanced .github/workflows/create_certs.yml to handle certificate validation, renewal, and recreation automatically.
Fastfile Enhancements:
- Updated Fastfile to include a new check_and_renew_certificates lane for checking and renewing certificates programmatically.
- Added detailed logging and output management for certificate renewal processes.
Validation Workflow Improvements:
- Modified .github/workflows/validate_secrets.yml to continue on errors in fastlane validate_secrets, avoiding workflow interruptions for invalid certificates.
- Improved error messages and annotations for better clarity and guidance during validation.
Documentation Updates:
- Updated testflight.md with new instructions for certificate handling:
- Explained the automation of annual certificate renewal.
- Simplified guidance for users managing multiple GitHub usernames or organizations.
- Removed outdated manual certificate creation steps.

Benefits:

Automation: Eliminates manual intervention for certificate renewal and reduces build disruptions caused by expired certificates.
Reliability: Ensures workflows continue to operate even when certificates are invalid, providing actionable feedback to users.
Scalability: Simplifies managing certificates across multiple repositories or organizations with minimal configuration.
Ease of Use: Documentation updates provide clear instructions for users, reducing onboarding complexity for certificate management.

Notes:

The variable ENABLE_NUKE_CERTS must be set to true in the repository or organization settings for automatic certificate recreation.
The optional variable FORCE_NUKE_CERTS can be used to override default behavior and force certificate nuking when necessary.

‼️Follow-Up Actions‼️

Users are encouraged to update their repository secrets and variables to align with the new workflows.
For those using multiple GitHub usernames, refer to the updated instructions in testflight.md for certificate handling.

This PR significantly improves the browser build experience by automating critical parts of the build and certificate management process.

marionbarker · 2025-01-15T19:57:31Z

Test:

Test with a GitHub organization account

I have ENABLE_NUKE_CERTS as org variable set to true
I revoked my Distribution Certificate at: https://developer.apple.com/account/resources/certificates/list
I did the action: Build Trio - it was successful
- https://github.com/loopdocs-tester/Trio/actions/runs/12792708525
- Got 3 notifications: certs were revoked and recreated; warning about other apps and assurance that existing apps will keep working

Test with a personal GitHub account

Configure a Trio fork in my personal GitHub account, marionbarker and create a new Match-Secrets
After a successful build, revoke the Distribution Certificates (there were two: one for loopdocs-tester and one for marionbarker)
Use the Create Certificates in marionbarker - it was successful
- https://github.com/marionbarker/Trio/actions/runs/12794216238
Build
- successful build: https://github.com/marionbarker/Trio/actions/runs/12795423261

Test when using two GitHub accounts

Move this to a separate comment because it is a special case and somewhat confusing.

bjornoleh · 2025-01-15T20:17:31Z

It would be interesting to know what happens when nuke_certs is run when you have more than one distribution certificate. You can test this with the FORCE_NUKE_CERTS variable.

marionbarker · 2025-01-15T20:56:09Z

Test when using two GitHub accounts

NOTE- this is a special case. Most users will only build from one account (each account needs a separate Distribution Certificate). This situation is why there is a FORCE_NUKE_CERTS environment setting. Most people will not need to use that.

Previous test of build was successful for marionbarker; so switch to loopdocs-tester account.

Success: run action Create Certificates in loopdocs-tester
- https://github.com/loopdocs-tester/Trio/actions/runs/12794321156
Success: run build action in loopdocs-tester:
- https://github.com/loopdocs-tester/Trio/actions/runs/12795723767
- Check and I still only have one certificate - probably because I started with a new Match-Secrets for marionbarker and have only built Trio for each username
- I have seen that before where only one certificate is needed and other times where there are two

Test another build that does not have the updated files

Use loopdocs-tester/LoopFollow_Second with main branch as default branch.

As expected build fails
The create certs fails too though with Certificate 'W4ZQ664Y86' (stored in your storage) is not available on the Developer Portal for the user

Use FORCE_NUKE_CERTS

Return to loopdocs-tester Trio and add set FORCE_NUKE_CERTS = true as organization environment variable.

Success: run Create Certificates
- this nuked and created new
immediately restore FORCE_NUKE_CERTS = false

Builds now work with both usernames

Return to loopdocs-tester/LoopFollow_Second with main branch as default branch.

Success: run create certs
Success: run build Loop Follow

Return to marionbarker Trio and test the build

Success: run build Trio

Return to loopdocs-tester Trio and test the build

Success: run build Trio

marionbarker

The changes that are here are reviewed, tested and approved.
However, the fastlane/testflight.md file also needs to be modified.

I am marking this approved, in case you want the testflight.md file to be a separate PR. If you update that file for this PR, I will review again.

Will avoid halting the workflow on invalid certs during validate_secrets.yml, as this is handled in crate_certs.yml Add grep for "Your certificate .* is not valid"', but exit without error Remove unnecessary annotation output for 'Unable to create a valid authorization token for the App Store Connect API.' Remove misleading grep "No code signing identity found" -e "Could not install WWDR certificate" and error "No code signing identity found" or "Could not install WWDR certificate"

bjornoleh · 2025-01-18T12:00:02Z

We got to test the workflow with an actually expired certificate before the last commit. This caused the
"Validate Fastlane Secrets" step in validate_secrets.yml to exit fastlane validate_secretswith error and stop the workflow:

[13:49:02]: Called from Fastfile at line 250
[13:49:02]: ```
[13:49:02]:     248:	    find_bundle_id("#{BUNDLE_ID}")
[13:49:02]:     249:	
[13:49:02]:  => 250:	    match(
[13:49:02]:     251:	      type: "appstore",
[13:49:02]:     252:	      git_basic_authorization: Base64.strict_encode64("#{GITHUB_REPOSITORY_OWNER}:#{GH_PAT}"),
[13:49:02]: ```

 Your certificate 'X59XX98686.cer' is not valid, please check end date and renew it if necessary

005339f attempts to fix this.

The workflow did already and does still work in the case of manually revoking the Distribution certificate, now we need a confirmation that the latest changes also work with an actually expired certificate.

I believe @MikePlante1 is a candidate for testing this soon. Please first make an attempt to build with an expired certificate using Trio dev, so that we can check the logs and confirm that the certificate is expires. Then push the certs_trio branch to your repository and run 4. Build Trio from the certs_trio branch. This will automatically include a run of validate_secrets.yml and create_certificates.yml before the build is started. Hopefully no further action is needed.

Check for success by confirming

the presence of a new Distribution certificate in https://developer.apple.com/account/resources/certificates/list
Match-Secrets repository commit history showing updated files in certs/distribution and profiles/appstore

bjornoleh · 2025-01-18T12:07:49Z

To see the incoming changes to docs, please refer to https://github.com/bjornoleh/Trio/pull/27/files, which is not yet included in the PR (but will be when everything is confirmed to be working).

MikePlante1 · 2025-01-20T16:49:04Z

My cert expired 2 days ago, so I took the opportunity to test this PR, which worked successfully:

❌ My default branch was set to the latest dev, and I ran 4. Build Trio, which failed as expected

https://github.com/MikePlante1/Trio/actions/runs/12871244468

❌ I duplicated by dev branch, called it cert-test, merged this PR into it, set it as my default branch, ran 4. Build Trio, which failed

https://github.com/MikePlante1/Trio/actions/runs/12871672047

✅ In my Trio repo, I went to ⚙️ > Secrets and variables > Actions > Variables > New repository variable, and created a new variable named with name ENABLE_NUKE_CERTS and value true. I then re-ran 4. Build Trio, which built successfully.

https://github.com/MikePlante1/Trio/actions/runs/12871883339

bjornoleh · 2025-01-20T17:36:34Z

Thank you @MikePlante1 for testing! This looks perfect to me!

I have pushed the update to testflight.md (readme changes only, no impact on code).

@marionbarker , hopefully this will be the final review! :-)

MikePlante1

Approving, as my test with expired certificates was successful.

marionbarker

Sorry - my PR had a typo for testflight.md.
I will approve after this minor fix.

MikePlante1

Reapproving after typo fix.

marionbarker

Minor changes - can be ignored. I would not have commented except the "Certficate" has been fixed in other repos so the diff caught my eye.

- fix typo in "Certificates" - remove a whitespace in validate_secrets.yml

bjornoleh · 2025-01-26T13:52:51Z

Please give another approval after fixing the typo and whitespace as mentioned above by @marionbarker

Tagging @MikePlante1 too :-)

marionbarker

LGTM

MikePlante1

Reapproving

Sjoerd-Bo3

LGTN

Automate handling of Distribution certificates and profiles

f06675b

bjornoleh requested a review from marionbarker January 15, 2025 14:25

marionbarker previously approved these changes Jan 15, 2025

View reviewed changes

bjornoleh dismissed marionbarker’s stale review via 005339f January 18, 2025 11:40

bjornoleh marked this pull request as ready for review January 18, 2025 11:41

bjornoleh requested review from AndreasStokholm, MikePlante1, Sjoerd-Bo3, aug0211, dnzxy and t1dude as code owners January 18, 2025 11:41

MikePlante1 previously approved these changes Jan 20, 2025

View reviewed changes

marionbarker requested changes Jan 20, 2025

View reviewed changes

Comment thread fastlane/testflight.md Outdated

bjornoleh dismissed MikePlante1’s stale review via dae5c36 January 20, 2025 18:51

bjornoleh force-pushed the certs_trio branch from 6ef04f0 to dae5c36 Compare January 20, 2025 18:51

MikePlante1 previously approved these changes Jan 20, 2025

View reviewed changes

marionbarker previously approved these changes Jan 20, 2025

View reviewed changes

marionbarker reviewed Jan 26, 2025

View reviewed changes

Comment thread .github/workflows/validate_secrets.yml Outdated

Comment thread fastlane/testflight.md Outdated

marionbarker mentioned this pull request Jan 26, 2025

Match main updates in dev, Automate certificates, Update to Fastlane 2.225.0 LoopKit/LoopWorkspace#234

Merged

Update testflight.md for automated Distribution certificate renewal

cc43497

- fix typo in "Certificates" - remove a whitespace in validate_secrets.yml

bjornoleh dismissed stale reviews from marionbarker and MikePlante1 via cc43497 January 26, 2025 13:51

bjornoleh force-pushed the certs_trio branch from dae5c36 to cc43497 Compare January 26, 2025 13:51

marionbarker approved these changes Jan 26, 2025

View reviewed changes

MikePlante1 approved these changes Jan 26, 2025

View reviewed changes

Sjoerd-Bo3 approved these changes Jan 26, 2025

View reviewed changes

Sjoerd-Bo3 merged commit 282a5c5 into nightscout:dev Jan 26, 2025

This was referenced Jan 26, 2025

Release v0.2.3 #462

Merged

Release v0.2.3 #463

Merged

Uh oh!

Uh oh!

Conversation

bjornoleh commented Jan 15, 2025 • edited by dnzxy Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary:

Key Changes:

Benefits:

Notes:

‼️Follow-Up Actions‼️

Uh oh!

marionbarker commented Jan 15, 2025

Test:

Test with a GitHub organization account

Test with a personal GitHub account

Test when using two GitHub accounts

Uh oh!

bjornoleh commented Jan 15, 2025

Uh oh!

marionbarker commented Jan 15, 2025

Test when using two GitHub accounts

Test another build that does not have the updated files

Use FORCE_NUKE_CERTS

Builds now work with both usernames

Uh oh!

marionbarker left a comment

Choose a reason for hiding this comment

Uh oh!

bjornoleh commented Jan 18, 2025

Uh oh!

bjornoleh commented Jan 18, 2025

Uh oh!

MikePlante1 commented Jan 20, 2025

Uh oh!

bjornoleh commented Jan 20, 2025

Uh oh!

MikePlante1 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marionbarker left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MikePlante1 left a comment

Choose a reason for hiding this comment

Uh oh!

marionbarker left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bjornoleh commented Jan 26, 2025

Uh oh!

marionbarker left a comment

Choose a reason for hiding this comment

Uh oh!

MikePlante1 left a comment

Choose a reason for hiding this comment

Uh oh!

Sjoerd-Bo3 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

bjornoleh commented Jan 15, 2025 •

edited by dnzxy

Loading

MikePlante1 left a comment •

edited

Loading