Skip to content

Make nightly release publication deterministic before updating the Homebrew cask #309

@nicobistolfi

Description

@nicobistolfi

Summary

The nightly workflow sometimes leaves the rolling main-nightly GitHub release in draft state instead of publishing it as a prerelease, while still updating the Homebrew cask to point at the new asset URLs. That creates a broken install path where brew reinstall vigilante-nightly fails with 404s even though the workflow run itself reported success. The workflow should make release publication state deterministic and must verify the release is publicly downloadable before updating the tap.

Problem

  • The nightly pipeline occasionally produces a main-nightly release that remains draft instead of ending in published prerelease state.
  • When that happens, the Homebrew cask is still updated to point at https://github.com/aliengiraffe/vigilante/releases/download/main-nightly/..., but those asset URLs return 404 because draft release assets are not publicly downloadable.
  • This breaks the documented installation path: brew update && brew reinstall vigilante-nightly.
  • The failure is operationally confusing because the workflow can still appear successful while the published install artifact is not actually consumable.

Context

  • Repository: aliengiraffe/vigilante.
  • The current nightly workflow in .github/workflows/nightly.yml builds snapshot artifacts, deletes any existing main-nightly release, recreates it with gh release create ... --prerelease, and then updates the Homebrew tap.
  • A recent failure showed the cask pointing to version 0.0.0-nightly.20260326170625.503113fdfe81 while curl -I against the Linux asset URL returned 404, and gh release view main-nightly showed the release as draft at that time.
  • The latest nightly currently appears healthy again, so this is intermittent rather than a permanently broken workflow.
  • The fix should focus on publication correctness and verification, not on changing the user-facing installation command.

Desired Outcome

  • Every successful nightly workflow run leaves main-nightly in a publicly available prerelease state, never as a lingering draft.
  • The Homebrew cask is only updated after the workflow has confirmed that the new nightly release is published and its referenced assets are reachable.
  • If release publication fails or remains draft, the workflow should fail clearly and must not push a broken cask update.
  • Out of scope: redesigning nightly distribution away from GitHub Releases or changing the Homebrew install surface area.

Implementation Notes

  • Investigate why gh release create ... --prerelease can still leave the rolling nightly release as draft in some runs. Treat this as a workflow correctness bug rather than a tap bug.
  • Make the release-publication step deterministic. If the workflow depends on deleting and recreating main-nightly, ensure the recreated release is fully published before continuing.
  • Add an explicit post-publication verification step before the tap update. At minimum, confirm the release is not draft and that the expected asset URLs for the generated nightly version are accessible.
  • If GitHub release state is eventually consistent, add a bounded retry/poll step in the workflow rather than assuming creation is immediately visible in the desired final state.
  • The tap update should be gated on successful publication verification; it must not run when the release remains draft or the assets are still unavailable.

Acceptance Criteria

  • A successful nightly workflow run always leaves main-nightly in isDraft=false and isPrerelease=true state.
  • The workflow fails if the rolling nightly release remains draft after creation instead of continuing to the Homebrew tap update.
  • The workflow verifies that the generated nightly asset URLs are reachable before committing the new cask version.
  • The Homebrew tap is not updated when release publication is incomplete or the referenced assets are unavailable.
  • brew reinstall vigilante-nightly does not regress into the recent 404 failure mode caused by a draft nightly release.

Testing Expectations

  • Add or update workflow-level validation that checks the final release state for main-nightly after the publish step.
  • Add regression coverage for the gate that blocks tap updates when the release is still draft or assets are unavailable.
  • Add tests for any helper scripts introduced to validate release state or asset availability.
  • Include a manual verification note or reproducible validation command showing how to confirm the resulting release is published and downloadable after the workflow completes.

Operational / UX Considerations

  • The documented install path should remain trustworthy. If the nightly workflow says success, users should be able to install that nightly immediately from Homebrew.
  • Failure should be loud and early in CI instead of surfacing later as a user-visible 404 from GitHub release downloads.
  • Keep the workflow logs explicit enough that operators can tell whether the failure happened during release publication or during tap update gating.

Metadata

Metadata

Assignees

Labels

vigilante:doneVigilante completed its work on the issue and no further automation is expected.

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions