Skip to content

feat(ci): compose release qualification from targets, flows, and suites #2972

Closed

Description

@SDAChess

Description

Create a reusable release qualification entry point that consumes an immutable candidate artifact set and composes three independently maintained dimensions:

  • targets describe the platform, architecture, runtime, package mechanism, and environment lifecycle;
  • flows describe how a release is installed or changed, such as clean install or forward upgrade;
  • suites describe the behavior to verify, such as driver conformance.

The qualification matrix must select supported combinations explicitly rather than assume every target, flow, and suite forms a valid Cartesian product.

Context

The first consumer will be the Fedora rootless Podman proof-of-concept covering clean installation and N to N+1 forward upgrade. Existing release canaries mix environment setup, installation, and assertions in individual workflow jobs, making each new matrix entry expensive to add and difficult to compare.

Definition of Done

  • Qualification accepts an immutable candidate artifact manifest containing the release version, source commit, and artifact identities or digests.
  • Targets, flows, and suites have separate definitions and lifecycle boundaries.
  • Adding a target does not require copying or modifying flow or suite implementations.
  • Adding a flow or suite does not require embedding target-specific provisioning logic.
  • Supported target, flow, and suite combinations are selected explicitly without generating an unsupported Cartesian product.
  • The same qualification entry point can run locally and from GitHub Actions.
  • Each selected combination produces a clear pass/fail result plus machine-readable results and retained diagnostics.
  • Documentation explains how to add one target, one flow, or one suite independently.

Activity

  1. added this to the OpenShell 0.1.0 milestone on Aug 27, 2026
  2. added
    topic:testingUnit, integration, regression, conformance, E2E tests, coverage, flakes, and test infrastructure.
    on Aug 27, 2026
  3. github-actions commented on Sep 10, 2026

    @github-actions

    This issue has had no activity for 14 days and is now marked stale. It may be closed in 7 days if there is no further activity. Comment or remove the state:stale label to keep it open.

  4. removed
    state:staleInactive item at risk of automatic closure.
    on Sep 16, 2026
  5. SDAChess commented on Sep 17, 2026

    @SDAChess
    MemberAuthor

    PR #3419 completes this issue under the revised tmachine composition model.

    The original target / flow / suite design is superseded by three independent dimensions:

    • a scenario defines the machine and runtime setup;
    • an installation profile defines how OpenShell is installed;
    • a testsuite defines the behavior exercised after installation.

    #3419 adds the missing installation boundary, allowing runtime scenarios to be reused across native and future package-based installations. A separate flow abstraction is no longer needed: clean installation is represented by an installation profile, while stateful operations such as installing N and upgrading to N+1 will be implemented as a testsuite.

    Once #3419 merges, #2972 can be considered complete. The N-to-N+1 work remains tracked separately by #2975.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

topic:testingUnit, integration, regression, conformance, E2E tests, coverage, flakes, and test infrastructure.

Type

No type

Projects

Relationships

None yet

Development

No branches or pull requests

Issue actions