Skip to content

test(conformance): migrate workspace lifecycle tests to workspace stories - #4607

Open
politerealism wants to merge 1 commit into
NVIDIA:mainfrom
politerealism:test/3954-workspace-stories/politerealism
Open

politerealism wants to merge 1 commit into
NVIDIA:mainfrom
politerealism:test/3954-workspace-stories/politerealism

Conversation

@politerealism

Copy link
Copy Markdown
Contributor

Summary

Move the workspace lifecycle tests from the legacy e2e crate into a new workspaces conformance area. Workspace create, inspect, scope and delete behavior isn't driver-specific, so it belongs with the other conformance stories and runs against any prepared gateway. Stacked on #4538, which relocates the suites.

Related Issue

Contributes to #3954. No separate issue: this migrates existing coverage.

Changes

  • Add a workspaces area (cli/tests/workspaces/) with three stories, and list it in the conformance README. The original workspace_full_crud_lifecycle checked seven behaviors in one test, so it's split to make a failure point at one behavior:
    • workspace_full_crud_lifecycle: create, get, listed as Active, delete when empty, gone afterwards. The name is kept so filters keep working.
    • template_is_scoped_to_its_workspace (new name): a resource is listed in its own workspace and in the all-workspaces view with its owner, and isn't visible in the default workspace.
    • workspace_terminating_rejects_creates: name kept. It now also covers the deletion guard, because a refused delete is what moves a workspace to Terminating, a phase in the public WorkspacePhase enum. While Terminating, new resources are rejected and existing ones can still be deleted; retrying the delete then succeeds. This is server behavior; the user docs don't describe the phase.
  • The scoped resource is a sandbox template, not a provider. The original used a provider of type openai, which needs a provider profile that only the wrapper lanes import. A freshly provisioned tmachine gateway has none, so those stories failed there (found by running the suite in the Ubuntu rootful Docker guest). A template is workspace-scoped, is covered by the same delete guard, and needs no profile, credential or image. This drops the provider list --all-workspaces check; the all-workspaces view is still checked, for templates.
  • Assertions check exit status and listed state from --output json, not message text. The CLI has no structured error and no structured workspace get, so those limits are documented in the area. Because a rejection by exit status could have another cause, the terminating story first creates and deletes the same template while the workspace is Active, so the later rejection can only be about the Terminating state.
  • Listings are paginated and capped at 100 pages. A missing workspaces or templates array in the output is an error, so a "not listed" check can't pass on a malformed listing.
  • A story removes its own resources. Cleanup runs only when a story fails, removes its templates and workspace, and reports anything other than "not found". The shared runner doesn't track workspaces, so this lives in small helpers (run, succeeds, fails, finish) in the area, and sandbox_templates has its own copy. I can move a shared version into the support crate if you want one.
  • Delete e2e/rust/tests/workspace_lifecycle.rs, its Cargo target and its entry in the Podman archive exclusion list.

Coverage after this change

The conformance archive and e2e_run_openshell_conformance run every test target in the conformance package, so the new workspaces binary runs in every lane that runs conformance. The old test was excluded from the Podman tmachine archive, so Podman gains coverage. The Docker and Kubernetes wrapper lanes run conformance first, so the coverage moves there. Kubernetes operator-workspace mode skips conformance (default workspace not allowlisted), as it did for the old test's lane, and the area documents that it isn't covered there.

Testing

  • mise run pre-commit passes
  • The three stories pass against a rootless Podman gateway: cargo test --manifest-path e2e/suites/conformance/Cargo.toml --test workspaces
  • Deliberate-failure checks on the same gateway, each failing exactly the intended story: the terminating story expecting Active, the CRUD story expecting Terminating, the scoping story looking in the default workspace, and the listing key missing from the JSON (fails the two stories that read workspaces).
  • tmachine guests, run on a throwaway branch that combines the three migrations: the full conformance archive passes in the Ubuntu rootful Docker guest (tmachine test ubuntu-docker-rootful binaries conformance) and in the Ubuntu K3s guest (tmachine test ubuntu-k3s k3s conformance). The first Docker run is what exposed the provider-profile dependency fixed in the workspaces PR, and it also failed once on the existing lifecycle stop_start (ContainerExited: Container exited while waiting for Ready), which passed on the rerun. Each PR alone was not run in tmachine.
  • Not run: macOS.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (not applicable)

@copy-pr-bot

copy-pr-bot Bot commented Oct 15, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

Copy link
Copy Markdown

This pull request has had no activity for 14 days and is now marked stale.

@github-actions github-actions Bot added the state:stale Inactive item at risk of automatic closure. label Oct 30, 2026
elezar added a commit to politerealism/OpenShell that referenced this pull request Nov 13, 2026
Move e2e/rust workspace_lifecycle into a conformance workspaces suite on
#[testkit::test], split into workspace_full_crud_lifecycle,
template_is_scoped_to_its_workspace and workspace_terminating_rejects_creates.
Use a sandbox template as the scoped resource so no provider profile is
needed, and assert on exit status and listed state rather than message text.
Drop the legacy test, its Cargo target, and its Podman archive exclusion.

Supersedes NVIDIA#4607.

Signed-off-by: Evan Lezar <[email protected]>
@elezar
elezar force-pushed the test/3954-workspace-stories/politerealism branch from a2554c4 to 30f47a9 Compare November 13, 2026 13:12
@github-actions

Copy link
Copy Markdown

Maintainer list change

Gains approval rights:

Confirm every change is intended. Anyone listed here can single-handedly satisfy OpenShell / Maintainer Approval.

Move e2e/rust workspace_lifecycle into a conformance workspaces suite on
#[testkit::test], split into workspace_full_crud_lifecycle,
template_is_scoped_to_its_workspace and workspace_terminating_rejects_creates.
Use a sandbox template as the scoped resource so no provider profile is
needed, and assert on exit status and listed state rather than message text.
Drop the legacy test, its Cargo target, and its Podman archive exclusion.

Supersedes NVIDIA#4607.

Signed-off-by: Evan Lezar <[email protected]>
@elezar
elezar force-pushed the test/3954-workspace-stories/politerealism branch from 30f47a9 to d63af4f Compare November 13, 2026 15:19
@elezar
elezar changed the base branch from codex/4119-suite-layout to main November 13, 2026 15:19
@elezar elezar added area:build CI/CD workflows, releases, build targets, packaging, and artifact publication. topic:testing Unit, integration, regression, conformance, E2E tests, coverage, flakes, and test infrastructure. test:e2e Requires end-to-end coverage and removed state:stale Inactive item at risk of automatic closure. labels Nov 13, 2026
@github-actions

Copy link
Copy Markdown

Label test:e2e applied, but pull-request/4607 does not exist yet. A maintainer needs to comment /ok to test d63af4f19e8a14827292aaafbf795aef9f2dbe49 to mirror this PR. Once the mirror exists, re-apply the label or re-run Branch E2E Checks from the Actions tab.

@elezar

elezar commented Nov 13, 2026

Copy link
Copy Markdown
Member

/ok-to-test d63af4f

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:build CI/CD workflows, releases, build targets, packaging, and artifact publication. test:e2e Requires end-to-end coverage topic:testing Unit, integration, regression, conformance, E2E tests, coverage, flakes, and test infrastructure.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants