[release/0.27] Add V2 logical container lifecycle management - #336

* Add V2 logical container resource foundation

Introduce logical container and network APIs, delegate logical container lifecycle operations to physical resources, and track persistent workload ownership. Standardize V2 deletion dispatch and terminal startup/stop outcomes with shared API-backed coverage.

Co-authored-by: Copilot App <[email protected]>

* Replace synthetic V2 controller tests with behavioral coverage

Exercise logical readiness through the shared API server and running physical controller using runtime port-mapping fault injection. Remove redundant private deletion-handler tests already covered by retained-resource integration cases.

Co-authored-by: Copilot App <[email protected]>

* Preserve lifecycle duplicate precedence and honor namespace workload IDs

Use stable key sorting for logical container environment, container labels, and build labels. Resolve namespace workload overrides during asynchronous persistent ownership settlement, retaining startup leases on lookup or recording errors and supporting overrides without a global default.

Co-authored-by: Copilot App <[email protected]>

* Import Aspire regression scaffold compatibility fix from #292

Preserve all build directives selected by the daily CLI template instead of requiring an explicit SDK directive. Leave generated AppHost validation to Aspire, supporting package-based templates without importing the unrelated watcher or dependency changes from PR #292.

Co-authored-by: Copilot App <[email protected]>

* Release logical container realization inputs after physical handoff

Make the selected runtime ID explicitly pending and clear it together with the generated session name after physical creation or verification. Project inline image references from the physical creation spec instead of retaining an adopted runtime ID as a marker.

Co-authored-by: Copilot App <[email protected]>

* Capture logical and physical container system diagnostics

Add scoped resource-stream nesting that preserves context, primary bindings, filters, and independent cleanup. Bind V2 container diagnostics by resource UID and forward owned physical container diagnostics to the logical owner's stream, including asynchronous operation failures. Keep resource-file producer names per entry and verify fanout without duplicate process or resource entries. This provides capture and routing only; V2 log subresource registration remains separate.

Co-authored-by: Copilot App <[email protected]>

* Expose namespaced V2 container log endpoints

Share watch-driven log storage and query handling across API versions while preserving V1 contracts. Add logical and physical container stdout, stderr, and system streamers with one physical-UID runtime capture, snapshot-only logical source resolution, independent client lifetimes, namespace preservation, and explicit unsupported-source errors. Project backing physical identity into logical status and exercise actual endpoints, cancellation, and existing V1 log behavior.

Co-authored-by: Copilot App <[email protected]>

* Drive namespace cleanup directly from relational metadata

Replace per-kind cleanup handlers with one uncached list and bounded delete executor, preserving dependency ordering and pending-resource diagnostics.

Co-authored-by: Copilot App <[email protected]>

* Keep V2 container log subresources with their parent types

Declare logical and physical container log subresources alongside their parents, retaining only resource-neutral storage construction and contracts in shared log types.

Co-authored-by: Copilot App <[email protected]>

* Use container-specific filenames for V2 log streaming

Co-authored-by: Copilot App <[email protected]>

* Expose V2 image acquisition logs through shared capture sources

Capture pull and build output, serialize image acquisition with exclusive leases, and project image startup and container runtime sources directly onto logical resources. Preserve the V1 watcher handoff, fix nested resource sink failure reporting, and share CaptureFile between V1 and V2 controllers.

Co-authored-by: Copilot App <[email protected]>

* Rename shared V2 logging package for resource consumers

Co-authored-by: Copilot App <[email protected]>

* Complete logical container log sources when physical backing disappears

Preserve the original physical source identity in a completed, unavailable descriptor so logical-only followers finish through the existing lifecycle callbacks. Cover physical deletion through an isolated real API server with no physical log request.

Co-authored-by: Copilot App <[email protected]>

* Unify resource log capture filename generation

Use one filename contract for command and runtime captures, preserve V1 startup filenames and runtime capture suffixes, and capture physical image output with ordinary stdout/stderr source names.

Co-authored-by: Copilot App <[email protected]>

* Complete missing-runtime log sources and focus regression coverage

Finish physical and logical followers after confirmed runtime disappearance, wait for image log-writer release in its assertion, and remove unnecessary simulated image aliases. Cover failed pull diagnostics and stream closure at the production Docker/Podman adapter boundary.

Co-authored-by: Copilot App <[email protected]>

* Protect logical backing lifetimes and settle image inspections

Hold owned physical containers and inline images until logical cleanup permits release. Preserve operation outcomes through status conflicts, clean up dependency-pending startup, and verify uncertain backing creates immediately.

Limit logical startup output to owned images, bound image inspection retries, and align reconciliation and integration tests with repository patterns.

Co-authored-by: Copilot App <[email protected]>

* Align physical resource reconciliation and namespace guards

Move physical resource initialization, state updates, and status projection into management and deletion handlers. Discard completed container and image data alongside finalizer removal, and release resource logs during cleanup.

Replace namespace state-machine stages with a common V2 namespace guard while preserving operation settlement and logical backing holds.

Co-authored-by: Copilot App <[email protected]>

* Make V2 finalizer sequencing explicit

Use an explicit ensureFinalizer call followed by management only when no finalizer change is needed, matching the network controller's control flow without changing deletion guards.

Co-authored-by: Copilot App <[email protected]>

* Simplify V2 reconciliation and logical backing coordination

Separate granular lifecycle states from outstanding asynchronous work. Discover logical backings by owner metadata, require expected children before handoff, and propagate deletion to every owned child.

Preserve cleanup status before releasing finalizers, keep adopted runtimes out of workload ownership records, and retry post-pull image inspection without repeating acquisition.

Co-authored-by: Copilot App <[email protected]>

* Stop existing replacement targets and settle workloads on identity

Acquire and stop the existing named runtime when stop and replace are requested before physical creation. Settle logical workload registration when the physical runtime ID is reported, without waiting for a settled lifecycle phase.

Co-authored-by: Copilot App <[email protected]>

* Release runtime captures through observed log sources

Use a log-source provider capability for primary capture identity and release the observed source UID on deletion, including logical-only log consumers. Preserve existing startup and completion handling.

Co-authored-by: Copilot App <[email protected]>

* Cover bounded volume removal during namespace deletion

Exercise the real API and running controllers with an in-use volume, verifying the grace period, abandonment status, finalizer release, and completion of namespace deletion using existing test infrastructure.

Co-authored-by: Copilot App <[email protected]>

* Remove unused V1 resource streamer adapter

Co-authored-by: Copilot App <[email protected]>

* Append authoritative DCP runtime labels after caller labels

Remove the first-match setter and rely on last-value-wins conversion so duplicate caller labels cannot override system metadata. Cover physical label collisions and persistent logical-container reuse.

Co-authored-by: Copilot App <[email protected]>

* Synchronize log readiness with resource lifecycle processing

Dispatch lifecycle events in per-resource FIFO order and replace request event backlogs with coalesced latest-state notifications. Share a per-resource RW lock across snapshot processing and lifecycle callbacks so deletion cannot overtake requests processing the previous state, while allowing concurrent readers.

Cover latest-state consumption, both deletion/registration orderings, concurrent requests, cross-resource progress, scheduling fairness, and disposal. Preserve existing streamer and finite-copy behavior.

Co-authored-by: Copilot App <[email protected]>

* Requeue image acquisition when exclusive pull leases are held

Acquire pull leases once per operation attempt instead of waiting inside the bounded worker queue. Keep direct pulls and build-base acquisition pending on contention and retry through normal reconciliation without consuming pull retries.

Allow WithResourceLease to forward existing acquisition options so single-attempt calls retain exclusivity and automatic release. Cover same-process contention, unrelated work progress, and recovery after lease release.

Co-authored-by: Copilot App <[email protected]>

* Align V2 lifecycle-key collection ordering with V1

Sort mounts by target, ports by protocol and host port, copied files by destination, and build secrets by ID before hashing the copied configuration. Preserve equal-key declaration order and caller-provided digest behavior.

Cover reordering equivalence, meaningful ordering, input immutability, and persistent runtime reuse after port reordering. Leave acquisition-policy inclusion unchanged.

Co-authored-by: Copilot App <[email protected]>

* Exclude dynamic networks from V2 container lifecycle keys

Match V1 by omitting network attachments and aliases from the copied configuration used for hashing. Preserve the declared spec and avoid making dynamic connection changes trigger persistent container replacement.

Cover membership, ordering, aliases, empty inputs, and immutability; update the intentional V2 golden key and resource plan.

Co-authored-by: Copilot App <[email protected]>

* Separate namespace readiness checks from controller status policy

Return readiness, condition reason, and lookup error from a read-only namespace check. Let all six callers choose their phases, Ready conditions, logging, and reconciliation flags while preserving existing admission behavior.

Document the reduced helper contract and cover initial lookup failure, recovery, and post-initialization stop behavior through the real API/controller environment.

Co-authored-by: Copilot App <[email protected]>

* Clarify logical container backing lookup diagnostic

Use the reviewer's suggested message to describe listing the physical containers backing the logical resource.

Co-authored-by: Copilot App <[email protected]>

* Confirm recorded logical backings are absent before deletion completes

When a physical UID is recorded but the cached owner list is empty, confirm absence through an uncached namespace list filtered by controller-owner UID. Reuse existing container/image deletion loops for discovered children and retry lookup errors without releasing the logical finalizer.

Cover successful creation before cache observation, duplicate owned backings, unrelated resource preservation, and cleanup completion after actual removal. Document the UID-gated confirmation rule.

Co-authored-by: Copilot App <[email protected]>

* Copy finite stdio logs asynchronously

Return a completion channel for non-follow and completed-resource log copies so client backpressure does not hold resource processing locks. Preserve first-EOF behavior without registering lifecycle followers, and let storage cancellation close the response pipe to unblock delivery.

Cover finite-copy formatting, EOF, slow readers, lifecycle progress, request cancellation, and storage disposal.

Co-authored-by: Copilot App <[email protected]>

---------

Co-authored-by: Copilot App <[email protected]>
(cherry picked from commit 5901d3d)