Add V2 logical and physical container execution by danegsta · Pull Request #309 · microsoft/dcp

David Negstad (danegsta) · 2026-10-20T23:24:46Z

V2 containers need a way to run additional commands without treating each command as another container or a host OS process. This adds namespaced, one-shot container execution with separate runtime ownership and logical container targeting.

LogicalContainerExec waits for its logical container's current-generation readiness and uses its published physical reference. An owned PhysicalContainerExec pins the target identity and owns command launch, completion, output capture, and attached-session cancellation. Both use the existing V2 reconciliation, shared stdout/stderr/system logging, and metadata-driven namespace cleanup patterns. Creation recovery adopts an already-created owned execution before reconsidering startup prerequisites, and retrying cannot silently switch to a replacement container.

Scope and boundaries

Command arguments and environment values are literal; host environment files are passed to the runtime. V1 service-template expansion is deferred.
Executions do not restart, persist, adopt PIDs, or stop/remove their target containers. Cancelling the attached runtime session does not guarantee termination of the in-container process; the existing runtime contract provides no such confirmation.
PTY support remains follow-up work. The V2 plan now covers process terminals, primary-container terminals, and container-exec terminals using the existing terminal API/HMP patterns.

Validation

Real-API/running-controller regressions cover execution and nonzero exit, immediate completion, launch failure, cancellation during launch and runtime, target replacement, lost creation responses, status conflicts, log following, and namespace cleanup. Build, the full native race-enabled test suite, and Windows/macOS/Linux lint passed with real-runtime/E2E suites disabled. Linux cross-build passed; a subsequent Linux-container rerun was unavailable because local Docker/Podman were not responsive.

This layer targets danegsta-logical-process-resource.

Introduce logical container and network APIs, delegate logical container lifecycle operations to physical resources, and track persistent workload ownership. Standardize V2 deletion dispatch and terminal startup/stop outcomes with shared API-backed coverage. Co-authored-by: Copilot App <[email protected]>

Exercise logical readiness through the shared API server and running physical controller using runtime port-mapping fault injection. Remove redundant private deletion-handler tests already covered by retained-resource integration cases. Co-authored-by: Copilot App <[email protected]>

Delegate runtime network realization to physical resources, manage persistent lifecycle keys and shared leases, and translate logical network attachments for new containers without changing adoption behavior. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Bring the feature branch onto the base used by PR CI, including the upstream WSLC runtime changes. Update three runtime-label test callers to the map contract while preserving slice-valued API labels. Co-authored-by: Copilot App <[email protected]>

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Use stable key sorting for logical container environment, container labels, and build labels. Resolve namespace workload overrides during asynchronous persistent ownership settlement, retaining startup leases on lookup or recording errors and supporting overrides without a global default. Co-authored-by: Copilot App <[email protected]>

Resolve workload annotation overrides during persistent network lease settlement, retaining leases on namespace lookup failures, and preserve duplicate label precedence in lifecycle keys. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Preserve all build directives selected by the daily CLI template instead of requiring an explicit SDK directive. Leave generated AppHost validation to Aspire, supporting package-based templates without importing the unrelated watcher or dependency changes from PR #292. Co-authored-by: Copilot App <[email protected]>

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Make the selected runtime ID explicitly pending and clear it together with the generated session name after physical creation or verification. Project inline image references from the physical creation spec instead of retaining an adopted runtime ID as a marker. Co-authored-by: Copilot App <[email protected]>

Retain the runtime ID only as a pending adoption input and clear it once the physical resource owns the selected identity. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Replace per-controller ownership permutations and private environments with one persisted-ownership scenario in the shared real API/controller environment. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Preserve the parent pending-input lifetime and inline-image projection changes alongside child network realization documentation. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

… refs Use logical network readiness and its published physical reference instead of duplicating physical lookups. Create inline image resources before waiting for networks so image preparation can proceed independently. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Use shared setup for scoped lifecycle and ownership scenarios, consolidate deletion ordering coverage, move duplicate-label sorting regression to API tests, and remove unrelated cancellation and runtime retry checks. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Delegate process execution to identity-bound physical resources, with session, persistent, and existing lifecycle modes. Record retained process ownership before releasing startup leases, including deletion during launch. Treat process launch errors as terminal. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Add scoped resource-stream nesting that preserves context, primary bindings, filters, and independent cleanup. Bind V2 container diagnostics by resource UID and forward owned physical container diagnostics to the logical owner's stream, including asynchronous operation failures. Keep resource-file producer names per entry and verify fanout without duplicate process or resource entries. This provides capture and routing only; V2 log subresource registration remains separate. Co-authored-by: Copilot App <[email protected]>

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Keep startup settlement and ownership recording in the logical controller before requesting physical deletion. Remove the startup finalizer handshake, deletion-time physical status projection, and the guard introduced to support them. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Share watch-driven log storage and query handling across API versions while preserving V1 contracts. Add logical and physical container stdout, stderr, and system streamers with one physical-UID runtime capture, snapshot-only logical source resolution, independent client lifetimes, namespace preservation, and explicit unsupported-source errors. Project backing physical identity into logical status and exercise actual endpoints, cancellation, and existing V1 log behavior. Co-authored-by: Copilot App <[email protected]>

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Replace per-kind cleanup handlers with one uncached list and bounded delete executor, preserving dependency ordering and pending-resource diagnostics. Co-authored-by: Copilot App <[email protected]>

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Declare logical and physical container log subresources alongside their parents, retaining only resource-neutral storage construction and contracts in shared log types. Co-authored-by: Copilot App <[email protected]>

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Co-authored-by: Copilot App <[email protected]>

Capture pull and build output, serialize image acquisition with exclusive leases, and project image startup and container runtime sources directly onto logical resources. Preserve the V1 watcher handoff, fix nested resource sink failure reporting, and share CaptureFile between V1 and V2 controllers. Co-authored-by: Copilot App <[email protected]>

Preserve image-log source projection before waiting on published logical network readiness. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Cache bounded capture disposal with OnceValue and log output-removal failures without blocking process finalization, replacement, or retirement of confirmed-gone records. Update cleanup regressions and documentation, share the process key prefix, clarify log-stream completion and raw filtering, and distinguish missing process data from the API object. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Align exec output disposal with bounded best-effort removal; cached disposal failures must not block finalization or retain deleted execution state. Preserve held capture lifetime and durable final disposition. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 89db152a-aa9d-4e1d-9058-d4c0090c88d6

Share a host-lifetime bounded cleanup queue across process and image reconcilers. Preserve capture-drain ordering and best-effort disposal, reuse CloseAndLog, and keep file removal independent of resource finalization and process operations. Cover blocked cleanup through the real API, await asynchronous removal in existing tests, and consolidate overlapping serial log coverage. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Share the host-scoped cleanup queue with exec; close writers before completion while disposing files asynchronously without blocking finalization. Preserve held captures and verify deletion with blocked cleanup workers. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 89db152a-aa9d-4e1d-9058-d4c0090c88d6

Document best-effort capture removal on resource deletion after process exit or confirmed absence, including PID reuse. Clarify the identity timestamp's use for subsequent persistent-process PID-reuse protection and retain the Linux since-boot distinction. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

…e' into danegsta-v2-container-execution

Preserve exec registrations, logging, cleanup ordering and shared test wiring while incorporating main's process output-test correction and container network status fixes. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 89db152a-aa9d-4e1d-9058-d4c0090c88d6

Copilot

🟡 Changes recommended

Asynchronous capture cleanup retains a resource-scoped logger after its sink is released, so a later disposal error can recreate the sink and leak a file descriptor.

1 open issue

Avoid recreating released log sink during async cleanup · New

1 resolved since last review

Retain finalizer until attached host session settlement is confirmed

2 files not reviewed

api/v2/zz_generated.deepcopy.go: Generated file
api/v2/zz_generated.model_name.go: Generated file

_{Review effort: Balanced}

Use the controller logger with execution name and UID for asynchronous capture disposal so late cleanup errors cannot reopen released resource log sinks. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 89db152a-aa9d-4e1d-9058-d4c0090c88d6

Karol Zadora-Przylecki (karolz-ms)

One correctness issue remains in confirmed target-loss handling; see the inline comment.

Karol Zadora-Przylecki (karolz-ms)

Additional findings on the V2 container exec controllers (inline). These supplement the existing comments on confirmed target loss and attached-session settlement, which are not repeated here.

What I checked: build, Windows/macOS/Linux lint, generated-code drift (none), unit tests plus the full integration suite with -race, repeated runs of the exec tests, and an ad-hoc real-Podman run of PhysicalContainerExec. All clean except the items below.

🤖 Generated with GitHub Copilot.

Project Missing as Failed with RuntimeContainerMissing so logical containers and exec prerequisites reject confirmed absence through existing failure handling. Keep transient Unknown retryable and deletion completion independent of phase. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 89db152a-aa9d-4e1d-9058-d4c0090c88d6

Honor DCP_PRESERVE_EXECUTABLE_LOGS in shared capture and path cleanup, including image acquisition and runtime-container log descriptors. Keep writer closure and explicit CaptureFile disposal independent of retention policy; cover preservation and normal removal with shared and real API regressions. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 89db152a-aa9d-4e1d-9058-d4c0090c88d6

Avoid formatting nil lookup errors for changed, deleting, or terminal targets. Preserve ContainerUnavailable handling and assert the resulting diagnostic through the real API. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 89db152a-aa9d-4e1d-9058-d4c0090c88d6

Count successful, failed, and NotFound initial resource reads consistently with the other controllers. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 89db152a-aa9d-4e1d-9058-d4c0090c88d6

Apply ResourceVersionChangedPredicate to namespace watches and logical exec's owned physical watch, matching established V2 controller registrations. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 89db152a-aa9d-4e1d-9058-d4c0090c88d6

Move setExitCode beside common status setters and update process and container exec callers without changing comparison or cloning behavior. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 89db152a-aa9d-4e1d-9058-d4c0090c88d6

Add runtime-gated API/controller coverage using shared runtime discovery, resource tracking, and the container probe. Exercise inputs, output and exit results, attached execution deletion, target exit, and exited-target rejection. Real-runtime execution is reserved for enabled CI. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 89db152a-aa9d-4e1d-9058-d4c0090c88d6

Copilot

🔵 Needs a closer look

The new public execution APIs and asynchronous lifecycle coordination warrant human architectural review, especially because Docker/Podman/WSLC runtime behavior remains unverified.

No open issues 🎉

1 resolved since last review

Avoid recreating released log sink during async cleanup

2 files not reviewed

api/v2/zz_generated.deepcopy.go: Generated file
api/v2/zz_generated.model_name.go: Generated file

_{Review effort: Balanced}

Integrate main e1c3eb6 after PR #309 squash merged. Resolve duplicated-history conflicts while preserving the complete service-layer tree and regenerating duplicate OpenAPI definitions. Co-authored-by: Copilot App <[email protected]>

David Negstad (danegsta) and others added 30 commits October 14, 2026 16:36

Merge parent main compatibility updates

091e3d6

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Merge parent namespace ownership fixes

bcc48a2

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Merge parent Aspire scaffold compatibility fix

368ba91

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Limit selected network identity to physical handoff

b9e93a4

Retain the runtime ID only as a pending adoption input and clear it once the physical resource owns the selected identity. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Merge parent logical container state lifetime cleanup

dc92663

Preserve the parent pending-input lifetime and inline-image projection changes alongside child network realization documentation. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Merge parent V2 container system logging

041aa39

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Merge parent V2 container log endpoints

e18a460

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Drive namespace cleanup directly from relational metadata

8fa462b

Replace per-kind cleanup handlers with one uncached list and bounded delete executor, preserving dependency ordering and pending-resource diagnostics. Co-authored-by: Copilot App <[email protected]>

Merge parent common namespace cleanup executor

46260ed

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Keep V2 container log subresources with their parent types

4c5e7ab

Declare logical and physical container log subresources alongside their parents, retaining only resource-neutral storage construction and contracts in shared log types. Co-authored-by: Copilot App <[email protected]>

Merge parent container log resource placement cleanup

54cdc9a

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Use container-specific filenames for V2 log streaming

2f6d400

Co-authored-by: Copilot App <[email protected]>

Merge main to adopt the upstream Aspire regression fix

7bfc3d4

Co-authored-by: Copilot App <[email protected]>

Merge parent image log capture and shared streaming contracts

bab8f59

Preserve image-log source projection before waiting on published logical network readiness. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

David Negstad (danegsta) and others added 6 commits November 3, 2026 18:06

Merge remote-tracking branch 'origin/danegsta-logical-process-resourc…

7d85456

…e' into danegsta-v2-container-execution

Base automatically changed from danegsta-logical-process-resource to main November 4, 2026 18:53

David Negstad (danegsta) requested a balanced review from Copilot November 4, 2026 20:23

Copilot started reviewing on behalf of David Negstad (danegsta) November 4, 2026 20:24

View session

Copilot AI reviewed Nov 4, 2026

View reviewed changes

Comment thread controllers/physical_container_exec_controller.go Resolved

Karol Zadora-Przylecki (karolz-ms) reviewed Nov 4, 2026

View reviewed changes

Comment thread controllers/physical_container_exec_controller.go Resolved

Karol Zadora-Przylecki (karolz-ms) reviewed Nov 4, 2026

View reviewed changes

David Negstad (danegsta) and others added 7 commits November 4, 2026 14:16

Record shared Get telemetry for container exec reconcilers

9a97179

Count successful, failed, and NotFound initial resource reads consistently with the other controllers. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 89db152a-aa9d-4e1d-9058-d4c0090c88d6

Karol Zadora-Przylecki (karolz-ms) approved these changes Nov 4, 2026

View reviewed changes

David Negstad (danegsta) requested a balanced review from Copilot November 4, 2026 23:43

Copilot started reviewing on behalf of David Negstad (danegsta) November 4, 2026 23:43

View session

Copilot AI reviewed Nov 4, 2026

View reviewed changes

David Negstad (danegsta) enabled stacked auto-merge (squash) November 5, 2026 00:00

David Negstad (danegsta) disabled stacked auto-merge November 5, 2026 00:15

David Negstad (danegsta) merged commit e1c3eb6 into main Nov 5, 2026
13 checks passed

David Negstad (danegsta) deleted the danegsta-v2-container-execution branch November 5, 2026 00:15

Navigation Menu

Repository navigation

Add V2 logical and physical container execution - #309

Add V2 logical and physical container execution#309
David Negstad (danegsta) merged 166 commits into
mainfrom
danegsta-v2-container-execution

David Negstad (danegsta) commented Oct 20, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Karol Zadora-Przylecki (karolz-ms) left a comment

Uh oh!

Uh oh!

Karol Zadora-Przylecki (karolz-ms) left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

David Negstad (danegsta) commented Oct 20, 2026

Scope and boundaries

Validation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Karol Zadora-Przylecki (karolz-ms) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Karol Zadora-Przylecki (karolz-ms) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants