Add V2 logical process lifecycle management and logging by danegsta · Pull Request #306 · microsoft/dcp

David Negstad (danegsta) · 2026-10-19T21:02:07Z

V2 already has a physical process execution resource, but no logical layer to own process lifetime policy and persistent reuse. This adds LogicalProcess, keeping launch, observation, stop, and cleanup in PhysicalProcess while the logical controller owns lifecycle keys, reuse, leases, and persistence.

Builds on the logical-container and logical-container-network layers now merged into main.

Behavior

Supports session, persistent, and existing modes. Persistent identity follows the logical namespace/name; matching live records are reused, stale records are discarded, and changed lifecycle keys delegate replacement through physical resources. Existing references require both PID and identity timestamp. Physical PID references accept an additive expected timestamp to reject PID reuse without changing existing PID-only callers.
Records persistent process identity and any configured workload association before releasing the startup lease. Unscoped reuse preserves an existing workload owner; a replacement process does not inherit the retired process's owner. Status/readiness project current physical observations rather than drive reconciliation.
Requests backing deletion immediately, including while launch or persistence settlement is pending. A shared logical hold keeps the physical API object available while its controller settles runtime cleanup or retention and durably reports the outcome. The logical controller then settles registration and its startup lease before releasing that hold. Namespace cleanup respects the logical-to-physical dependency.
Exposes logical and physical stdout, stderr, and system log endpoints. Physical resources own capture and forward owned diagnostics into the logical system stream; logical output streaming consumes only its projected immediate physical source snapshot. Output completion is projected during deletion so logical-only followers can drain without blocking capture cleanup.
Retained processes inherit direct file handles and use session-independent output storage. Persistent records preserve stdout/stderr paths, and subsequent reuse reattaches those captures through identity-bound physical references. Retired output is cleaned up after confirmed exit, honoring DCP_PRESERVE_EXECUTABLE_LOGS. Referenced processes without attached capture return completed-empty output.

Physical process launch errors are now terminal, matching V1 executable launch and physical-container start behavior. IDE/debug protocol support and a cleanup lifecycle mode are not included.

Validation

Generated API artifacts and built test prerequisites; passed affected package and simulated process/network/container-image log endpoint tests with race detection on macOS, focused capture/streamer/simulated process checks in Linux, and lint for Windows, macOS, and Linux. No local real-runtime or E2E suites were run.

Introduce logical container and network APIs, delegate logical container lifecycle operations to physical resources, and track persistent workload ownership. Standardize V2 deletion dispatch and terminal startup/stop outcomes with shared API-backed coverage. Co-authored-by: Copilot App <[email protected]>

Exercise logical readiness through the shared API server and running physical controller using runtime port-mapping fault injection. Remove redundant private deletion-handler tests already covered by retained-resource integration cases. Co-authored-by: Copilot App <[email protected]>

Delegate runtime network realization to physical resources, manage persistent lifecycle keys and shared leases, and translate logical network attachments for new containers without changing adoption behavior. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Bring the feature branch onto the base used by PR CI, including the upstream WSLC runtime changes. Update three runtime-label test callers to the map contract while preserving slice-valued API labels. Co-authored-by: Copilot App <[email protected]>

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Use stable key sorting for logical container environment, container labels, and build labels. Resolve namespace workload overrides during asynchronous persistent ownership settlement, retaining startup leases on lookup or recording errors and supporting overrides without a global default. Co-authored-by: Copilot App <[email protected]>

Resolve workload annotation overrides during persistent network lease settlement, retaining leases on namespace lookup failures, and preserve duplicate label precedence in lifecycle keys. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Preserve all build directives selected by the daily CLI template instead of requiring an explicit SDK directive. Leave generated AppHost validation to Aspire, supporting package-based templates without importing the unrelated watcher or dependency changes from PR #292. Co-authored-by: Copilot App <[email protected]>

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Make the selected runtime ID explicitly pending and clear it together with the generated session name after physical creation or verification. Project inline image references from the physical creation spec instead of retaining an adopted runtime ID as a marker. Co-authored-by: Copilot App <[email protected]>

Retain the runtime ID only as a pending adoption input and clear it once the physical resource owns the selected identity. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Replace per-controller ownership permutations and private environments with one persisted-ownership scenario in the shared real API/controller environment. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Preserve the parent pending-input lifetime and inline-image projection changes alongside child network realization documentation. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

… refs Use logical network readiness and its published physical reference instead of duplicating physical lookups. Create inline image resources before waiting for networks so image preparation can proceed independently. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Use shared setup for scoped lifecycle and ownership scenarios, consolidate deletion ordering coverage, move duplicate-label sorting regression to API tests, and remove unrelated cancellation and runtime retry checks. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Delegate process execution to identity-bound physical resources, with session, persistent, and existing lifecycle modes. Record retained process ownership before releasing startup leases, including deletion during launch. Treat process launch errors as terminal. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Add scoped resource-stream nesting that preserves context, primary bindings, filters, and independent cleanup. Bind V2 container diagnostics by resource UID and forward owned physical container diagnostics to the logical owner's stream, including asynchronous operation failures. Keep resource-file producer names per entry and verify fanout without duplicate process or resource entries. This provides capture and routing only; V2 log subresource registration remains separate. Co-authored-by: Copilot App <[email protected]>

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Keep startup settlement and ownership recording in the logical controller before requesting physical deletion. Remove the startup finalizer handshake, deletion-time physical status projection, and the guard introduced to support them. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Share watch-driven log storage and query handling across API versions while preserving V1 contracts. Add logical and physical container stdout, stderr, and system streamers with one physical-UID runtime capture, snapshot-only logical source resolution, independent client lifetimes, namespace preservation, and explicit unsupported-source errors. Project backing physical identity into logical status and exercise actual endpoints, cancellation, and existing V1 log behavior. Co-authored-by: Copilot App <[email protected]>

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Replace per-kind cleanup handlers with one uncached list and bounded delete executor, preserving dependency ordering and pending-resource diagnostics. Co-authored-by: Copilot App <[email protected]>

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Declare logical and physical container log subresources alongside their parents, retaining only resource-neutral storage construction and contracts in shared log types. Co-authored-by: Copilot App <[email protected]>

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Co-authored-by: Copilot App <[email protected]>

Capture pull and build output, serialize image acquisition with exclusive leases, and project image startup and container runtime sources directly onto logical resources. Preserve the V1 watcher handoff, fix nested resource sink failure reporting, and share CaptureFile between V1 and V2 controllers. Co-authored-by: Copilot App <[email protected]>

Preserve image-log source projection before waiting on published logical network readiness. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Encode selected process fields individually and normalize explicit environment declarations with platform-aware last-value-wins semantics. Preserve inheritance defaults, explicit keys and V1 hashing, with differential environment-equivalence coverage. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Share V1's persistent process output directory with retained V2 captures, and honor executable log preservation when disposing created process output. Cover session cleanup followed by adoption, preserved session and persistent output, and unchanged directory configuration/elevation behavior. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Copilot

🟡 Changes recommended

Critical defects can stall Windows cleanup when only a logical log follower is open and erase an adopted process’s existing workload owner when no workload ID resolves, excluding it from workload cleanup.

2 open issues

6 resolved since last review

2 files not reviewed

api/v2/zz_generated.deepcopy.go: Generated file
api/v2/zz_generated.model_name.go: Generated file

_{Review effort: Balanced}

Set the shared local/CI test timeout explicitly and keep the Windows CI driver aligned without changing per-test contexts or parallelism. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Continue projecting the verified physical backing snapshot during cleanup so logical-only followers can drain and release capture readers. Preserve completion changes across deletion returns and cover namespace deletion without starting a physical log watch. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Only replace an existing process workload association when the resolved workload ID is nonempty. Cover scoped-to-unscoped reuse and ownerless replacement in the existing integration regression. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Distinguish operating system processes from PhysicalProcess API resources in comments, documentation, and generated schema descriptions. No behavioral or public identifier changes. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Replace the inheritance boolean with V2-owned ambientEnvironment.behavior in both logical and physical process configurations. Preserve inheritance defaults, explicit environment precedence, and published lifecycle keys; validate supported behaviors and keep the generated API artifacts in sync. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Karol Zadora-Przylecki (karolz-ms)

Copilot-assisted review (non-approving): one reliability concern, one API-doc inconsistency (posted as two inline comments), and one low-confidence security question.

Cache bounded capture disposal with OnceValue and log output-removal failures without blocking process finalization, replacement, or retirement of confirmed-gone records. Update cleanup regressions and documentation, share the process key prefix, clarify log-stream completion and raw filtering, and distinguish missing process data from the API object. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Share a host-lifetime bounded cleanup queue across process and image reconcilers. Preserve capture-drain ordering and best-effort disposal, reuse CloseAndLog, and keep file removal independent of resource finalization and process operations. Cover blocked cleanup through the real API, await asynchronous removal in existing tests, and consolidate overlapping serial log coverage. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Document best-effort capture removal on resource deletion after process exit or confirmed absence, including PID reuse. Clarify the identity timestamp's use for subsequent persistent-process PID-reuse protection and retain the Linux since-boot distinction. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

Keep follower completion and recreated-capture isolation checks without waiting for guaranteed file removal with open Windows readers. Clean up leftover captures in test teardown after the followers close. Co-authored-by: Copilot App <[email protected]> Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9

David Negstad (danegsta) and others added 30 commits October 14, 2026 16:36

Merge parent main compatibility updates

091e3d6

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Merge parent namespace ownership fixes

bcc48a2

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Merge parent Aspire scaffold compatibility fix

368ba91

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Limit selected network identity to physical handoff

b9e93a4

Retain the runtime ID only as a pending adoption input and clear it once the physical resource owns the selected identity. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Merge parent logical container state lifetime cleanup

dc92663

Preserve the parent pending-input lifetime and inline-image projection changes alongside child network realization documentation. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Merge parent V2 container system logging

041aa39

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Merge parent V2 container log endpoints

e18a460

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Drive namespace cleanup directly from relational metadata

8fa462b

Replace per-kind cleanup handlers with one uncached list and bounded delete executor, preserving dependency ordering and pending-resource diagnostics. Co-authored-by: Copilot App <[email protected]>

Merge parent common namespace cleanup executor

46260ed

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Keep V2 container log subresources with their parent types

4c5e7ab

Declare logical and physical container log subresources alongside their parents, retaining only resource-neutral storage construction and contracts in shared log types. Co-authored-by: Copilot App <[email protected]>

Merge parent container log resource placement cleanup

54cdc9a

Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

Use container-specific filenames for V2 log streaming

2f6d400

Co-authored-by: Copilot App <[email protected]>

Merge main to adopt the upstream Aspire regression fix

7bfc3d4

Co-authored-by: Copilot App <[email protected]>

Merge parent image log capture and shared streaming contracts

bab8f59

Preserve image-log source projection before waiting on published logical network readiness. Co-authored-by: Copilot App <[email protected]> Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2

David Negstad (danegsta) and others added 2 commits November 3, 2026 11:33

David Negstad (danegsta) requested a balanced review from Copilot November 3, 2026 20:08

Copilot started reviewing on behalf of David Negstad (danegsta) November 3, 2026 20:10

View session

Copilot AI reviewed Nov 3, 2026

View reviewed changes

Comment thread controllers/logical_process_controller.go Resolved

Comment thread controllers/logical_process_controller.go Resolved

David Negstad (danegsta) and others added 3 commits November 3, 2026 12:28

David Negstad (danegsta) mentioned this pull request Nov 3, 2026

Add V2 services and service-backed logical ports #349

Merged

Karol Zadora-Przylecki (karolz-ms) reviewed Nov 3, 2026

View reviewed changes

Comment thread api/v2/logical_process_types.go Resolved

Karol Zadora-Przylecki (karolz-ms) reviewed Nov 3, 2026

View reviewed changes

Comment thread internal/dcp/commands/cleanup.go Resolved

Karol Zadora-Przylecki (karolz-ms) reviewed Nov 3, 2026

View reviewed changes

Comment thread internal/logs/v2resourcelogs/resource_logstreamer.go Resolved

Karol Zadora-Przylecki (karolz-ms) reviewed Nov 3, 2026

View reviewed changes

Comment thread internal/logs/v2resourcelogs/resource_logstreamer.go Resolved

Karol Zadora-Przylecki (karolz-ms) reviewed Nov 4, 2026

View reviewed changes

Comment thread controllers/physical_process_controller.go Resolved

Karol Zadora-Przylecki (karolz-ms) reviewed Nov 4, 2026

View reviewed changes

Comment thread internal/logs/capture_file.go Resolved

Karol Zadora-Przylecki (karolz-ms) reviewed Nov 4, 2026

View reviewed changes

Comment thread scripts/test-ci.ps1 Resolved

Karol Zadora-Przylecki (karolz-ms) reviewed Nov 4, 2026

View reviewed changes

Comment thread test/integration/v2_process_logs_test.go Resolved

Karol Zadora-Przylecki (karolz-ms) reviewed Nov 4, 2026

View reviewed changes

Comment thread Makefile Resolved

Karol Zadora-Przylecki (karolz-ms) reviewed Nov 4, 2026

View reviewed changes

Comment thread controllers/physical_process_controller.go Resolved

Comment thread api/v2/physical_process_types.go Resolved

Comment thread controllers/logical_process_controller.go Resolved

Comment thread internal/logs/v2resourcelogs/resource_logstreamer.go Resolved

Karol Zadora-Przylecki (karolz-ms) approved these changes Nov 4, 2026

View reviewed changes

David Negstad (danegsta) and others added 4 commits November 3, 2026 18:06

David Negstad (danegsta) enabled stacked auto-merge (squash) November 4, 2026 18:49

David Negstad (danegsta) disabled stacked auto-merge November 4, 2026 18:53

David Negstad (danegsta) merged commit 2d247fb into main Nov 4, 2026
12 checks passed

David Negstad (danegsta) deleted the danegsta-logical-process-resource branch November 4, 2026 18:53

Navigation Menu

Repository navigation

Add V2 logical process lifecycle management and logging - #306

Add V2 logical process lifecycle management and logging#306
David Negstad (danegsta) merged 125 commits into
mainfrom
danegsta-logical-process-resource

David Negstad (danegsta) commented Oct 19, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Karol Zadora-Przylecki (karolz-ms) left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

David Negstad (danegsta) commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Behavior

Validation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Karol Zadora-Przylecki (karolz-ms) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

David Negstad (danegsta) commented Oct 19, 2026 •

edited

Loading