Skip to content

Add V2 services and service-backed logical ports - #349

Merged
David Negstad (danegsta) merged 218 commits into
mainfrom
danegsta-v2-services-and-endpoints
Nov 7, 2026
Merged

David Negstad (danegsta) merged 218 commits into
mainfrom
danegsta-v2-services-and-endpoints

Conversation

@danegsta

@danegsta David Negstad (danegsta) commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Problem

V2 logical workloads need explicit relationships between their exposed ports and services. V1's single effective address cannot describe all bindings of a dual-stack service, and its annotation-based registration leaves the port/service relationship implicit.

Solution

Add namespaced LogicalService and ServiceEndpoint resources. DCP-managed services publish multiple effective address/port pairs and retain V1 proxy and proxyless behavior. Services have no lifecycle mode or predefined-address configuration; a separate resource for externally backed services is deferred to a follow-up PR.

Logical processes and containers declare named spec.ports, each with either a service reference or an inline service template. Templates create a predictably named service if absent and reuse matching services without taking ownership or overwriting them. Services survive logical workload replacement; endpoints remain tied to one workload UID. Endpoint cleanup begins immediately on logical deletion, independently of backing cleanup, and terminal workloads withdraw their endpoints.

Logical arguments and environment values support named-port serving templates and service-reference consumer templates, including address-family selection. Hashing and physical creation use the same evaluated launch values, which remain pinned through child handoff and ambiguous-create recovery. Transient evaluation dependencies keep preparation Pending; fatal evaluation errors are terminal and a later Stop does not reopen preparation. Lifecycle keys retain V1 boundaries: service identity and proxy policy do not independently affect reuse, while resolved container runtime port settings do. Explicit environment overrides are normalized before evaluation so overwritten templates cannot introduce dependencies or failures.

Shared TCP/UDP proxy backend connections are independent of frontend bindings, allowing IPv4/IPv6 forwarding in either direction. UDP backend sockets are connected to the selected peer, preserving existing UDP hostname resolution and rejecting replies from unrelated senders. Unchanged service routes are not reapplied, preserving established UDP streams while listener-health checks continue.

API scope

  • V2 logical container ports move from spec.container.ports to the unified spec.ports collection; process ports use the same service-backed model.
  • Persistent and existing logical processes require explicit nonzero serving ports, regardless of lifecycle-key configuration. Session processes retain dynamic allocation.
  • Physical resource schemas are unchanged. Service preparation stays within the existing logical pending/preparation handlers and uses uncached confirmation for ambiguous creation outcomes.
  • No service modes, predefined-address service support, service persistence, terminal support, or changes to V1 template semantics are introduced.

Validation

  • Native builds, scoped lint, and API generation passed.
  • Relevant API, controller, networking, template, and cleanup unit suites passed with race detection. Real-API integration selections with simulated backends cover routing, templates, workload replacement, early and terminal endpoint withdrawal, port validation, stop recovery, and pending-versus-terminal preparation behavior.
  • Real local socket tests exercised all four frontend/backend family combinations and hostname backends for both TCP and UDP, plus unrelated UDP reply rejection and stream retention across unchanged reconciliation. IPv6 cases ran without skipping.
  • A session-only IPv4-only capability simulation passed template/networking coverage and dependency-readiness integration; formatting-only IPv6 coverage remains active.
  • No Linux builds, true-container-runtime integration tests, or E2E tests were run locally. Tests requiring intentionally unbuilt Linux helper artifacts remain for CI. A broader integration selection was run at reduced concurrency after host localhost resolver timeouts.

Dependency

Targets main after parent #309 squash merged. Native stack 299 is preserved, including its merged history and downstream draft #350.

Introduce logical container and network APIs, delegate logical container lifecycle operations to physical resources, and track persistent workload ownership. Standardize V2 deletion dispatch and terminal startup/stop outcomes with shared API-backed coverage.

Co-authored-by: Copilot App <[email protected]>
Exercise logical readiness through the shared API server and running physical controller using runtime port-mapping fault injection. Remove redundant private deletion-handler tests already covered by retained-resource integration cases.

Co-authored-by: Copilot App <[email protected]>
Delegate runtime network realization to physical resources, manage persistent lifecycle keys and shared leases, and translate logical network attachments for new containers without changing adoption behavior.

Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
Bring the feature branch onto the base used by PR CI, including the upstream WSLC runtime changes. Update three runtime-label test callers to the map contract while preserving slice-valued API labels.

Co-authored-by: Copilot App <[email protected]>
Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
Use stable key sorting for logical container environment, container labels, and build labels. Resolve namespace workload overrides during asynchronous persistent ownership settlement, retaining startup leases on lookup or recording errors and supporting overrides without a global default.

Co-authored-by: Copilot App <[email protected]>
Resolve workload annotation overrides during persistent network lease settlement, retaining leases on namespace lookup failures, and preserve duplicate label precedence in lifecycle keys.

Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
Preserve all build directives selected by the daily CLI template instead of requiring an explicit SDK directive. Leave generated AppHost validation to Aspire, supporting package-based templates without importing the unrelated watcher or dependency changes from PR #292.

Co-authored-by: Copilot App <[email protected]>
Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
Make the selected runtime ID explicitly pending and clear it together with the generated session name after physical creation or verification. Project inline image references from the physical creation spec instead of retaining an adopted runtime ID as a marker.

Co-authored-by: Copilot App <[email protected]>
Retain the runtime ID only as a pending adoption input and clear it once the physical resource owns the selected identity.

Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
Replace per-controller ownership permutations and private environments with one persisted-ownership scenario in the shared real API/controller environment.

Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
Preserve the parent pending-input lifetime and inline-image projection changes alongside child network realization documentation.

Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
… refs

Use logical network readiness and its published physical reference instead of duplicating physical lookups. Create inline image resources before waiting for networks so image preparation can proceed independently.

Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
Use shared setup for scoped lifecycle and ownership scenarios, consolidate deletion ordering coverage, move duplicate-label sorting regression to API tests, and remove unrelated cancellation and runtime retry checks.

Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
Delegate process execution to identity-bound physical resources, with session, persistent, and existing lifecycle modes. Record retained process ownership before releasing startup leases, including deletion during launch. Treat process launch errors as terminal.

Co-authored-by: Copilot App <[email protected]>
Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9
Add scoped resource-stream nesting that preserves context, primary bindings, filters, and independent cleanup. Bind V2 container diagnostics by resource UID and forward owned physical container diagnostics to the logical owner's stream, including asynchronous operation failures. Keep resource-file producer names per entry and verify fanout without duplicate process or resource entries. This provides capture and routing only; V2 log subresource registration remains separate.

Co-authored-by: Copilot App <[email protected]>
Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
Keep startup settlement and ownership recording in the logical controller before requesting physical deletion. Remove the startup finalizer handshake, deletion-time physical status projection, and the guard introduced to support them.

Co-authored-by: Copilot App <[email protected]>
Copilot-Session: aa4abbe1-3392-4398-9f96-566f68ce0fa9
Share watch-driven log storage and query handling across API versions while preserving V1 contracts. Add logical and physical container stdout, stderr, and system streamers with one physical-UID runtime capture, snapshot-only logical source resolution, independent client lifetimes, namespace preservation, and explicit unsupported-source errors. Project backing physical identity into logical status and exercise actual endpoints, cancellation, and existing V1 log behavior.

Co-authored-by: Copilot App <[email protected]>
Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
Replace per-kind cleanup handlers with one uncached list and bounded delete executor, preserving dependency ordering and pending-resource diagnostics.

Co-authored-by: Copilot App <[email protected]>
Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
Declare logical and physical container log subresources alongside their parents, retaining only resource-neutral storage construction and contracts in shared log types.

Co-authored-by: Copilot App <[email protected]>
Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
Capture pull and build output, serialize image acquisition with exclusive leases, and project image startup and container runtime sources directly onto logical resources. Preserve the V1 watcher handoff, fix nested resource sink failure reporting, and share CaptureFile between V1 and V2 controllers.

Co-authored-by: Copilot App <[email protected]>
Preserve image-log source projection before waiting on published logical network readiness.

Co-authored-by: Copilot App <[email protected]>

Copilot-Session: bbe01c56-5ae4-4f36-8134-93b3b2718fd2
Comment thread controllers/logical_ports.go Resolved
Comment thread internal/resourcecleanup/resources.go Resolved

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings, de-duplicated against the existing review threads. Verified at head 0a271ab: build, lint and tests pass on macOS, and the proxy/networking/templating/API unit suites plus the new V2 service integration tests pass on Linux (Podman). Repro details for each finding are in the inline comments.

Comment thread internal/proxy/netproxy.go Resolved
Comment thread controllers/logical_ports.go Resolved
Comment thread controllers/logical_ports.go Resolved
Comment thread controllers/logical_ports.go Resolved
Comment thread internal/proxy/netproxy.go Resolved
Validate endpoint updates consistently with immutable V2 services and replace changed workload bindings through deletion and creation. Cover real-API update/patch rejection, finalizer cleanup, endpoint replacement identity, and validation hooks across persisted V2 resources.

Co-authored-by: Copilot App <[email protected]>
Separate creation, running, stopping, terminal and unknown handling across logical and physical controllers. Preserve identity guards and ambiguous API-create confirmation while removing generation gates, inline phase handoffs, and result-handler recovery. Withdraw endpoints independently of stop completion, synchronize the complete owned endpoint set, and retain bounded worker retries without polling image readiness inside reconciliation.

Update lifecycle regressions and documentation. Full make lint passes on Windows, macOS and Linux; bounded native simulated real-API integration selections and controller unit races pass. Full test-suite execution remains for CI.

Co-authored-by: Copilot App <[email protected]>
Define typed reason constants beside the relevant APIs, share common logical-resource and execution reasons, and preserve existing status strings across handlers, projections, and tests. Restore Stop transitions before backing lookup, keep missing pinned image backings Unknown, and add the physical container Resolve projection. Align backing/log assertions with the approved Unknown and diagnostic contracts and extend lookup-failure regression coverage.

Co-authored-by: Copilot App <[email protected]>
Preserve desired lifecycle keys in logical status while adopting retained runtime identities for Stop. Keep persistent process record keys unchanged during adoption and cover mismatched keys and pending replacement handoff.

Co-authored-by: Copilot App <[email protected]>
Stop polling terminal services and retain their startup failure diagnostics. Release terminal logical startup leases independently of backing observations, dispatch preparation failure through terminal handling, and centralize owner-scoped container/image identity lookups.

Extend existing terminal regressions and remove the timing-dependent test that required an intermediate Pending-with-ID snapshot.

Co-authored-by: Copilot App <[email protected]>
Retain effective launch values and handed-off port selections instead of whole configuration copies. Recover existing children before reevaluating inputs, reuse PersistentProcessRecord, and remove redundant image/service/network fields.

Return preparation errors explicitly, classify terminating or failed services through established preparation-failed states, share logical-to-physical container configuration mapping, and synchronize deletion tests on committed work rather than a reader-specific hook.

Co-authored-by: Copilot App <[email protected]>
Defer ambiguous API write recovery to reconciliation, validate exact AlreadyExists collisions, and fail incompatible service templates terminally.

Co-authored-by: Copilot App <[email protected]>
Order namespace endpoint cleanup after logical processes and containers so live owners do not attempt to recreate withdrawn endpoints.

Co-authored-by: Copilot App <[email protected]>
Filter redundant endpoint and namespace updates, honor canceled reconciliation contexts, and record standard Get outcome counters.

Co-authored-by: Copilot App <[email protected]>
Retain cross-family UDP flows across backend connection-refused errors with verbose diagnostics. Preserve V1 bracketed IPv6 representation through V2 service publication, endpoint derivation, and templates, using the shared formatter without capability-dependent formatting.

Co-authored-by: Copilot App <[email protected]>
Keep initial startup Pending until endpoints and required container port mappings are available. Log and retry subsequent endpoint failures without changing workload lifecycle/readiness or classifying them as physical failures.

Co-authored-by: Copilot App <[email protected]>
Cover address normalization, port/protocol filtering, unpublished mappings, ordering and deduplication. Exercise existing-mode process and container endpoint publication, ownership, withdrawal, and retained runtime/service lifetimes.

Co-authored-by: Copilot App <[email protected]>
Match explicit host ports directly and assign remaining distinct host ports numerically to dynamic declarations in spec order per container-port/protocol pair. Preserve all addresses per selected binding and explicit aliases, with documentation and unit/integration coverage.

Co-authored-by: Copilot App <[email protected]>
Flatten physical volumes into one create-or-reuse contract, remove replacement and reference-only options, and retain explicit cleanup policy. Preserve physical log completion independently of logical lifecycle, including output drain after exit.

Co-authored-by: Copilot App <[email protected]>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings at head ee9ffa0, de-duplicated against the existing (all resolved) review threads. Most of them are in code that landed after the previous review pass (state-specific handlers, volume simplification, endpoint publication gating). Build/unit checks for api/v2, proxy, networking and templating pass; generated code regenerates cleanly.

Comment thread controllers/logical_ports.go Resolved
Comment thread controllers/logical_ports.go Resolved
Comment thread api/v2/container_exec_types.go Resolved
Register lifetime cancellation as the earliest cleanup hook so namespace cleanup finishes before API server shutdown.

Co-authored-by: Copilot App <[email protected]>
@danegsta
David Negstad (danegsta) merged commit c5edc40 into main Nov 7, 2026
12 checks passed
@danegsta
David Negstad (danegsta) deleted the danegsta-v2-services-and-endpoints branch November 7, 2026 06:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants