atelet: register the device plugin on a new socket each time - #3486

…ubstrate#3486)

Follow-up to agent-substrate#3323, as @SergeyKanzhelev suggested there. kubelet expects
a device plugin to switch to a new socket when it re-registers. Since
1.37 it rejects a Register for a socket it still has connected, and two
overlapping Registers for one socket can wedge the resource as "already
connected" until kubelet restarts:
kubernetes/kubernetes#139338

- the device plugin listens on a new `ate.dev-kvm-<random>.sock` for
each registration, and removes sockets left over from earlier atelets
- the device plugin stays up until atelet exits again. agent-substrate#3323 stopped it
on SIGTERM to avoid overlapping with the replacement pod, but the
DaemonSet only starts the replacement once the old pod is gone, so that
just left `ate.dev/kvm` unhealthy for the whole drain

<!-- e.g. "AI-assisted, I reviewed and tested it." No AI co-author
trailers. -->
AI use: I used Claude to draft this and reviewed it.

<!-- Breaking change? Describe what users must do on upgrade and ask for
the `breaking-change` label. -->

- [x] Tests pass
- [x] Docs updated

---------

Signed-off-by: Benjamin Elder <[email protected]>
(cherry picked from commit 38e9ef6)
Signed-off-by: Eitan Yarmush <[email protected]>

Bring `release-1.0` up to `main` at
5a5b192 with the 19 changes merged
since the previous batch in #3536. The final tree matches that commit
exactly.

**Please use Rebase and merge to preserve the individual commits and
author attribution.** All 19 applied cleanly in upstream order. Original
authors, author dates, co-author and sign-off trailers, and `-x` source
references are retained.

Upgrade notes:

- #3533: old gVisor MEMORY snapshots, including goldens, no longer
restore. Recreate them after upgrading. MicroVM MEMORY snapshots still
resume with their old mounts until recreated. `/tmp` is now always
tmpfs; ROOTFS snapshots no longer preserve its contents, and
image-provided files there are hidden. MicroVM `/tmp` counts against
actor memory and is capped at half the guest's memory. gVisor `/dev/shm`
is now a 64 MiB tmpfs. Mount a `durableDir` at either path when
persistence or more space is needed. `/run` now comes from the image
rootfs.
- #3447: remove `--actor-jwt-jwks-file` from custom credential-exchange
sidecar configs. Set `--actor-jwt-issuer-ca-file` only when the issuer
uses a private CA other than the service DNS CA.

- #3450: do not reuse a deleted atespace name. See the updated atespace
name-reuse guidance.

Includes #3505, which overlaps the open backport in #3547. GKE
provisioning and teardown now live in
[substrate-gke](https://github.com/ai-on-gke/substrate-gke) rather than
this repository.

<details>
<summary>Included PRs, in cherry-pick order</summary>

1. #3500: demos/{sandbox,jupyter,multi-template}: fix tutorial
prerequisites, walkthroughs, and counter volume
2. #3498: demos/counter: inline gVisor walkthrough and fix setup
commands
3. #3511: e2e tests: Extend the existing tests to cover tag actor.
4. #2775: Generate API reference documentation and publish it on
Substrate website
5. #3119: Document how people, pods and CI jobs should authenticate to
ate-api
6. #3505: tools: remove setup-gcp and hack/teardown.sh
7. #3147: e2e: resume fidelity after SandboxConfig version edits
8. #3370: ate-setup: reconcile ate-api-server ConfigMaps and roll out on
config changes
9. #1607: hack: reject breaking proto changes across each proto package
10. #3415: docs: update and add security diagrams
11. #3485: Resuming a microVM actor on the same node no longer reads its
rootfs upper back from disk
12. #3447: Verify actor JWTs in the credential exchange through OIDC
discovery
13. #3533: ocispec: mount tmpfs at /dev/shm, and /tmp on both runtimes
14. #3450: docs: Document that atespace names must not be reused
15. #3410: Document using more than one egress credential provider
16. #3486: atelet: register the device plugin on a new socket each time
17. #3553: ate-setup: wait for the demo namespace in delete demo
18. #3560: Fix multiple issues in claude-code-multiplex demo
19. #3517: docs: remove inaccurate early-stage architecture
documentation

</details>

Validation: every intermediate tree matches its upstream source commit,
all 19 authorship records and original attribution trailers are
preserved, and `git diff --check` passes. The original 18 backport
commits are unchanged.

AI use: AI-assisted cherry-picking, attribution checks, and review.

- [ ] Tests pass
- [x] Docs updated

---------

Signed-off-by: Mofi Rahman <[email protected]>
Signed-off-by: Eitan Yarmush <[email protected]>
Signed-off-by: Nayana Bidari <[email protected]>
Signed-off-by: Julian Gutierrez Oschmann <[email protected]>
Signed-off-by: Max Thompson <[email protected]>
Signed-off-by: Steven Shriver <[email protected]>
Signed-off-by: Grant McCloskey <[email protected]>
Signed-off-by: Shrutiya <[email protected]>
Signed-off-by: Haiyan Meng <[email protected]>
Signed-off-by: Greg Castle <[email protected]>
Signed-off-by: Benjamin Elder <[email protected]>
Signed-off-by: Anish Ramasekar <[email protected]>
Signed-off-by: Shruti Nair <[email protected]>
Signed-off-by: Haven Xia <[email protected]>
Co-authored-by: Mofi Rahman <[email protected]>
Co-authored-by: nybidari <[email protected]>
Co-authored-by: Julian Gutierrez Oschmann <[email protected]>
Co-authored-by: Max Thompson <[email protected]>
Co-authored-by: Grant McCloskey <[email protected]>
Co-authored-by: Steven Shriver <[email protected]>
Co-authored-by: shrutiyam-glitch <[email protected]>
Co-authored-by: haiyanmeng <[email protected]>
Co-authored-by: Greg Castle <[email protected]>
Co-authored-by: Benjamin Elder <[email protected]>
Co-authored-by: Anish Ramasekar <[email protected]>
Co-authored-by: Shruti Nair <[email protected]>
Co-authored-by: Haven Xia <[email protected]>
Co-authored-by: Adam Wathieu <[email protected]>

Follow-up to #3323, as @SergeyKanzhelev suggested there. kubelet expects
a device plugin to switch to a new socket when it re-registers. Since
1.37 it rejects a Register for a socket it still has connected, and two
overlapping Registers for one socket can wedge the resource as "already
connected" until kubelet restarts:
kubernetes/kubernetes#139338

- the device plugin listens on a new `ate.dev-kvm-<random>.sock` for
each registration, and removes sockets left over from earlier atelets
- the device plugin stays up until atelet exits again. #3323 stopped it
on SIGTERM to avoid overlapping with the replacement pod, but the
DaemonSet only starts the replacement once the old pod is gone, so that
just left `ate.dev/kvm` unhealthy for the whole drain

<!-- e.g. "AI-assisted, I reviewed and tested it." No AI co-author
trailers. -->
AI use: I used Claude to draft this and reviewed it.

<!-- Breaking change? Describe what users must do on upgrade and ask for
the `breaking-change` label. -->

- [x] Tests pass
- [x] Docs updated

---------

Signed-off-by: Benjamin Elder <[email protected]>
(cherry picked from commit 38e9ef6)
Signed-off-by: Eitan Yarmush <[email protected]>