[release-1.0] Cherry-pick #2392: Run containers as the image's USER and WORKDIR, with layer file owners kept - #3473

…s kept (agent-substrate#2392)

Containers ignored the image's `USER` and `WORKDIR` and ran as root in
`/`, and atelet's unpack dropped the file owners image layers record. So
an image that hands a directory to its user, such as jupyter's
`/home/jovyan`, didn't work as built.

- atelet applies each tar entry's uid/gid and keeps setuid, setgid and
sticky bits. It adds `CHOWN`, `FOWNER` and `FSETID` to its
`DAC_OVERRIDE`. The image cache layout moves to version 3.
- The process runs as the image's `USER`, resolved as Docker does
against the image's own `/etc/passwd` and `/etc/group`, and starts in
its `WORKDIR` (default `/`).
- The jupyter demo runs as jovyan and drops `DAC_OVERRIDE` and
`--allow-root`.

The `USER` and `WORKDIR` commits are @dims's from agent-substrate#1918, kept as
written. This PR supersedes agent-substrate#1918.

Fixes agent-substrate#2036

#### Breaking change

Please add the `breaking-change` label.

- After an upgrade or rollback, atelet refuses the existing image cache.
Delete `/var/lib/ate/image-cache` on each node once no actor runs on it,
as atelet's error says. After an upgrade, a node reboot also works.
- A custom atelet securityContext needs `CHOWN`, `DAC_OVERRIDE`,
`FOWNER` and `FSETID`.
- Images with a `USER` stop running as root, and images with a `WORKDIR`
stop starting in `/`. A non-root `USER` is expected to have no effective
capabilities, so `securityContext.capabilities.add` no longer helps it.
This is checked under runc and in the gVisor source, not yet inside a
gVisor or microVM sandbox.
- gVisor snapshots of such images don't restore, because `runsc restore`
checks each container's user and working directory. Recreate those
templates. MicroVM snapshots resume as they were.
- Files an earlier run wrote as root into a durable dir stay root-owned.

Tested: `go test -race` on atelet, imagecache, ocispec, tarutil, kata
and the probe fixture, on macOS and on Linux as non-root, and imagecache
with `-count=3 -shuffle=on`. The root-only imagecache and tarutil tests
pass as root on a privileged Linux container. `hack/verify-all.sh`
passes. The jupyter image serves `/api` under Docker as 1000:100 with
every capability dropped. @dims ran the jupyter demo and a non-root node
image as their users on kind, on both runtimes, before agent-substrate#1904 merged; on
microVM uid 1000 couldn't bind :80 then. A new imagefs e2e starts an
actor from a probe image with `USER nonroot:nonroot` and `WORKDIR
/home/nonroot`. It checks uid and gid 65532, no supplementary group
besides 65532, no effective capabilities, the working directory, and
that the user can create a file there. CI runs it on gVisor and on
microVM. At this head nothing ran in a gVisor or microVM sandbox yet:
not the jupyter demo and not the imagefs e2e.

- [ ] Tests pass
- [x] Docs updated

---------

Signed-off-by: Anish Ramasekar <[email protected]>
Signed-off-by: Davanum Srinivas <[email protected]>
Co-authored-by: Davanum Srinivas <[email protected]>
(cherry picked from commit 5d226a5)
Signed-off-by: Benjamin Elder <[email protected]>