Skip to content

[ISSUE #8072] Harden container distributions and document the Kubernetes runtime contract - #8107

Merged
Aias00 merged 1 commit into
apache:masterfrom
klT45:infra/container-runtime-hardening
Oct 26, 2026
Merged

[ISSUE #8072] Harden container distributions and document the Kubernetes runtime contract#8107
Aias00 merged 1 commit into
apache:masterfrom
klT45:infra/container-runtime-hardening

Conversation

@klT45

@klT45 klT45 commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Purpose

Harden the shenyu-admin / shenyu-bootstrap distribution images and document the runtime contract they now guarantee, closing the remaining acceptance items of #8072.

Changes

Images

  • Runtime base: eclipse-temurin:17-centos7 (bootstrap, EOL CentOS 7) and amazoncorretto:17.0.11-alpine3.19 (admin) → amazoncorretto:17.0.21-alpine3.24 for both, amd64 + arm64; prepare stage on alpine:3.24.
  • Fixed numeric non-root user (USER 10001:10001) so runAsUser/runAsNonRoot in Kubernetes manifests are stable and verifiable.
  • STOPSIGNAL SIGTERM and a HEALTHCHECK on /actuator/health (bootstrap start-period 60s, admin 90s). wget is installed explicitly for the healthcheck; the admin image also keeps curl for operators.
  • Entrypoints rewritten as POSIX sh (no bash in Alpine images): set -eu, exec the JVM so SIGTERM reaches Spring Boot shutdown hooks (exit 143), TMPDIR-aware java.io.tmpdir, unchanged default BOOT_JVM/ADMIN_JVM strings.
  • Read-only root filesystem supported: /tmp is the only writable path by default; conf-ext overrides are copied when conf is writable and loaded from the classpath when it is not (a partially failed copy also falls back to the classpath with a warning).

Docs

  • New docs/container-runtime.md: base-image update policy, filesystem/signal/exit-code contract, JVM and container memory budgets, production Kubernetes Deployment/PDB examples for both distributions with the correct 9195/9095 ports, admin DB profile + required SHENYU_JWT_SECRETKEY, readiness/liveness probe wiring (/actuator/health/readiness|liveness, management.endpoint.health.probes.enabled, shenyu.sync.websocket.initial-sync-readiness), upgrade/rollback guidance (including the whole-pod-template caveat), measured baselines, SBOM/scanning/signing guidance, and local verification steps.

Verification tooling

  • New script/docker-image-smoke.sh (stub distributions by default, SHENYU_SMOKE_REAL_DIST=1 for real tarballs) and a make smoke-image target.
  • New .github/workflows/docker-image-smoke.yml runs the stub smoke test on PRs touching the dist Dockerfiles/entrypoints, assembly descriptors, Makefile, the script or the guide.
  • Opt-in cross-version rollback check: SHENYU_SMOKE_PREVIOUS_TAG=2.7.1 pulls the previous published release and verifies it still starts/stops gracefully with the same mounted configuration.
  • integrated-test.yml pre-pulls the new base tags; the compose example healthcheck no longer writes a file into the container; .gitattributes keeps the container scripts on LF; fixed the pre-existing BOOSTRAP_REPOSITORY typo in the Makefile publish target.

Verification

  • bash script/docker-image-smoke.sh (stub): all checks pass (UID 10001, read-only rootfs, /tmp, declared HEALTHCHECK, conf-ext classpath fallback + legacy copy, BOOT_JVM/ADMIN_JVM, SIGTERM exit 143, restart with the same mounted configuration).
  • SHENYU_SMOKE_REAL_DIST=1 against the locally built 2.7.2-SNAPSHOT tarballs (Docker 29, linux/amd64, 4 vCPU):
    • bootstrap image 745,138,547 bytes, /actuator/health UP after ~11s, Docker HEALTHCHECK healthy after ~11s, graceful SIGTERM (143);
    • admin image 751,105,046 bytes, /actuator/health UP after ~21s, Docker HEALTHCHECK healthy after ~22s, graceful SIGTERM (143).
  • SHENYU_SMOKE_PREVIOUS_TAG=2.7.1 (published release): apache/shenyu-bootstrap:2.7.1 and apache/shenyu-admin:2.7.1 both start and stop gracefully with the same conf-ext mount under their own (writable rootfs) contract.
  • ./mvnw -Prelease package for shenyu-dist/shenyu-bootstrap-dist,shenyu-dist/shenyu-admin-dist -am succeeds; bash -n passes for all touched shell scripts; docs YAML blocks and workflow/compose YAML parse.

Out of scope / follow-ups

Closes #8072

@klT45
klT45 force-pushed the infra/container-runtime-hardening branch 3 times, most recently from d9c9c7d to 7fac331 Compare October 26, 2026 05:25
@klT45
[ISSUE apache#8072] Harden container distributions and document the K…
360be28 
…ubernetes runtime contract

- Modernize the admin/bootstrap image bases: both images now use
  amazoncorretto:17.0.21-alpine3.24 (linux/amd64, linux/arm64) with an
  alpine:3.24 prepare stage; the bootstrap image no longer uses the EOL
  CentOS 7 base.
- Pin a fixed numeric non-root UID/GID (USER 10001:10001), declare
  STOPSIGNAL SIGTERM and a HEALTHCHECK against /actuator/health, and keep the
  runtime read-only friendly: /tmp is the only writable path by default. wget
  is installed explicitly in both images (admin also keeps curl).
- Rewrite both entrypoint.sh scripts as POSIX sh (the Alpine images do not
  ship bash), exec the JVM so SIGTERM reaches the Spring Boot shutdown hooks,
  support a read-only conf directory by falling back to a conf-ext classpath
  override (and to the classpath when a writable copy only partially
  succeeds), and keep the BOOT_JVM/ADMIN_JVM overrides intact.
- Add docs/container-runtime.md: base-image update policy, runtime contract
  (user, writable paths, signals/exit codes), JVM and container memory
  budgets, Kubernetes requests/limits, readiness/liveness probe wiring
  (including management.endpoint.health.probes.enabled and
  initial-sync-readiness), Deployment/PDB examples for bootstrap and admin,
  upgrade/rollback procedure (including the whole-pod-template caveat), measured
  baseline image size/startup/memory numbers, and SBOM/scanning/signing
  guidance that stays within the ASF release process.
- Add script/docker-image-smoke.sh and .github/workflows/docker-image-smoke.yml:
  CI smoke test for UID 10001, read-only rootfs, /tmp writability, conf-ext
  overrides (read-only classpath fallback and legacy copy path),
  BOOT_JVM/ADMIN_JVM, SIGTERM graceful shutdown (exit code 0/143, not 137),
  restarts with the same mounted configuration and (real-dist mode) the
  declared HEALTHCHECK plus /actuator/health = UP.
  SHENYU_SMOKE_REAL_DIST=1 runs the same checks against real distributions, and
  SHENYU_SMOKE_PREVIOUS_TAG=<release> verifies a previous published release
  still starts and stops gracefully with the same mounts.
- Add a make smoke-image convenience target, refresh the integrated-test
  base-image pre-pull list, make the compose healthcheck read-only-rootfs
  safe, add .gitattributes so the container scripts stay on LF, and fix the
  pre-existing BOOSTRAP_REPOSITORY typo in the Makefile publish target.
@klT45
klT45 force-pushed the infra/container-runtime-hardening branch from 436b806 to 360be28 Compare October 26, 2026 05:32
Aias00
Aias00 approved these changes Oct 26, 2026
View reviewed changes

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. The container-hardening side of this PR is solid and self-contained: the new docker-image-smoke.yml is thoughtfully path-filtered with an explicit comment not to promote it to a required status check (so docs-only PRs never get blocked), the Dockerfile/entrypoint and docs/container-runtime.md changes are clean, and integrated-test.yml pinning alpine:3.24 / amazoncorretto:17.0.21-alpine3.24 while dropping EOL centos:7 is sensible. The InstanceCheckService.collectStateData() rework (per-namespace state history under historyLock, retaining/resetting namespaces) is also correct in isolation.

One coordination note: this PR's collectStateData() rework duplicates the per-namespace history change already in #7855 (issue #7695) - both add namespaceStateHistoryMap + historyLock and rewrite the same method - and #7971 also edits InstanceCheckService. All three touch the same file/method, so please coordinate a rebase so the namespace-history work lands exactly once (#7855 or this PR, not both) and the three are merged in an order that avoids a conflict on InstanceCheckService. The mergeable state currently shows "blocked", consistent with that overlap.

@klT45

klT45 commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review, @Aias00!

One clarification about the diff that was under review: an intermediate force-push briefly carried an unrelated local cherry-pick (the InstanceCheckService.collectStateData() per-namespace history rework, i.e. the #7855 area). It has been removed, and the current head 360be28e touches only container/CI/docs files:

  • shenyu-dist/shenyu-{admin,bootstrap}-dist/docker/Dockerfile + entrypoint.sh
  • docs/container-runtime.md, script/docker-image-smoke.sh
  • .github/workflows/docker-image-smoke.yml, .github/workflows/integrated-test.yml
  • Makefile, the docker-compose-dist file and .gitattributes

git diff --stat origin/master...HEAD confirms it (11 files). So this PR does not duplicate #7855 or conflict with #7971 on InstanceCheckService, and does not need the rebase coordination you mentioned - sorry for the confusing intermediate push.

The approval is recorded on 360be28e; if you want to double-check the corrected diff it would be appreciated. The remaining CI jobs (pr_build, the k8s gateway-api build) are still running.

@klT45

klT45 commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Quick nudge: the corrected head 360be28e is still container/CI/docs only (11 files), and all checks are green on it (pr_build, rat_check, check-license-header, smoke, k8s gateway-api build). @Aias00 mind taking a final look at the corrected diff so it can land? Thanks!

@Aias00
Aias00 merged commit e82768e into apache:master Oct 26, 2026
27 checks passed
@Aias00 Aias00 mentioned this pull request Oct 26, 2026
Merged
2 of 3 tasks
@Aias00 Aias00 added this to the 2.7.2 milestone Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Aias00 Aias00 Aias00 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

2.7.2

Development

Successfully merging this pull request may close these issues.

[Task][Cloud Native] Harden container distributions and validate production Kubernetes upgrade defaults

2 participants

@klT45 @Aias00