Repository navigation
remove non mitm dataplanes - #2015
Lior Lieberman (LiorLieberman) merged 5 commits into
Conversation
0a635c1 to
b15d2fc
Compare
Eitan Yarmush (EItanya)
left a comment
There was a problem hiding this comment.
remove non mitm dataplanes, add caching for ci
Can we please split this into 2 PRs? It would make it much much easier to review
I saw that coming :-), let me do it. |
b15d2fc to
1200ecc
Compare
1200ecc to
3612c1b
Compare
| f.StringVar(&opts.AdditionalEgressExtprocService, "experimental-additional-egress-extproc-service", "", "Run an additional ext_proc authorization filter served by NS/SVC:PORT (requires --experimental-use-sdsmint)") | ||
| f.BoolVar(&opts.ExperimentalEgressCredentialInjection, "experimental-egress-credential-injection", false, "Point the egress gateway's MITM-leg handler at a credential provider so a matching EgressPolicy rule injects its credential (requires --experimental-use-sdsmint and --atenet-dataplane=envoy)") | ||
| f.StringVar(&opts.AdditionalEgressExtprocService, "experimental-additional-egress-extproc-service", "", "Run an additional ext_proc authorization filter served by NS/SVC:PORT (requires --atenet-dataplane=envoy)") | ||
| f.BoolVar(&opts.ExperimentalEgressCredentialInjection, "experimental-egress-credential-injection", false, "Point the egress gateway's MITM-leg handler at a credential provider so a matching EgressPolicy rule injects its credential (requires --atenet-dataplane=envoy)") |
There was a problem hiding this comment.
--experimental-egress-credential-injection will be removed in a follow up
3612c1b to
e7b2d35
Compare
| echo " the gateway is deployed (e.g. with --deploy-atenet); the credential provider" | ||
| echo " itself is deployed separately. Implies --experimental-use-sdsmint; requires" | ||
| echo " --atenet-dataplane=envoy. (experimental)" | ||
| echo " itself is deployed separately. Requires --atenet-dataplane=envoy." |
There was a problem hiding this comment.
Eitan Yarmush (@EItanya) to change in a follow up
| // hack/install-ate-kind.sh --deploy-atenet --experimental-egress-credential-injection | ||
| // E2E_EGRESS_CREDINJECT=1 hack/run-e2e-kind.sh ./internal/e2e/suites/egresscredinject -v -args --no-color | ||
| func TestActorEgressCredentialInjection(t *testing.T) { | ||
| if os.Getenv("E2E_EGRESS_CREDINJECT") == "" { |
There was a problem hiding this comment.
will be removed in a follow up
Bowei Du (bowei)
left a comment
There was a problem hiding this comment.
let's get this in and fix-up any nit comments after as it's so big.
|
oh no, conflicts... we gotta solve the runner issue, its impossible to work like that |
The agentgateway dataplane had two egress overlays: agentgateway-egress by default, which relayed tunnelled TLS untouched and could decide it by address only, and agentgateway-egress-mitm behind --experimental-use-sdsmint, which mints a per-SNI leaf, reads every request and decides it by name. Hostname rules for HTTPS only ever worked on the second. It is now the only one: --atenet-dataplane=agentgateway installs it with or without the flag, and the installer creates the egress-mitm-ca-pool it mounts for that dataplane unconditionally. The flag now selects only the envoy gateway. The remaining overlay is rebased onto atenet-egress-with-sdsmint.yaml and its component strips what agentgateway does not run: the sdsmint sidecar, its volumes and the non-root pod security context they need. The Service gains the PreferDualStack the plain manifest had. Rendered, the Deployment, Service and ConfigMaps are byte-identical to what the overlay produced on the plain manifest; only the envoy bootstrap ConfigMap, which agentgateway never reads, differs. Nothing on the agentgateway side depends on atenet-egress.yaml any more. The CI step that swapped in the minting gateway after the standard lanes now runs only in the envoy lane; the agentgateway lane has had it since install, and the standard lanes do no TLS through the gateway unless E2E_EGRESS_MITM is set, so their order is unchanged.
… only one Two envoy egress gateways could be installed: the plain one by default, which forwarded tunnelled TLS untouched and so could decide it by address only, and behind --experimental-use-sdsmint the terminating one, which mints a per-SNI leaf, reads every request and decides it by name. Hostname rules for HTTPS and credential injection only ever existed on the second, and every deployment that wanted them had to opt in. With the agentgateway egress already always minting, the terminating gateway is now the only one on both dataplanes, and the flag is gone from ate-setup, the shell shim and the benchmark automation. Envoy keeps atenet-egress-with-sdsmint.yaml; atenet-egress.yaml is deleted. The remaining manifest gains the dual-stack sockets the plain one got in 5012abf, which left the sdsmint variant IPv4-only. The e2e job installs the gateway once, with credential injection enabled on envoy through ATE_CREDENTIAL_INJECTION_ENABLED, instead of redeploying it for six more lanes. The identity suite now runs last, on its own: it replaces the egress CA pool to prove the projected trust bundle follows it, and the gateway picks a rotation up only once kubelet has propagated the Secret into its pod, so for a while after each rotation it signs with a CA no actor trusts. That never mattered while the plain gateway ignored the pool and the sdsmint lanes ran after the rotations were over; with one gateway it would fail whatever TLS test ran alongside. E2E_EGRESS_MITM selected the gateway variant for the networking and egressmitm suites and is gone with it; the tests it gated always run. The shell twin of the injection splice, hack/experimental-egress-credential-injection.sh, has been sourced by nothing since install-ate.sh became a shim, and is removed rather than updated.
…gress gateway With the plain gateway gone, "the sdsmint gateway", "the MITM gateway" and the -with-sdsmint and -mitm names distinguished nothing. The envoy manifest is atenet-egress.yaml again, the agentgateway overlay and its component are agentgateway-egress, and comments, docs and test names say "the egress gateway" or "the gateway". sdsmint stays where it is the thing being configured: its package, the sidecar container and its volumes in the envoy manifest, and the kustomize patch that removes them for agentgateway. The egress demos lose their -mitm variants the same way. The plain egress and egress-microvm demos trusted only public roots, which nothing behind the gateway chains to, so they could no longer do HTTPS; the -mitm variants, which project the gateway's trust bundle, are now the egress demos under the plain names, and the e2e fixtures and the CI deploy step follow. The egress-mitm-ca-pool Secret, the egress-mitm.ate.dev bundle and the EgressMITMTrust reconciler keep their names: they name the CA, not a variant.
422beee to
e33b2b1
Compare
e5a1a65
agent-substrate#2015 moved the identity suite into its own CI jobs. The test replaced the cluster-wide egress CA pool and caused actors stopped trusting the old CA right away, while gateways kept signing with the old ca until kubelet refreshed their Secret. This caused TLS TLS to fail for any suite running at the same time and therfore we splitted it. The suite now adds a fresh CA that never signs and restores the pool when it ends. Bundles are compared as the same PEM blocks in any order, because atelet shuffles bundles with more than one certificate. What the suite checks (exact file contents, live refresh, suspend/resume) is unchanged, and new unit tests check that the signing CA never changes. - [x] Tests pass - [x] Appropriate changes to documentation are included in the PR
Remove the plain (non-terminating) egress gateways on both dataplanes. The "mitm" gateway, which mints a per-SNI leaf and decides every request by name, is the only one, and we should stop call it mitm gateway.
--experimental-use-sdsmintis also gone.Highly recommend review each commit at a time;
-mitmdemos. More specifically for the this commit: Manifest isatenet-egress.yaml(its not again, the overlay and component areagentgateway-egress, prose says "the egress gateway". The-mitmegress demos become the egress demos under the plain names (the plain ones trusted only public roots and could not do HTTPS through the gateway); e2e fixtures and the CI deploy step follow.