feat: allow other istio gateways on ingress deployment#66
Merged
thesuperzapper merged 1 commit intoJan 27, 2024
Merged
Conversation
Signed-off-by: Mathew Wicks <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR completely changes how the EnvoyFilters are structured in deployKF, allowing users who want to host the deployKF Istio Gateway (CRD) on an existing Istio Ingress Gateway Deployment (Pods) can do so.
This change will require you to PRUNE sync in ArgoCD for the
deploykf-istio-gatewayapp.For reference, while you can now use an existing Ingress Gateway Deployment (Pods), deployKF will still manage its own virtual Gateway (CRD), as this is how we know which routes are part of deployKF (and thus need authentication).
Additionally, you can't use the same hostname (
deploykf_core.deploykf_istio_gateway.gateway.hostname) and port (deploykf_core.deploykf_istio_gateway.gateway.ports.https) combination for any of your other Gateways, or you will break deployKF (or more likely, they will just end up behind deployKF's auth). You should also avoid making gateways with subdomains of your configured hostname, as we already make use of these, and will probably add more in the future.For example, if you configure
deploykf.example.comand port8443, we currently use the following (with the configured HTTP port just being a redirect to HTTPS):deploykf.example.com:8443argo-server.deploykf.example.com:8443minio-api.deploykf.example.com:8443minio-console.deploykf.example.com:8443Advanced Stuff:
For those interested (and for my own future reference), the complexity here is that Envoy-Proxy (what Istio actually uses internally, and which we configure directly with Istio EnvoyFilters), does not provide a straightforward way to disable HTTP filters by default, and then only enable them for specific routes (so as to not affect existing routes on the gateway).
So to achieve this, we have to use a different approach for each filter-type we need:
envoy.extensions.filters.http.lua.v3.Lua)default_source_codebe a function that does nothing, and define the actual scripts undersource_codes, and then select that code for specific routes withenvoy.extensions.filters.http.lua.v3.LuaPerRoute.envoy.extensions.filters.http.ext_authz.v3.ExtAuthz)filter_enabled_metadataselector, which requires us to set DynamicMetadata for the routes that need to be enabled.envoy.extensions.filters.http.jwt_authn.v3.JwtAuthentication)rulesas empty, and define the actual requirements inrequirement_map, so we can useenvoy.extensions.filters.http.jwt_authn.v3.PerRouteConfigto select the appropriate validation rule withrequirement_namefor each route.Some useful commands to dump the Envoy-Proxy configs are:
istioctl proxy-config listener deploykf-gateway-XXXXXXXX-XXXX.deploykf-istio-gateway -o yaml:filterChainsofenvoy.filters.network.http_connection_manageristioctl proxy-config route deploykf-gateway-XXXXXXXX-XXXX.deploykf-istio-gateway -o yaml:typedPerFilterConfigconfigs are visible.