Skip to content

feat: allow other istio gateways on ingress deployment#66

Merged
thesuperzapper merged 1 commit into
deployKF:mainfrom
thesuperzapper:allow-other-istio-gateways
Jan 27, 2024
Merged

feat: allow other istio gateways on ingress deployment#66
thesuperzapper merged 1 commit into
deployKF:mainfrom
thesuperzapper:allow-other-istio-gateways

Conversation

@thesuperzapper

Copy link
Copy Markdown
Member

This PR completely changes how the EnvoyFilters are structured in deployKF, allowing users who want to host the deployKF Istio Gateway (CRD) on an existing Istio Ingress Gateway Deployment (Pods) can do so.

This change will require you to PRUNE sync in ArgoCD for the deploykf-istio-gateway app.

For reference, while you can now use an existing Ingress Gateway Deployment (Pods), deployKF will still manage its own virtual Gateway (CRD), as this is how we know which routes are part of deployKF (and thus need authentication).

Additionally, you can't use the same hostname (deploykf_core.deploykf_istio_gateway.gateway.hostname) and port (deploykf_core.deploykf_istio_gateway.gateway.ports.https) combination for any of your other Gateways, or you will break deployKF (or more likely, they will just end up behind deployKF's auth). You should also avoid making gateways with subdomains of your configured hostname, as we already make use of these, and will probably add more in the future.

For example, if you configure deploykf.example.com and port 8443, we currently use the following (with the configured HTTP port just being a redirect to HTTPS):

  • deploykf.example.com:8443
  • argo-server.deploykf.example.com:8443
  • minio-api.deploykf.example.com:8443
  • minio-console.deploykf.example.com:8443

Advanced Stuff:

For those interested (and for my own future reference), the complexity here is that Envoy-Proxy (what Istio actually uses internally, and which we configure directly with Istio EnvoyFilters), does not provide a straightforward way to disable HTTP filters by default, and then only enable them for specific routes (so as to not affect existing routes on the gateway).

So to achieve this, we have to use a different approach for each filter-type we need:

  1. Lua (envoy.extensions.filters.http.lua.v3.Lua)
    • These are straightforward, we can make default_source_code be a function that does nothing, and define the actual scripts under source_codes, and then select that code for specific routes with envoy.extensions.filters.http.lua.v3.LuaPerRoute.
  2. Ext Authz (envoy.extensions.filters.http.ext_authz.v3.ExtAuthz)
    • These are complex, the only way to disable them is with the filter_enabled_metadata selector, which requires us to set DynamicMetadata for the routes that need to be enabled.
    • We use a Lua filter to set the DynamicMetadata (because the Lua filters are easier to enable per route)
  3. JWT AuthN (envoy.extensions.filters.http.jwt_authn.v3.JwtAuthentication)
    • These are straightforward, we can just set rules as empty, and define the actual requirements in requirement_map, so we can use envoy.extensions.filters.http.jwt_authn.v3.PerRouteConfig to select the appropriate validation rule with requirement_name for each route.

Some useful commands to dump the Envoy-Proxy configs are:

  • istioctl proxy-config listener deploykf-gateway-XXXXXXXX-XXXX.deploykf-istio-gateway -o yaml:
    • Dumps the listener configs, which show the HTTP filterChains of envoy.filters.network.http_connection_manager
  • istioctl proxy-config route deploykf-gateway-XXXXXXXX-XXXX.deploykf-istio-gateway -o yaml:
    • Dumps the routes HTTP router, where our typedPerFilterConfig configs are visible.

@thesuperzapper thesuperzapper added this to the v0.1.4 milestone Dec 11, 2023
@thesuperzapper thesuperzapper added the status/ready-to-merge status - this will be merged into an upcoming release label Dec 11, 2023
@thesuperzapper
thesuperzapper merged commit 8b3434e into deployKF:main Jan 27, 2024
@thesuperzapper
thesuperzapper deleted the allow-other-istio-gateways branch January 27, 2024 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature changelog/significant status/ready-to-merge status - this will be merged into an upcoming release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant