-
Notifications
You must be signed in to change notification settings - Fork 8.2k
Description
Currently, we have ENABLE_NATIVE_SIDECARS={true,false}. A user can set this to true IFF they know the API server is on 1.29 and all nodes that run Istio containers are on 1.29. (Technically, it can still be turned off in 1.29 with a feature flag, but its default on in k8s 1.29).
This makes automatic enablement challenging, as we currently only know the API server version.
If we want to automatically enable this flag, we have two options:
-
Detect API server version is at least N, if so turn it on. If we make N=1.29 it will certainly break users just upgrading to 1.29. If we make N=1.33, it will only break users outside the OSS supported skew range but will also delay this by 15 months.
-
Add support to read all node versions in the injector; turn on only if all nodes are >=1.29. We already have a node informer elsewhere, so we just need to pass this through to the injector. Note at injection time we do not know the node a pod will run on, so we need to aggregate all nodes, not just a specific one on each request. There may be false negatives here, as users could have some node pools with 1.29 and ensure Istio pods only run on those (via nodeSelector, etc); I don't think this is safe (or simple) to try to support, as we may not be the last webhook to run. We would need all nodes to be 1.29. Of course, users could always explicitly set this to enabled. This has a theoretical race condition if a new 1.28 node comes up and we haven't seen it yet, but this seems exceptionally unlikely.
Alternatively, we can just not do this at all and make users manually opt in. Keeping in mind this is a "Beta" feature in Kubernetes, it seems reasonable to not automatically turn it on for all users yet. However, the "automatic" mode could still be useful even if its off-by-default, so that users can opt into it but with safeguards.