What happened:
Recently I upgraded the gateway-api to v1.1.0 from the standard channel, but I found that both envoy-gateway and cilium-operator are in error, even though I have not directly used GRPCRoute gateway.networking.k8s.io/v1alpha2. (istio controller is fine)
From the v1.1.0 release notes:
GRPCRoute has graduated to GA (v1) and is now part of the Standard Channel. If
you are already using the experimental version GRPCRoute, we recommend holding
off on upgrading to the standard channel version of GRPCRoute until the
controllers you're using have been updated to support GRPCRoute v1. Until then,
it is safe to upgrade to the experimental channel version of GRPCRoute in v1.1
that includes both v1alpha2 and v1 API versions.
It seems to me that the experimental channel is more safe to upgrade, which is unintuitive.
I have two questions regarding this issue:
Is it possible to keep old api versions in the standard channel and deprecate them only after some period and future releases?
- Does this issue supposed to be fixed by the envoy-gateway and cilium-operator implementations? They both requires some experimental gateway api crds for the controller to start, even when users don't use them at all. And upgrade the gateway-api using the stable channel accidentally breaks them.
Edited:
I wrongly made the assumption that the v1alpha2 GRPCRoute was installed from the gateway api standard channel, but in reality it was installed by envoy-gateway helm charts which embed the experimental grpcroutes crd. (The cilium-operator also requires the experimental gateway api crds, but istio only requires stable gateway crds so when I upgrade the gateway api it's not broken.)
envoy-gateway log:
Error: failed to create provider Kubernetes: failted to create gatewayapi controller: no matches for kind "GRPCRoute" in version "gateway.networking.k
8s.io/v1alpha2"
Usage:
envoy-gateway server [flags]
Aliases:
server, serve
Flags:
-c, --config-path string The path to the configuration file.
-h, --help help for server
failed to create provider Kubernetes: failted to create gatewayapi controller: no matches for kind "GRPCRoute" in version "gateway.networking.k8s.io/v
1alpha2"
cilium-operator log:
time="2024-05-11T05:43:22Z" level=info msg="Checking for required GatewayAPI resources" requiredGVK="[gateway.networking.k8s.io/v1, Kind=gatewayclasse
s gateway.networking.k8s.io/v1, Kind=gateways gateway.networking.k8s.io/v1, Kind=httproutes gateway.networking.k8s.io/v1beta1, Kind=referencegrants ga
teway.networking.k8s.io/v1alpha2, Kind=grpcroutes gateway.networking.k8s.io/v1alpha2, Kind=tlsroutes]" subsys=gateway-api
time="2024-05-11T05:43:22Z" level=error msg="Invoke failed" ="gateway-api.initGatewayAPIController (pkg/gateway-api/cell.go:70)" error="failed to crea
te gateway controller: failed to setup reconciler: no matches for kind \"GRPCRoute\" in version \"gateway.networking.k8s.io/v1alpha2\"" subsys=hive
time="2024-05-11T05:43:22Z" level=info msg=Stopping subsys=hive
time="2024-05-11T05:43:22Z" level=fatal msg="failed to start: failed to create gateway controller: failed to setup reconciler: no matches for kind \"G
RPCRoute\" in version \"gateway.networking.k8s.io/v1alpha2\"" subsys=cilium-operator-generic
What you expected to happen:
standard-install is more safe to upgrade.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
What happened:
Recently I upgraded the gateway-api to v1.1.0 from the standard channel, but I found that both envoy-gateway and cilium-operator are in error, even though I have not directly used GRPCRoute gateway.networking.k8s.io/v1alpha2. (istio controller is fine)
From the v1.1.0 release notes:
It seems to me that the experimental channel is more safe to upgrade, which is unintuitive.I have two questions regarding this issue:
Is it possible to keep old api versions in the standard channel and deprecate them only after some period and future releases?Edited:
I wrongly made the assumption that the v1alpha2 GRPCRoute was installed from the gateway api standard channel, but in reality it was installed by envoy-gateway helm charts which embed the experimental grpcroutes crd. (The cilium-operator also requires the experimental gateway api crds, but istio only requires stable gateway crds so when I upgrade the gateway api it's not broken.)
envoy-gateway log:
cilium-operator log:
What you expected to happen:
standard-install is more safe to upgrade.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?: