Skip to content

gateway-api v1.1.0 standard-install breaks envoy-gateway and cilium-operator #3075

@networkhermit

Description

@networkhermit

What happened:

Recently I upgraded the gateway-api to v1.1.0 from the standard channel, but I found that both envoy-gateway and cilium-operator are in error, even though I have not directly used GRPCRoute gateway.networking.k8s.io/v1alpha2. (istio controller is fine)

From the v1.1.0 release notes:

GRPCRoute has graduated to GA (v1) and is now part of the Standard Channel. If
you are already using the experimental version GRPCRoute, we recommend holding
off on upgrading to the standard channel version of GRPCRoute until the
controllers you're using have been updated to support GRPCRoute v1. Until then,
it is safe to upgrade to the experimental channel version of GRPCRoute in v1.1
that includes both v1alpha2 and v1 API versions.

It seems to me that the experimental channel is more safe to upgrade, which is unintuitive.

I have two questions regarding this issue:

  1. Is it possible to keep old api versions in the standard channel and deprecate them only after some period and future releases?
  2. Does this issue supposed to be fixed by the envoy-gateway and cilium-operator implementations? They both requires some experimental gateway api crds for the controller to start, even when users don't use them at all. And upgrade the gateway-api using the stable channel accidentally breaks them.

Edited:

I wrongly made the assumption that the v1alpha2 GRPCRoute was installed from the gateway api standard channel, but in reality it was installed by envoy-gateway helm charts which embed the experimental grpcroutes crd. (The cilium-operator also requires the experimental gateway api crds, but istio only requires stable gateway crds so when I upgrade the gateway api it's not broken.)

envoy-gateway log:

Error: failed to create provider Kubernetes: failted to create gatewayapi controller: no matches for kind "GRPCRoute" in version "gateway.networking.k
8s.io/v1alpha2"
Usage:
  envoy-gateway server [flags]

Aliases:
  server, serve

Flags:
  -c, --config-path string   The path to the configuration file.
  -h, --help                 help for server

failed to create provider Kubernetes: failted to create gatewayapi controller: no matches for kind "GRPCRoute" in version "gateway.networking.k8s.io/v
1alpha2"

cilium-operator log:

time="2024-05-11T05:43:22Z" level=info msg="Checking for required GatewayAPI resources" requiredGVK="[gateway.networking.k8s.io/v1, Kind=gatewayclasse
s gateway.networking.k8s.io/v1, Kind=gateways gateway.networking.k8s.io/v1, Kind=httproutes gateway.networking.k8s.io/v1beta1, Kind=referencegrants ga
teway.networking.k8s.io/v1alpha2, Kind=grpcroutes gateway.networking.k8s.io/v1alpha2, Kind=tlsroutes]" subsys=gateway-api
time="2024-05-11T05:43:22Z" level=error msg="Invoke failed" ="gateway-api.initGatewayAPIController (pkg/gateway-api/cell.go:70)" error="failed to crea
te gateway controller: failed to setup reconciler: no matches for kind \"GRPCRoute\" in version \"gateway.networking.k8s.io/v1alpha2\"" subsys=hive
time="2024-05-11T05:43:22Z" level=info msg=Stopping subsys=hive
time="2024-05-11T05:43:22Z" level=fatal msg="failed to start: failed to create gateway controller: failed to setup reconciler: no matches for kind \"G
RPCRoute\" in version \"gateway.networking.k8s.io/v1alpha2\"" subsys=cilium-operator-generic

What you expected to happen:

standard-install is more safe to upgrade.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.lifecycle/rottenDenotes an issue or PR that has aged beyond stale and will be auto-closed.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions