feat: Add priority groups in backends#14379
Merged
Merged
Conversation
Signed-off-by: David Jumani <[email protected]>
gateway-bot
added
kind/feature
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a new priorityGroups backend mode to the kgateway Backend CRD and backend translation plugin, enabling active/passive failover by flattening endpoints from referenced static Backends into a single Envoy cluster with multiple priority levels.
Changes:
- Extends the
BackendAPI/CRD withpriorityGroups(ordered groups of same-namespace Backend refs) and updates generated deepcopy + Helm CRDs/validation. - Implements priority-group endpoint flattening in the backend plugin, translating groups ->
LocalityLbEndpointswithpriorityset by group order (and enabling Envoy DNS cluster type when needed). - Adds unit and e2e coverage validating translation, DNS behavior, error cases, and failover/recovery driven by active health checks.
Reviewed changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| test/e2e/features/backends/testdata/priority-groups.yaml | Adds e2e manifest covering a priority-groups Backend and health check policy. |
| test/e2e/features/backends/suite.go | Adds an e2e test that validates failover and recovery across priority groups. |
| pkg/kgateway/extensions2/plugins/backend/priority_groups.go | Implements IR + translation for priority-groups backends into Envoy cluster load assignment priorities. |
| pkg/kgateway/extensions2/plugins/backend/priority_groups_test.go | Unit tests for priority-group IR construction, DNS behavior, errors, and non-mutation of IR. |
| pkg/kgateway/extensions2/plugins/backend/plugin.go | Wires priority-groups translation into backend plugin flow and IR equality. |
| pkg/kgateway/extensions2/plugins/backend/ec2_test.go | Updates translation-function construction to match new signature. |
| pkg/kgateway/extensions2/plugins/backend/aws_test.go | Updates translation-function construction to match new signature. |
| install/helm/kgateway-crds/templates/gateway.kgateway.dev_backends.yaml | Updates Helm-rendered CRD schema/validation for priorityGroups. |
| design/13643-backend-priority-groups.md | Adds design doc describing API, translation details, limitations, and alternatives. |
| api/v1alpha1/kgateway/zz_generated.deepcopy.go | Regenerates deepcopy code for new API types/fields. |
| api/v1alpha1/kgateway/backend_types.go | Adds priorityGroups types/validation and the new backend type constant/enum value. |
Files not reviewed (1)
- api/v1alpha1/kgateway/zz_generated.deepcopy.go: Generated file
Signed-off-by: David Jumani <[email protected]>
Signed-off-by: David Jumani <[email protected]>
Signed-off-by: David Jumani <[email protected]>
Signed-off-by: David Jumani <[email protected]>
andy-fong
reviewed
Jul 10, 2026
| locality.LbEndpoints = append(locality.LbEndpoints, ep.GetLbEndpoints()...) | ||
| } | ||
| } | ||
| loadAssignment.Endpoints = append(loadAssignment.Endpoints, locality) |
Contributor
There was a problem hiding this comment.
it's possible that locality.LbEndpoints is empty, should we check for that?
Contributor
Author
There was a problem hiding this comment.
I don't think so - it should error out if there's an invalid backendref and not translate the backend itself
| // PriorityGroups is an ordered list of backend groups used for failover. | ||
| // Traffic is sent to the backends of the first group; each subsequent | ||
| // group is only used when the backends of all preceding groups are | ||
| // unhealthy. The health check can be configured via the BackendConfigPolicy |
Contributor
There was a problem hiding this comment.
can be a follow up. What if there is health check configure on the individual backend itself. Should we honor that first and fallback to this one if not set?
Contributor
Author
There was a problem hiding this comment.
I have a branch for that and will raise a PR once this one merges. I'd like feedback on the overall API before going further
andy-fong
approved these changes
Jul 10, 2026
sadieleob
pushed a commit
to sadieleob/cloud-native-playgrounds
that referenced
this pull request
Jul 17, 2026
Multi-tier active/passive failover using the new priorityGroups Backend type in kgateway enterprise 2.3.0+. Includes Kind setup, install steps, demo manifests, and a live test script. Ref: kgateway-dev/kgateway#14379
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a new priorityGroups backend type to the Backend CRD, enabling active/passive failover between static backends.
A priority groups Backend holds an ordered list of groups, each referencing one or more static Backends in the same namespace. The referenced backends' endpoints are merged into a single Envoy cluster as locality endpoints, with one LocalityLbEndpoints per group whose priority matches the group's position in the list (first group is priority 0). Envoy sends traffic to the highest-priority group and spills over to subsequent groups as endpoints become unhealthy, driven by an active health check attached via BackendConfigPolicy — failover and recovery happen entirely in the data plane. Members of the same group share a priority level and are load balanced together.
Only static backends may be referenced. Other backend types (AWS Lambda, GCP, DynamicForwardProxy) cannot be expressed as endpoints in another cluster's load assignment since they rely on per-cluster filters and transport sockets; see the design doc for details and the internal-listener alternative that would lift this restriction.
Fixes #13643
Change Type
/kind feature
Changelog
Additional Notes