feat(policy): Adding BTP support for UDP/TCPRoute by alexwo · Pull Request #3004 · envoyproxy/gateway

alexwo · 2024-03-24T16:50:42Z

What this PR does / why we need it:
Enables the application of BTP rules on gateway and TCP/UDP routes.
Modifications have been incorporated into the IR mapper and XDS translator.

Adds below BTP rules for tcp and tls routes:

LoadBalancer
Timeout
TCPKeepalive
CircuitBreaker
HealthCheck
ProxyProtcol

Adds below BTP rules for UDP routes:

LoadBalancer
Timeout

Which issue(s) this PR fixes:
#2880

Signed-off-by: Alexander Volchok <[email protected]>

…lexwo/gateway into btp_support_for_tcp_udp_routes Signed-off-by: Alexander Volchok <[email protected]>

Signed-off-by: Alexander Volchok <[email protected]>

codecov · 2024-03-24T18:53:30Z

Codecov Report

Attention: Patch coverage is 62.60870% with 43 lines in your changes are missing coverage. Please review.

Project coverage is 64.52%. Comparing base (d020ce7) to head (e047aee).

Files	Patch %	Lines
internal/ir/zz_generated.deepcopy.go	0.00%	28 Missing and 7 partials ⚠️
internal/gatewayapi/backendtrafficpolicy.go	84.61%	4 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3004      +/-   ##
==========================================
- Coverage   64.55%   64.52%   -0.04%     
==========================================
  Files         121      121              
  Lines       21235    21344     +109     
==========================================
+ Hits        13709    13772      +63     
- Misses       6672     6706      +34     
- Partials      854      866      +12

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Alexander Volchok <[email protected]>

alexwo · 2024-03-24T19:15:40Z

/retest

zhaohuabing · 2024-03-25T00:32:46Z

Should we enforce BTP setting validation on TCP/UDP route and update the status?

For example, HTTP-related settings such as FaultInjection, Retries, etc., can't be set when the targeting xRoute is TCP/UDP route or the targeting gateway doesn't contain HTTPRoute.

JuniorJPDJ · 2024-03-25T03:18:07Z

Can I also please ask to add TLSRoute support too?

alexwo · 2024-03-25T06:38:38Z

Can I also please ask to add TLSRoute support too?
@JuniorJPDJ
This is already included in the current pull request. Once merged, It will be possible to apply BTP either to specific TLS routes or to the entire gateway.

alexwo · 2024-03-25T06:43:47Z

Should we enforce BTP setting validation on TCP/UDP route and update the status?

For example, HTTP-related settings such as FaultInjection, Retries, etc., can't be set when the targeting xRoute is TCP/UDP route or the targeting gateway doesn't contain HTTPRoute.

Sounds good to me. Alternatively, we could update the status of each route to indicate which policies from the BTP are in effect, providing clarity to the user on how their route is affected.

Since this functionality isn't available currently, it might be best to introduce it through a separate change.
WDYT?

zhaohuabing · 2024-03-26T02:30:12Z

Should we enforce BTP setting validation on TCP/UDP route and update the status?
For example, HTTP-related settings such as FaultInjection, Retries, etc., can't be set when the targeting xRoute is TCP/UDP route or the targeting gateway doesn't contain HTTPRoute.

Sounds good to me. Alternatively, we could update the status of each route to indicate which policies from the BTP are in effect, providing clarity to the user on how their route is affected.

Since this functionality isn't available currently, it might be best to introduce it through a separate change. WDYT?

Sounds good. Could you please raise an issue to track this?

arkodg · 2024-03-26T04:28:38Z

hey @zhaohuabing we can't enforce validation here, because if the policy is applied to the Gateway, they could apply to HTTPRoutes as well as TCPRoutes

internal/xds/translator/listener.go

arkodg · 2024-03-26T04:36:35Z

internal/xds/translator/testdata/out/xds-ir/listener-tcp-keepalive.clusters.yaml

  outlierDetection: {}
  perConnectionBufferLimitBytes: 32768
  type: EDS
+  upstreamConnectionOptions:


the test is for listener side keep alive, how did this config come here (cluster) ?

The TCP keep-alive setting is specified in the IR, resulting in a keep-alive configuration for the TCP listener cluster. Typically, such settings are adjusted through a traffic policy, i'm not certain if this make sense in this context.

- name: "fourth-listener" address: "0.0.0.0" tcpKeepalive: probes: 10 port: 10083 destination: name: "tcp-route-dest" settings: - endpoints: - host: "1.2.3.4" port: 50000

can you raise a follow up issue for this ?

arkodg · 2024-03-26T04:37:22Z

internal/xds/translator/cluster.go

 	}

-	if tcpkeepalive.Probes != nil {
+	if tcpkeepalive.IdleTime != nil {


thanks for catching this

arkodg · 2024-03-26T04:45:38Z

internal/gatewayapi/backendtrafficpolicy.go

+		if t.MergeGateways && gatewayName != policyTarget {
+			continue
+		}
+


you probably also need some code similar to L500-510

// If any of the features are already set, it means that a more specific // policy(targeting xRoute) has already set it, so we skip it. // TODO: zhaohuabing group the features into a struct and check if all of them are set if r.RateLimit != nil || r.LoadBalancer != nil || r.ProxyProtocol != nil || r.HealthCheck != nil || r.CircuitBreaker != nil || r.FaultInjection != nil || r.TCPKeepalive != nil || r.Retry != nil || r.Timeout != nil { continue }

arkodg · 2024-03-26T04:45:56Z

internal/gatewayapi/backendtrafficpolicy.go

+			continue
+		}
+
+		udp.LoadBalancer = lb


same as above comment

arkodg · 2024-03-26T04:47:30Z

thanks for adding support for this @alexwo !

zhaohuabing · 2024-03-26T04:55:01Z

hey @zhaohuabing we can't enforce validation here, because if the policy is applied to the Gateway, they could apply to HTTPRoutes as well as TCPRoutes

Should we enforce validation when the target is a xRoute? It would be confusing if HTTP-related setting is configured for TCP/UDP route.

And it would be nice if we could allow users to know which settings in the BTP can be applied to which type of route. Probably add some comments to BTP settings?

alexwo · 2024-03-26T14:24:36Z

/retest

alexwo · 2024-03-26T14:43:16Z

/retest

alexwo · 2024-03-26T16:28:34Z

/retest

alexwo · 2024-03-26T16:43:10Z

/retest

alexwo · 2024-03-26T21:08:28Z

/retest

Signed-off-by: Alexander Volchok <[email protected]>

alexwo · 2024-03-26T22:23:13Z

/retest

guydc

LGTM, thanks for fixing the coverage check!

alexwo · 2024-03-26T22:40:31Z

/retest

alexwo · 2024-03-26T22:51:52Z

/retest

guydc · 2024-03-26T23:06:00Z

/retest

JuniorJPDJ · 2024-03-27T00:37:09Z

I just migrated to the latest commit and it works with TLSRoute! Thanks!

* Adding BTP support for UDP/TCP Routes Signed-off-by: Alexander Volchok <[email protected]> * fixining lint Signed-off-by: Alexander Volchok <[email protected]> * fix IdleTime check Signed-off-by: Alexander Volchok <[email protected]> * adding missing generates Signed-off-by: Alexander Volchok <[email protected]> * fixing code review comments Signed-off-by: Alexander Volchok <[email protected]> * incresing code coverage Signed-off-by: Alexander Volchok <[email protected]> --------- Signed-off-by: Alexander Volchok <[email protected]> Co-authored-by: zirain <[email protected]> Co-authored-by: Huabing Zhao <[email protected]> Signed-off-by: Yael Shechter <[email protected]>

* build(deps): bump github.com/prometheus/common from 0.50.0 to 0.51.1 (#3010) Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.50.0 to 0.51.1. - [Release notes](https://github.com/prometheus/common/releases) - [Commits](prometheus/common@v0.50.0...v0.51.1) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Huabing Zhao <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * docs: fix squash link (#3013) Signed-off-by: zirain <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * api: support IdleTimeout in clientTrafficPolicy Signed-off-by: Yael Shechter <[email protected]> * build(deps): bump fortio.org/fortio from 1.63.4 to 1.63.5 (#3011) Bumps [fortio.org/fortio](https://github.com/fortio/fortio) from 1.63.4 to 1.63.5. - [Release notes](https://github.com/fortio/fortio/releases) - [Commits](fortio/fortio@v1.63.4...v1.63.5) --- updated-dependencies: - dependency-name: fortio.org/fortio dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Huabing Zhao <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * ci: trigger docs action (#3017) Signed-off-by: zirain <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * ci: fix cherrypick permission (#2985) Signed-off-by: zirain <[email protected]> Co-authored-by: Huabing Zhao <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * fix: set path prefix for http ext auth service (#3018) Signed-off-by: huabing zhao <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * feat: add Chinese language support for doc site (#2532) * Add Chinese support for doc site Signed-off-by: Wilson Wu <[email protected]> * Add zh i18n for menus Signed-off-by: Wilson Wu <[email protected]> * Add zh content placeholders Signed-off-by: Wilson Wu <[email protected]> * Fix about zh page Signed-off-by: Wilson Wu <[email protected]> * Apply suggestions from code review Co-authored-by: Huabing Zhao <[email protected]> Signed-off-by: Wilson Wu <[email protected]> * docs: `%s/Application/API/g` (#2929) Signed-off-by: Wilson Wu <[email protected]> * remove old version for zh content Signed-off-by: Wilson Wu <[email protected]> * Update the latest version link Signed-off-by: Wilson Wu <[email protected]> * Fix default content dir Signed-off-by: Wilson Wu <[email protected]> * Translate landing page and fix broken link Signed-off-by: Wilson Wu <[email protected]> * Fix broken link Signed-off-by: Wilson Wu <[email protected]> * Add a contribution page Signed-off-by: Wilson Wu <[email protected]> --------- Signed-off-by: Wilson Wu <[email protected]> Co-authored-by: Huabing Zhao <[email protected]> Co-authored-by: Arko Dasgupta <[email protected]> Co-authored-by: zirain <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * feat(translator): implement connection limit (#2952) * implement connection limit Signed-off-by: Guy Daich <[email protected]> * fix lint Signed-off-by: Guy Daich <[email protected]> * fix lint 2 Signed-off-by: Guy Daich <[email protected]> * fix ir, coverage Signed-off-by: Guy Daich <[email protected]> * fix lint 3 Signed-off-by: Guy Daich <[email protected]> * open more connection in e2e Signed-off-by: Guy Daich <[email protected]> * fix error type Signed-off-by: Guy Daich <[email protected]> * add additional connections Signed-off-by: Guy Daich <[email protected]> * make limit value required Signed-off-by: Guy Daich <[email protected]> * add error-flow unit test Signed-off-by: Guy Daich <[email protected]> * fix lint 4 Signed-off-by: Guy Daich <[email protected]> * assert policy accepted in test Signed-off-by: Guy Daich <[email protected]> * rename limit => connectionLimit Signed-off-by: Guy Daich <[email protected]> --------- Signed-off-by: Guy Daich <[email protected]> Co-authored-by: zirain <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * docs: remove 0.1.0 from linkinator ignore list (#3015) Signed-off-by: yuluo <[email protected]> Co-authored-by: zirain <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * disable connection limit test (#3025) Signed-off-by: huabing zhao <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * fix: use correct connection limit attribute name in e2e, docs (#3028) fix attribute name Signed-off-by: Guy Daich <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * docs: document patches in envoy proxy (#3027) * docs: document patches in envoy proxy Signed-off-by: Yael Shechter <[email protected]> * fix linting issue Signed-off-by: Yael Shechter <[email protected]> * fix pr comment Signed-off-by: Yael Shechter <[email protected]> --------- Signed-off-by: Yael Shechter <[email protected]> * feat(policy): Adding BTP support for UDP/TCPRoute (#3004) * Adding BTP support for UDP/TCP Routes Signed-off-by: Alexander Volchok <[email protected]> * fixining lint Signed-off-by: Alexander Volchok <[email protected]> * fix IdleTime check Signed-off-by: Alexander Volchok <[email protected]> * adding missing generates Signed-off-by: Alexander Volchok <[email protected]> * fixing code review comments Signed-off-by: Alexander Volchok <[email protected]> * incresing code coverage Signed-off-by: Alexander Volchok <[email protected]> --------- Signed-off-by: Alexander Volchok <[email protected]> Co-authored-by: zirain <[email protected]> Co-authored-by: Huabing Zhao <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * fix: add missing http filters to the http filter chain (#2970) * fix: add missing http filters to the http filter chain Signed-off-by: huabing zhao <[email protected]> * refactor Signed-off-by: huabing zhao <[email protected]> * fix lint Signed-off-by: huabing zhao <[email protected]> * add comments Signed-off-by: huabing zhao <[email protected]> * remove refactor Signed-off-by: huabing zhao <[email protected]> * remove refactor Signed-off-by: huabing zhao <[email protected]> * fix gen Signed-off-by: huabing zhao <[email protected]> * fix lint Signed-off-by: Huabing Zhao <[email protected]> --------- Signed-off-by: huabing zhao <[email protected]> Signed-off-by: Huabing Zhao <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * fix pr comment Signed-off-by: Yael Shechter <[email protected]> * ci: grant pull-requests permission to cherrypick action (#3039) Signed-off-by: zirain <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * Update ADOPTERS.md for Airspace Link (#3045) Signed-off-by: Fabrice Aneche <[email protected]> Co-authored-by: zirain <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * feat: added option to specify resources on the OIDC security policy (#3030) added option to specify resources on the OIDC filter Signed-off-by: jaynis <[email protected]> Co-authored-by: Huabing Zhao <[email protected]> Signed-off-by: Yael Shechter <[email protected]> * run make commands Signed-off-by: Yael Shechter <[email protected]> * fix: allow websockets in url rewrite (#3022) allow websockets in url rewrite Signed-off-by: Jesse Haka <[email protected]> Co-authored-by: zirain <[email protected]> Signed-off-by: Yael Shechter <[email protected]> --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Yael Shechter <[email protected]> Signed-off-by: zirain <[email protected]> Signed-off-by: huabing zhao <[email protected]> Signed-off-by: Wilson Wu <[email protected]> Signed-off-by: Guy Daich <[email protected]> Signed-off-by: yuluo <[email protected]> Signed-off-by: Alexander Volchok <[email protected]> Signed-off-by: Huabing Zhao <[email protected]> Signed-off-by: Fabrice Aneche <[email protected]> Signed-off-by: jaynis <[email protected]> Signed-off-by: Jesse Haka <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Huabing Zhao <[email protected]> Co-authored-by: zirain <[email protected]> Co-authored-by: Wilson Wu <[email protected]> Co-authored-by: Arko Dasgupta <[email protected]> Co-authored-by: Guy Daich <[email protected]> Co-authored-by: YuLuo <[email protected]> Co-authored-by: Alex Volchok <[email protected]> Co-authored-by: Fabrice Aneche <[email protected]> Co-authored-by: jaynis <[email protected]> Co-authored-by: Jesse Haka <[email protected]>

alexwo added 2 commits March 24, 2024 17:54

Adding BTP support for UDP/TCP Routes

4c80c11

Signed-off-by: Alexander Volchok <[email protected]>

Merge branch 'main' into btp_support_for_tcp_udp_routes

b0e7aa6

alexwo changed the title ~~feat(policy): Adding BTP support for TCPRoute~~ feat(policy): Adding BTP support for UDP/TCPRoute Mar 24, 2024

alexwo added 3 commits March 24, 2024 18:20

fixining lint

0ba14c9

Signed-off-by: Alexander Volchok <[email protected]>

Merge branch 'btp_support_for_tcp_udp_routes' of https://github.com/a…

8499012

…lexwo/gateway into btp_support_for_tcp_udp_routes Signed-off-by: Alexander Volchok <[email protected]>

fix IdleTime check

b2ce980

Signed-off-by: Alexander Volchok <[email protected]>

adding missing generates

b72f289

Signed-off-by: Alexander Volchok <[email protected]>

alexwo marked this pull request as ready for review March 24, 2024 19:47

alexwo requested a review from a team as a code owner March 24, 2024 19:47

Merge branch 'main' into btp_support_for_tcp_udp_routes

07201a8

Merge branch 'main' into btp_support_for_tcp_udp_routes

4f29a62

alexwo and others added 4 commits March 25, 2024 07:44

Merge branch 'main' into btp_support_for_tcp_udp_routes

92e8d41

Merge branch 'main' into btp_support_for_tcp_udp_routes

92c9f2c

Merge branch 'main' into btp_support_for_tcp_udp_routes

3319c5a

Merge branch 'main' into btp_support_for_tcp_udp_routes

8e1122a

arkodg reviewed Mar 26, 2024

View reviewed changes

internal/xds/translator/listener.go Outdated Show resolved Hide resolved

arkodg reviewed Mar 26, 2024

View reviewed changes

arkodg requested a review from a team March 26, 2024 09:50

Merge branch 'main' into btp_support_for_tcp_udp_routes

4765bd8

alexwo mentioned this pull request Mar 26, 2024

Document BTP supported features for HTTP/UDP/TCPRoutes #3029

Open

Merge branch 'main' into btp_support_for_tcp_udp_routes

a2591db

guydc previously approved these changes Mar 26, 2024

View reviewed changes

Merge branch 'main' into btp_support_for_tcp_udp_routes

d104234

incresing code coverage

e047aee

Signed-off-by: Alexander Volchok <[email protected]>

alexwo dismissed stale reviews from guydc and arkodg via e047aee March 26, 2024 22:18

guydc approved these changes Mar 26, 2024

View reviewed changes

arkodg approved these changes Mar 26, 2024

View reviewed changes

arkodg mentioned this pull request Mar 26, 2024

Fix TCPKeepAlive config for TCPRoute #3032

Closed

guydc merged commit a11aa70 into envoyproxy:main Mar 26, 2024

alexwo deleted the btp_support_for_tcp_udp_routes branch March 27, 2024 05:09

JuniorJPDJ mentioned this pull request Mar 27, 2024

Support downstream and upstream Proxy Protocol #1328

Closed

Conversation

alexwo commented Mar 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Mar 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

alexwo commented Mar 24, 2024

Uh oh!

zhaohuabing commented Mar 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JuniorJPDJ commented Mar 25, 2024

Uh oh!

alexwo commented Mar 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexwo commented Mar 25, 2024

Uh oh!

zhaohuabing commented Mar 26, 2024

Uh oh!

arkodg commented Mar 26, 2024

Uh oh!

Uh oh!

arkodg Mar 26, 2024

Choose a reason for hiding this comment

Uh oh!

alexwo Mar 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arkodg Mar 26, 2024

Choose a reason for hiding this comment

Uh oh!

arkodg Mar 26, 2024

Choose a reason for hiding this comment

Uh oh!

arkodg Mar 26, 2024

Choose a reason for hiding this comment

Uh oh!

alexwo Mar 26, 2024

Choose a reason for hiding this comment

Uh oh!

arkodg Mar 26, 2024

Choose a reason for hiding this comment

Uh oh!

alexwo Mar 26, 2024

Choose a reason for hiding this comment

Uh oh!

arkodg commented Mar 26, 2024

Uh oh!

zhaohuabing commented Mar 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexwo commented Mar 26, 2024

Uh oh!

alexwo commented Mar 26, 2024

Uh oh!

alexwo commented Mar 26, 2024

Uh oh!

alexwo commented Mar 26, 2024

Uh oh!

alexwo commented Mar 26, 2024

Uh oh!

alexwo commented Mar 26, 2024

Uh oh!

guydc left a comment

Choose a reason for hiding this comment

Uh oh!

alexwo commented Mar 26, 2024

Uh oh!

alexwo commented Mar 26, 2024

Uh oh!

guydc commented Mar 26, 2024

Uh oh!

JuniorJPDJ commented Mar 27, 2024

Uh oh!

Reviewers

Assignees

alexwo commented Mar 24, 2024 •

edited

Loading

codecov bot commented Mar 24, 2024 •

edited

Loading

zhaohuabing commented Mar 25, 2024 •

edited

Loading

alexwo commented Mar 25, 2024 •

edited

Loading

alexwo Mar 26, 2024 •

edited

Loading

zhaohuabing commented Mar 26, 2024 •

edited

Loading