perf: introduce translator context by kkk777-7 · Pull Request #7535 · envoyproxy/gateway

kkk777-7 · 2025-11-16T07:21:15Z

What this PR does / why we need it:

Introduce Translator Context to improve GatewayAPI translator performance.

Now, Gateway API translator methods hold a lot of local maps to improve retrievals.
These can be all moved to a common preprocessing step, saved in the Translator context and reused across methods.

Also each resource is currently retrieved by performing a linear search over resource.Resources.
Therefore, for resources that are accessed frequently, maintaining a map is expected to improve CPU performance.

This PR's scope

Service Map
- this was removed once in the PR move ordering gateway-api resources to provider #6695
- since the go bench results showed no memory usage concerns, it has been added back
NamespaceMap
ServiceImportMap
BackendMap
SecretMap
ConfigMapMap
ClusterTrustBundleMap
EndpointSliceMap

Although the following items were initially considered, the improvements were minimal and would increase the complexity, so they were excluded from the scope.　Ref : #7535 (comment)

~~Policy Target Gateway Map~~
~~Policy Target Route Map~~

Which issue(s) this PR fixes:

Fixes #6711

Release Notes: No

Benchmark Detail

gobench result

gatewayapi translator benchmark (vs latest main branch)

Execution Time (sec/op) improves especially on large workloads, 50% faster.
- large workload : HTTPRoute(2k), GRPCRoute(250), UDPRoute(100), ConfigMap(500), SP(500), BTP(500), EEP(500), Service(2k), EndpointSlice(2k)
no regressions of memory usage or allocation counts in any workloads.

                               │    main.txt    │               fix.txt                │
                               │     sec/op     │    sec/op      vs base               │
GatewayAPITranslator/small-10     2.012m ± 164%   2.188m ± 144%        ~ (p=0.132 n=6)
GatewayAPITranslator/medium-10    5.752m ±   6%   5.497m ±  35%        ~ (p=0.132 n=6)
GatewayAPITranslator/large-10    116.67m ±   5%   64.12m ±   9%  -45.04% (p=0.002 n=6)
geomean                           11.05m          9.171m         -17.02%

                               │    main.txt    │               fix.txt                │
                               │      B/op      │      B/op       vs base              │
GatewayAPITranslator/small-10    803.7Ki ± 277%   801.9Ki ± 278%       ~ (p=0.900 n=6)
GatewayAPITranslator/medium-10   3.018Mi ±   1%   3.006Mi ±   3%       ~ (p=0.065 n=6)
GatewayAPITranslator/large-10    23.35Mi ±   0%   23.32Mi ±   0%  -0.10% (p=0.002 n=6)
geomean                          3.810Mi          3.801Mi         -0.24%

                               │   main.txt    │               fix.txt               │
                               │   allocs/op   │   allocs/op    vs base              │
GatewayAPITranslator/small-10    12.89k ± 127%   12.90k ± 127%  +0.06% (p=0.035 n=6)
GatewayAPITranslator/medium-10   52.63k ±   0%   52.64k ±   0%  +0.03% (p=0.002 n=6)
GatewayAPITranslator/large-10    412.3k ±   0%   412.3k ±   0%       ~ (p=0.065 n=6)
geomean                          65.41k          65.43k         +0.03%

codecov · 2025-11-16T07:30:11Z

Codecov Report

❌ Patch coverage is 95.79439% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.40%. Comparing base (ea2ab03) to head (860773e).
⚠️ Report is 544 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/gatewayapi/contexts.go	96.59%	2 Missing and 1 partial ⚠️
internal/gatewayapi/backendtlspolicy.go	89.47%	0 Missing and 2 partials ⚠️
internal/gatewayapi/backendtrafficpolicy.go	95.00%	0 Missing and 1 partial ⚠️
internal/gatewayapi/envoyextensionpolicy.go	85.71%	0 Missing and 1 partial ⚠️
internal/gatewayapi/route.go	95.00%	1 Missing ⚠️
internal/gatewayapi/securitypolicy.go	92.85%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7535      +/-   ##
==========================================
+ Coverage   72.34%   72.40%   +0.05%     
==========================================
  Files         232      232              
  Lines       34209    34278      +69     
==========================================
+ Hits        24749    24818      +69     
- Misses       7681     7683       +2     
+ Partials     1779     1777       -2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

kkk777-7 · 2025-11-16T12:39:10Z

wait #7534

kkk777-7 · 2025-11-16T12:49:57Z

/retest

shreealt · 2025-11-16T13:00:10Z

@kkk777-7 you might wanna merge the main branch to this branch to include the disk space cleaner.

Signed-off-by: kkk777-7 <[email protected]>

zhaohuabing · 2025-11-18T08:08:10Z

The GetService in the resource.go can be deleted - it's no longer used.

gateway/internal/gatewayapi/resource/resource.go

Lines 117 to 124 in 7cb5f72

    
           func (r *Resources) GetService(namespace, name string) *corev1.Service { 
        
           	for _, svc := range r.Services { 
        
           		if svc.Namespace == namespace && svc.Name == name { 
        
           			return svc 
        
           		} 
        
           	} 
        
           	return nil 
        
           }

This reverts commit 14a3784. Signed-off-by: kkk777-7 <[email protected]>

Signed-off-by: kkk777-7 <[email protected]>

zhaohuabing · 2025-11-19T03:11:22Z

Think out load: could we also add other resources(Namespace,Secret,ConfigMap, etc.) to the translator context to avoid linear lookups to improve CPU performance?

This could increase memory usage - shouldn't be much increase since we just mirror the slices that already in resource using pointers, but we should benchmark both CPU and memory to ensure we don’t introduce significant overhead.

zirain · 2025-11-19T03:23:13Z

Think out load: could we also add other resources(Namespace,Secret,ConfigMap, etc.) to the translator context to avoid linear lookups to improve CPU performance?

we should do that.

kkk777-7 · 2025-11-19T04:19:27Z

Think out load: could we also add other resources(Namespace,Secret,ConfigMap, etc.) to the translator context to avoid linear lookups to improve CPU performance?

+1
Even with the addition of a 2k service map, the memory increase was minimal, so i think the benefit on CPU performance will be actually the bigger win!
I’ll add these in context and share the benchmark results :)

zirain · 2025-11-19T09:36:45Z

GetEndpointSlicesForBackend is another bottleneck.

Signed-off-by: kkk777-7 <[email protected]>

kkk777-7 · 2025-11-30T11:39:16Z

/retest

kkk777-7 · 2025-11-30T11:41:04Z

@arkodg
Sorry for the delay, I've addressed all of your comments.
Please take another look when you have moment.

Signed-off-by: kkk777-7 <[email protected]>

arkodg

LGTM thanks

kkk777-7 · 2025-12-02T16:19:46Z

/retest

kkk777-7 · 2025-12-02T16:22:11Z

Friendly ping @zirain @zhaohuabing

* perf: introduce translator context Signed-off-by: kkk777-7 <[email protected]> * perf: add policy map in translator context Signed-off-by: kkk777-7 <[email protected]> * chore: update tranlate bench Signed-off-by: kkk777-7 <[email protected]> * Revert "perf: add policy map in translator context" This reverts commit 14a3784. Signed-off-by: kkk777-7 <[email protected]> * add translator context method, remove unused function Signed-off-by: kkk777-7 <[email protected]> * update gatewayapi test output Signed-off-by: kkk777-7 <[email protected]> * add resources in translator context Signed-off-by: kkk777-7 <[email protected]> * add gatewayapi translate bench Signed-off-by: kkk777-7 <[email protected]> * fix: set resources in translator context Signed-off-by: kkk777-7 <[email protected]> * fix: go lint Signed-off-by: kkk777-7 <[email protected]> * address comments Signed-off-by: kkk777-7 <[email protected]> * feat: support egctl translate options Signed-off-by: kkk777-7 <[email protected]> * update: bench test Signed-off-by: kkk777-7 <[email protected]> * update embedded field Signed-off-by: kkk777-7 <[email protected]> * fix gatewayapi testdata output Signed-off-by: kkk777-7 <[email protected]> --------- Signed-off-by: kkk777-7 <[email protected]> Co-authored-by: zirain <[email protected]> Signed-off-by: Rudrakh Panigrahi <[email protected]>

* fix json report (#8614) Signed-off-by: Huabing (Robin) Zhao <[email protected]> Signed-off-by: Rudrakh Panigrahi <[email protected]> * fix: deep copy status in translator layer to avoid race (#8778) Signed-off-by: Rudrakh Panigrahi <[email protected]> * chore: setup translator test (#7627) * chore: setup gatewayapi translator test Signed-off-by: kkk777-7 <[email protected]> * remove: duplicate resources in testfiles Signed-off-by: kkk777-7 <[email protected]> * update gatewayapi test output Signed-off-by: kkk777-7 <[email protected]> --------- Signed-off-by: kkk777-7 <[email protected]> Signed-off-by: Rudrakh Panigrahi <[email protected]> * perf: introduce translator context (#7535) * perf: introduce translator context Signed-off-by: kkk777-7 <[email protected]> * perf: add policy map in translator context Signed-off-by: kkk777-7 <[email protected]> * chore: update tranlate bench Signed-off-by: kkk777-7 <[email protected]> * Revert "perf: add policy map in translator context" This reverts commit 14a3784. Signed-off-by: kkk777-7 <[email protected]> * add translator context method, remove unused function Signed-off-by: kkk777-7 <[email protected]> * update gatewayapi test output Signed-off-by: kkk777-7 <[email protected]> * add resources in translator context Signed-off-by: kkk777-7 <[email protected]> * add gatewayapi translate bench Signed-off-by: kkk777-7 <[email protected]> * fix: set resources in translator context Signed-off-by: kkk777-7 <[email protected]> * fix: go lint Signed-off-by: kkk777-7 <[email protected]> * address comments Signed-off-by: kkk777-7 <[email protected]> * feat: support egctl translate options Signed-off-by: kkk777-7 <[email protected]> * update: bench test Signed-off-by: kkk777-7 <[email protected]> * update embedded field Signed-off-by: kkk777-7 <[email protected]> * fix gatewayapi testdata output Signed-off-by: kkk777-7 <[email protected]> --------- Signed-off-by: kkk777-7 <[email protected]> Co-authored-by: zirain <[email protected]> Signed-off-by: Rudrakh Panigrahi <[email protected]> * fix gen-check Signed-off-by: Rudrakh Panigrahi <[email protected]> --------- Signed-off-by: Huabing (Robin) Zhao <[email protected]> Signed-off-by: Rudrakh Panigrahi <[email protected]> Signed-off-by: kkk777-7 <[email protected]> Co-authored-by: Huabing (Robin) Zhao <[email protected]> Co-authored-by: Kota Kimura <[email protected]> Co-authored-by: zirain <[email protected]>

kkk777-7 marked this pull request as ready for review November 16, 2025 12:39

kkk777-7 requested a review from a team as a code owner November 16, 2025 12:39

kkk777-7 added 3 commits November 16, 2025 21:01

perf: introduce translator context

872400b

Signed-off-by: kkk777-7 <[email protected]>

perf: add policy map in translator context

14a3784

Signed-off-by: kkk777-7 <[email protected]>

chore: update tranlate bench

ce0cfcc

Signed-off-by: kkk777-7 <[email protected]>

zirain force-pushed the perf-translator-context branch from 9527aa1 to ce0cfcc Compare November 16, 2025 13:01

zirain reviewed Nov 16, 2025

View reviewed changes

Comment thread internal/gatewayapi/contexts.go

zhaohuabing reviewed Nov 18, 2025

View reviewed changes

Comment thread internal/gatewayapi/testdata/extensions/httproute-with-custom-backend-mixed-multiple.out.yaml

zhaohuabing reviewed Nov 18, 2025

View reviewed changes

Comment thread internal/gatewayapi/contexts.go Outdated

kkk777-7 added 2 commits November 19, 2025 01:53

Revert "perf: add policy map in translator context"

3614571

This reverts commit 14a3784. Signed-off-by: kkk777-7 <[email protected]>

add translator context method, remove unused function

3d4c714

Signed-off-by: kkk777-7 <[email protected]>

kkk777-7 force-pushed the perf-translator-context branch from f9b5a6a to 3d4c714 Compare November 18, 2025 17:05

update gatewayapi test output

19123b4

Signed-off-by: kkk777-7 <[email protected]>

zirain previously approved these changes Nov 20, 2025

View reviewed changes

zirain mentioned this pull request Nov 20, 2025

Perf: cpu spikes #7573

Open

kkk777-7 added 2 commits November 21, 2025 00:10

add resources in translator context

41a4daa

Signed-off-by: kkk777-7 <[email protected]>

add gatewayapi translate bench

0d63071

Signed-off-by: kkk777-7 <[email protected]>

kkk777-7 dismissed zirain’s stale review via 0d63071 November 20, 2025 15:50

kkk777-7 added 2 commits November 21, 2025 01:16

fix: set resources in translator context

4145704

Signed-off-by: kkk777-7 <[email protected]>

fix: go lint

25a048e

Signed-off-by: kkk777-7 <[email protected]>

kkk777-7 commented Nov 20, 2025

View reviewed changes

Comment thread test/gobench/translate_test.go Outdated

Comment thread internal/gatewayapi/testdata/backendtrafficpolicy-dns-lookup-family.in.yaml

arkodg reviewed Nov 24, 2025

View reviewed changes

Comment thread internal/gatewayapi/testdata/backendtrafficpolicy-with-endpointoverride-loadbalancer.out.yaml

arkodg reviewed Nov 24, 2025

View reviewed changes

Comment thread internal/gatewayapi/contexts.go Outdated

kkk777-7 added 3 commits November 27, 2025 02:46

address comments

11ee9b0

Signed-off-by: kkk777-7 <[email protected]>

feat: support egctl translate options

55d088a

Signed-off-by: kkk777-7 <[email protected]>

Merge branch 'main' into perf-translator-context

106844b

Signed-off-by: kkk777-7 <[email protected]>

kkk777-7 dismissed stale reviews from zirain and zhaohuabing via f8b98e9 November 30, 2025 08:12

update: bench test

6461f2c

Signed-off-by: kkk777-7 <[email protected]>

kkk777-7 force-pushed the perf-translator-context branch from f8b98e9 to 6461f2c Compare November 30, 2025 08:39

kkk777-7 mentioned this pull request Nov 30, 2025

chore: setup translator test #7627

Merged

arkodg reviewed Dec 1, 2025

View reviewed changes

Comment thread internal/gatewayapi/translator.go Outdated

kkk777-7 added 3 commits December 2, 2025 09:31

update embedded field

cffcc19

Signed-off-by: kkk777-7 <[email protected]>

Merge branch 'main' into perf-translator-context

6b6e3dc

Signed-off-by: kkk777-7 <[email protected]>

fix gatewayapi testdata output

789c18f

Signed-off-by: kkk777-7 <[email protected]>

arkodg approved these changes Dec 2, 2025

View reviewed changes

arkodg requested review from a team December 2, 2025 01:23

zirain approved these changes Dec 2, 2025

View reviewed changes

Merge branch 'main' into perf-translator-context

860773e

zirain enabled auto-merge (squash) December 2, 2025 22:52

zirain merged commit 986c609 into envoyproxy:main Dec 3, 2025
52 of 55 checks passed

rudrakhp added cherrypick/release-v1.6.7 cherrypick/release-v1.7.3 and removed cherrypick/release-v1.7.3 labels Apr 24, 2026

Uh oh!

Conversation

kkk777-7 commented Nov 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

This PR's scope

Benchmark Detail

gobench result

Uh oh!

codecov Bot commented Nov 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

kkk777-7 commented Nov 16, 2025

Uh oh!

kkk777-7 commented Nov 16, 2025

Uh oh!

shreealt commented Nov 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zhaohuabing commented Nov 18, 2025

Uh oh!

Uh oh!

zhaohuabing commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zirain commented Nov 19, 2025

Uh oh!

kkk777-7 commented Nov 19, 2025

Uh oh!

zirain commented Nov 19, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kkk777-7 commented Nov 30, 2025

Uh oh!

kkk777-7 commented Nov 30, 2025

Uh oh!

Uh oh!

arkodg left a comment

Choose a reason for hiding this comment

Uh oh!

kkk777-7 commented Dec 2, 2025

Uh oh!

kkk777-7 commented Dec 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

kkk777-7 commented Nov 16, 2025 •

edited

Loading

codecov Bot commented Nov 16, 2025 •

edited

Loading

shreealt commented Nov 16, 2025 •

edited

Loading

zhaohuabing commented Nov 19, 2025 •

edited

Loading