Skip to content

perf: compare service discovery nodes by addresss#12258

Merged
moonming merged 1 commit into
apache:masterfrom
nic-6443:nic/compare-nodes-by-addr
May 29, 2025
Merged

perf: compare service discovery nodes by addresss#12258
moonming merged 1 commit into
apache:masterfrom
nic-6443:nic/compare-nodes-by-addr

Conversation

@nic-6443

@nic-6443 nic-6443 commented May 26, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

Description

When the number of nodes discovered by the service is large, the comparison based on node content in the compare_upstream_node function will be very CPU-intensive. Since every request goes through compare_upstream_node, this results in much higher CPU consumption for upstreams based on service discovery compared to regular upstreams.

The current PR optimization approach is to first compare the memory addresses of nodes tables in compare_upstream_node. If the addresses are identical, it can quickly determine that two nodes are identical.

For this optimization to work, the following conditions must be met:

  • All existing and future service discoveries should not modify data on the same table. When a change occurs in a service_name corresponding to its nodes, a new table should be used to store data. I have confirmed that all existing service discoveries meet this condition.
  • In both service discovery and within the fill_nodes function, modifications to upstream data should not be made by cloning an old nodes table but rather by reusing the same table.

Benchmark Result

Before

1 Nodes
Upstream Nodes
❯ wrk -c 10 -t 5 -d 30s -R 50000 http://10.244.1.34:9080/get -H "Host: upstream.httpbin.dev"
Running 30s test @ http://10.244.1.34:9080/get
  5 threads and 10 connections
  Thread calibration: mean lat.: 3188.385ms, rate sampling interval: 11436ms
  Thread calibration: mean lat.: 3186.905ms, rate sampling interval: 11427ms
  Thread calibration: mean lat.: 3191.746ms, rate sampling interval: 11419ms
  Thread calibration: mean lat.: 3193.849ms, rate sampling interval: 11427ms
  Thread calibration: mean lat.: 3195.787ms, rate sampling interval: 11427ms
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.70s     3.65s   18.97s    57.66%
    Req/Sec     3.67k     2.71     3.67k    80.00%
  551436 requests in 30.00s, 91.50MB read
Requests/sec:  18381.28
Transfer/sec:      3.05MB
Service Discovery
❯ wrk -c 10 -t 5 -d 30s -R 50000 http://10.244.1.34:9080/get -H "Host: kubernetes.httpbin.dev"
Running 30s test @ http://10.244.1.34:9080/get
  5 threads and 10 connections
  Thread calibration: mean lat.: 3255.829ms, rate sampling interval: 11730ms
  Thread calibration: mean lat.: 3259.296ms, rate sampling interval: 11739ms
  Thread calibration: mean lat.: 3257.659ms, rate sampling interval: 11739ms
  Thread calibration: mean lat.: 3256.516ms, rate sampling interval: 11739ms
  Thread calibration: mean lat.: 3257.807ms, rate sampling interval: 11739ms
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.97s     3.74s   19.51s    57.70%
    Req/Sec     3.51k     3.35     3.52k    80.00%
  524343 requests in 30.00s, 87.01MB read
Requests/sec:  17478.22
Transfer/sec:      2.90MB
100 Nodes
Upstream Nodes
❯ wrk -c 10 -t 5 -d 30s -R 50000 http://10.244.1.34:9080/get -H "Host: upstream.httpbin.dev"
Running 30s test @ http://10.244.1.34:9080/get
  5 threads and 10 connections
  Thread calibration: mean lat.: 3437.621ms, rate sampling interval: 12722ms
  Thread calibration: mean lat.: 3431.776ms, rate sampling interval: 12713ms
  Thread calibration: mean lat.: 3440.046ms, rate sampling interval: 12722ms
  Thread calibration: mean lat.: 3439.794ms, rate sampling interval: 12730ms
  Thread calibration: mean lat.: 3433.747ms, rate sampling interval: 12730ms
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    14.45s     4.10s   21.50s    57.83%
    Req/Sec     2.83k     2.71     2.83k    80.00%
  425438 requests in 30.00s, 70.59MB read
Requests/sec:  14181.47
Transfer/sec:      2.35MB
Service Discovery
❯ wrk -c 10 -t 5 -d 30s -R 50000 http://10.244.1.34:9080/get -H "Host: kubernetes.httpbin.dev"
Running 30s test @ http://10.244.1.34:9080/get
  5 threads and 10 connections
  Thread calibration: mean lat.: 4340.687ms, rate sampling interval: 15581ms
  Thread calibration: mean lat.: 4336.465ms, rate sampling interval: 15589ms
  Thread calibration: mean lat.: 4338.800ms, rate sampling interval: 15589ms
  Thread calibration: mean lat.: 4341.254ms, rate sampling interval: 15589ms
  Thread calibration: mean lat.: 4339.235ms, rate sampling interval: 15589ms
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    17.33s     5.00s   25.95s    57.49%
    Req/Sec     1.34k     0.49     1.34k   100.00%
  201755 requests in 30.00s, 33.48MB read
Requests/sec:   6725.08
Transfer/sec:      1.12MB

After

1 Nodes
Upstream Nodes
❯ wrk -c 10 -t 5 -d 30s -R 50000 http://10.244.1.33:9080/get -H "Host: upstream.httpbin.dev"
Running 30s test @ http://10.244.1.33:9080/get
  5 threads and 10 connections
  Thread calibration: mean lat.: 3179.611ms, rate sampling interval: 11411ms
  Thread calibration: mean lat.: 3186.556ms, rate sampling interval: 11444ms
  Thread calibration: mean lat.: 3182.313ms, rate sampling interval: 11427ms
  Thread calibration: mean lat.: 3188.876ms, rate sampling interval: 11452ms
  Thread calibration: mean lat.: 3185.915ms, rate sampling interval: 11427ms
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.66s     3.66s   18.99s    57.61%
    Req/Sec     3.69k     1.79     3.69k    60.00%
  550068 requests in 30.00s, 91.28MB read
Requests/sec:  18335.79
Transfer/sec:      3.04MB
Service Discovery
❯ wrk -c 10 -t 5 -d 30s -R 50000 http://10.244.1.33:9080/get -H "Host: kubernetes.httpbin.dev"
Running 30s test @ http://10.244.1.33:9080/get
  5 threads and 10 connections
  Thread calibration: mean lat.: 3204.212ms, rate sampling interval: 11460ms
  Thread calibration: mean lat.: 3202.275ms, rate sampling interval: 11460ms
  Thread calibration: mean lat.: 3202.211ms, rate sampling interval: 11452ms
  Thread calibration: mean lat.: 3209.434ms, rate sampling interval: 11460ms
  Thread calibration: mean lat.: 3198.332ms, rate sampling interval: 11452ms
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.70s     3.65s   19.07s    57.70%
    Req/Sec     3.65k     2.65     3.65k    80.00%
  545739 requests in 30.00s, 90.56MB read
Requests/sec:  18191.54
Transfer/sec:      3.02MB
100 Nodes
Upstream Nodes
❯ wrk -c 10 -t 5 -d 30s -R 50000 http://10.244.1.84:9080/get -H "Host: upstream.httpbin.dev"
Running 30s test @ http://10.244.1.84:9080/get
  5 threads and 10 connections
  Thread calibration: mean lat.: 3581.313ms, rate sampling interval: 12812ms
  Thread calibration: mean lat.: 3587.216ms, rate sampling interval: 12828ms
  Thread calibration: mean lat.: 3580.373ms, rate sampling interval: 12820ms
  Thread calibration: mean lat.: 3580.219ms, rate sampling interval: 12820ms
  Thread calibration: mean lat.: 3586.169ms, rate sampling interval: 12828ms
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    14.35s     4.12s   21.36s    57.70%
    Req/Sec     2.84k     4.45     2.84k    80.00%
  431222 requests in 30.00s, 71.55MB read
Requests/sec:  14374.07
Transfer/sec:      2.39MB
Service Discovery
❯ wrk -c 10 -t 5 -d 30s -R 50000 http://10.244.1.84:9080/get -H "Host: kubernetes.httpbin.dev"
Running 30s test @ http://10.244.1.84:9080/get
  5 threads and 10 connections
  Thread calibration: mean lat.: 3553.303ms, rate sampling interval: 12771ms
  Thread calibration: mean lat.: 3556.625ms, rate sampling interval: 12779ms
  Thread calibration: mean lat.: 3553.023ms, rate sampling interval: 12763ms
  Thread calibration: mean lat.: 3559.705ms, rate sampling interval: 12787ms
  Thread calibration: mean lat.: 3549.895ms, rate sampling interval: 12763ms
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    14.36s     4.13s   21.53s    57.91%
    Req/Sec     2.81k     3.20     2.81k    80.00%
  423546 requests in 30.00s, 70.28MB read
Requests/sec:  14118.36
Transfer/sec:      2.34MB

Summary

1 Nodes 100 Nodes
Before (18381.28 - 17478.22) / 18381.28 = 5% (14181.47-6725.08) / 14181.47 = 52%
After (18335.79-18191.54) / 18335.79 = 0.8% (14374.07 - 14118.36) / 14374.07 = 1.8%

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

@dosubot dosubot Bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label May 26, 2025
@nic-6443
nic-6443 force-pushed the nic/compare-nodes-by-addr branch from 57e2a81 to 6db2724 Compare May 26, 2025 09:30
@nic-6443
nic-6443 force-pushed the nic/compare-nodes-by-addr branch from 6db2724 to 7b153c5 Compare May 26, 2025 09:31
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:XS This PR changes 0-9 lines, ignoring generated files. labels May 26, 2025
@moonming
moonming merged commit 77e33f9 into apache:master May 29, 2025
@nic-6443
nic-6443 deleted the nic/compare-nodes-by-addr branch June 4, 2025 06:51
laz-xyr pushed a commit to laz-xyr/apisix that referenced this pull request Jun 19, 2025
@nic-6443
perf: compare service discovery nodes by addresss (apache#12258)
1ff24b9
@nic-6443 nic-6443 mentioned this pull request Jun 20, 2025
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@membphis membphis membphis approved these changes
@nic-chen nic-chen nic-chen approved these changes
@AlinsRan AlinsRan AlinsRan approved these changes
@bzp2010 bzp2010 Awaiting requested review from bzp2010
@ronething ronething Awaiting requested review from ronething
@Revolyssup Revolyssup Awaiting requested review from Revolyssup

Assignees

No one assigned

Labels

size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@nic-6443 @membphis @nic-chen @AlinsRan @moonming