Skip to content

Commit 9a5d040

Browse files
committed
[ISSUE #8194] Add opt-in named upstream connection-pool groups for workload isolation
1 parent b4b8c4f commit 9a5d040

23 files changed

Lines changed: 2030 additions & 15 deletions

File tree

‎RELEASE-NOTES.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,16 @@
115115
the previous behavior with `shenyu.httpclient.keepAlive=false`.
116116
For a fixed pool, set `shenyu.httpclient.pool.maxConnections` explicitly to
117117
size the connection limit for the deployment; the default follows Reactor Netty.
118+
- Opt-in named upstream connection-pool groups are available through
119+
`shenyu.httpclient.pool-groups`. Groups must use the `FIXED` type, have an
120+
explicit unique name other than `shenyu-proxy`, and configure a positive
121+
`maxConnections` plus an omitted-or-positive `maxPendingAcquires` (at most 16
122+
groups). A divide rule selects one with `poolGroup`; an unknown group returns
123+
HTTP 503 instead of falling back. Definitions are read at startup only, and
124+
deployments without `pool-groups` keep the existing shared pool unchanged.
125+
Group metrics use Reactor Netty Micrometer meters when `metrics: true` and
126+
`micrometer-core` is present; they are not exported by the ShenYu Prometheus
127+
metrics endpoint in this release. See `docs/upstream-pool-groups.md`.
118128
- CORS origin patterns are compiled when the gateway starts. An invalid
119129
`shenyu.cross.allowed-origin.origin-regex` now fails startup with
120130
`PatternSyntaxException`, rather than failing the first matching request.

‎docs/upstream-pool-groups.md‎

Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
<!--
2+
Licensed to the Apache Software Foundation (ASF) under one or more
3+
contributor license agreements. See the NOTICE file distributed with
4+
this work for additional information regarding copyright ownership.
5+
The ASF licenses this file to You under the Apache License, Version 2.0
6+
(the "License"); you may not use this file except in compliance with
7+
the License. You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
-->
17+
18+
# Named upstream connection-pool groups
19+
20+
## Overview
21+
22+
A divide rule can select one of a finite set of named upstream connection-pool
23+
groups configured at gateway startup. Requests that use the same upstream origin
24+
but belong to different workloads can therefore use independent bounded
25+
connection and acquisition pools. A slow bulk workload cannot consume the
26+
connection/acquisition capacity reserved for an interactive workload merely
27+
because both call the same backend host.
28+
29+
This is an opt-in feature. With no `shenyu.httpclient.pool-groups` configured,
30+
the gateway keeps its existing single shared pool and request behavior is
31+
unchanged.
32+
33+
## Configuration
34+
35+
Pool groups live under the existing `shenyu.httpclient` prefix, next to the
36+
shared `pool` settings. Only the `FIXED` pool type is accepted for a group; each
37+
group is bounded by its own connection limit and pending-acquire queue.
38+
39+
```yaml
40+
shenyu:
41+
httpclient:
42+
pool-groups:
43+
- name: bulk
44+
type: FIXED
45+
maxConnections: 32
46+
acquireTimeout: 500
47+
maxPendingAcquires: 32
48+
- name: interactive
49+
type: FIXED
50+
maxConnections: 256
51+
acquireTimeout: 3000
52+
```
53+
54+
Rules for the group list:
55+
56+
- At most 16 groups.
57+
- `name` is required and unique (case-insensitive). It must not be
58+
`shenyu-proxy`, the name of the shared default pool.
59+
- `type` must be `FIXED`.
60+
- `maxConnections` must be strictly positive.
61+
- `acquireTimeout` must not be negative.
62+
- `maxPendingAcquires` must be omitted or strictly positive; `-1` and `0` are
63+
rejected for groups. The shared default pool keeps its historical behavior.
64+
- TLS, proxy, keep-alive, idle handlers, HTTP/2 negotiation and the shared
65+
Netty loop resources are inherited from `shenyu.httpclient`, exactly like the
66+
default client.
67+
- The group set is read only at startup. Adding, removing or changing a group
68+
requires a gateway restart.
69+
70+
To select a group, set `poolGroup` in the divide rule handle:
71+
72+
```json
73+
{
74+
"loadBalance": "random",
75+
"retry": 3,
76+
"poolGroup": "bulk"
77+
}
78+
```
79+
80+
The dashboard form does not expose `poolGroup` yet; use the Admin rule API or
81+
the handle JSON directly.
82+
83+
## Request behavior
84+
85+
The divide plugin publishes the selected group on the exchange before the HTTP
86+
client runs. The Netty and WebClient strategies both resolve that group once per
87+
attempt and use the group's client. The selection is pinned for the whole
88+
request lifetime, including retry/failover attempts and the response body; a
89+
rule update affects future requests only.
90+
91+
An unknown group name, a registry that has already been closed, or a rule that
92+
selects a group on a gateway where the group is not configured produces HTTP
93+
503 for that request. There is no silent fallback to another group or to the
94+
shared default pool. Requests without `poolGroup` always take the shared
95+
default client.
96+
97+
Group selection is per gateway process. Different gateways may configure
98+
different group names; a rule that references a group must only be published to
99+
gateways that define it. The limits are per group provider and per origin
100+
inside that provider, not cluster-wide quotas and not a request rate limiter.
101+
With HTTP/2, one connection multiplexes concurrent streams, so the isolation is
102+
connection/acquisition capacity rather than one connection per RPC.
103+
104+
## Metrics
105+
106+
Set `metrics: true` on a group to publish its Reactor Netty provider meters:
107+
108+
```yaml
109+
- name: bulk
110+
type: FIXED
111+
maxConnections: 32
112+
metrics: true
113+
```
114+
115+
The meters are native Reactor Netty Micrometer meters registered in the global
116+
Micrometer registry, for example:
117+
118+
- `reactor.netty.connection.provider.active.connections`
119+
- `reactor.netty.connection.provider.idle.connections`
120+
- `reactor.netty.connection.provider.pending.connections`
121+
- `reactor.netty.connection.provider.max.pending.connections`
122+
- `reactor.netty.connection.provider.pending.connections.time`
123+
124+
The provider name (the group name) is carried in the meter tags, so the metric
125+
cardinality is bounded by the configured groups and discovered origins. This
126+
requires `micrometer-core` on the classpath; the shipped bootstrap gets it
127+
through Spring Boot Actuator. It is not exported by the ShenYu Prometheus
128+
`MetricsRegister` endpoint in this delivery. To scrape the meters, add a
129+
Micrometer registry (for example `micrometer-registry-prometheus`) and expose
130+
it through your Micrometer/Actuator setup.
131+
132+
There is no separate Reactor Netty counter for a rejected acquisition. Use the
133+
pending queue depth, its configured maximum, and the pending-acquire timer's
134+
outcome to observe waiting and rejection behavior.
135+
136+
## Upgrade, migration and rollback
137+
138+
The feature is additive. Rolling upgrades should leave `pool-groups` absent on
139+
the new version until all gateways can serve the rules that reference a group;
140+
older gateways ignore an unknown `poolGroup` handle field. To roll back, remove
141+
`poolGroup` from the affected rule handles and restart the gateways without the
142+
`pool-groups` block. Removing the configuration restores the shared default pool
143+
and the previous behavior.
144+
145+
## Fixed-workload isolation measurement
146+
147+
`UpstreamPoolGroupIsolationTest` covers correctness. For a repeatable capacity
148+
measurement, the test sources also contain an opt-in benchmark that compares one
149+
shared provider with two independent providers against the same loopback origin
150+
(8 bulk clients on a 50 ms endpoint, 2 interactive clients on a fast endpoint,
151+
5 seconds per scenario):
152+
153+
```bash
154+
mvn -pl shenyu-plugin/shenyu-plugin-httpclient -am test \
155+
-Dtest=UpstreamPoolGroupIsolationBenchmark \
156+
-Dshenyu.poolgroup.benchmark=true -DfailIfNoTests=false
157+
```
158+
159+
Example output on Windows 10, JDK 17.0.18, 16 cores (fixed loopback workload,
160+
not a universal p50/p99 claim):
161+
162+
```text
163+
scenario,bulk_ok,bulk_fail,interactive_ok,interactive_fail,interactive_p50_ms,interactive_p99_ms
164+
shared,274,173293,85,82980,2,815
165+
groups,338,8,28774,0,0,1
166+
```
167+
168+
The shared scenario is one provider of 4 connections shared by both workloads;
169+
the groups scenario uses a bulk provider of 4 and an interactive provider of 8.
170+
The interactive workload is effectively starved in the shared scenario and
171+
unaffected in the grouped scenario, which is the fixed-workload evidence for
172+
the isolation boundary. Re-run the command on your deployment host before
173+
quoting latency numbers; results include loopback and JIT effects.
174+
175+
## Troubleshooting
176+
177+
| Symptom | Check |
178+
| --- | --- |
179+
| HTTP 503 with an unknown pool group message | The rule references a group not defined on this gateway, or the registry is shutting down. Check the exact group name and the gateway's startup configuration. |
180+
| Startup fails with a group validation message | Use `FIXED`, a positive `maxConnections`, an explicit unique name other than `shenyu-proxy`, and omit or use a positive `maxPendingAcquires`. |
181+
| Startup fails because a custom `HttpClient` bean is present | `pool-groups` owns its clients; remove the custom `HttpClient` bean or the group configuration. |
182+
| Metrics are missing | Set `metrics: true` on the group, ensure `micrometer-core` is present, and verify the meter is exported by your Micrometer registry. |

‎shenyu-bootstrap/src/main/resources/application.yml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -174,6 +174,24 @@ shenyu:
174174
# maxLifeTime: # unit: ms
175175
# evictionInterval: # unit: ms
176176
# metrics: false
177+
# # Optional named upstream connection-pool groups, see docs/upstream-pool-groups.md.
178+
# # Groups must use type FIXED, have an explicit unique name (not shenyu-proxy),
179+
# # and a positive maxConnections; maxPendingAcquires must be omitted or positive.
180+
# # The group set is read at startup only; a divide rule selects one with
181+
# # {"poolGroup":"bulk"}. Enable metrics: true to publish Reactor Netty provider
182+
# # meters through Micrometer (requires micrometer-core: present via actuator).
183+
# pool-groups:
184+
# - name: bulk
185+
# type: FIXED
186+
# maxConnections: 32
187+
# acquireTimeout: 500
188+
# maxPendingAcquires: 32
189+
# metrics: false
190+
# - name: interactive
191+
# type: FIXED
192+
# maxConnections: 256
193+
# acquireTimeout: 3000
194+
# metrics: false
177195
# proxy:
178196
# host:
179197
# port:

‎shenyu-common/src/main/java/org/apache/shenyu/common/constant/Constants.java‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,12 @@ public interface Constants {
134134
*/
135135
String HTTP_TIME_OUT = "httpTimeOut";
136136

137+
/**
138+
* The constant HTTP_POOL_GROUP: optional named upstream connection-pool group
139+
* chosen by the divide rule handle; absent means the default shared pool.
140+
*/
141+
String HTTP_POOL_GROUP = "httpPoolGroup";
142+
137143
/**
138144
* The constant HTTP_RETRY.
139145
*/

‎shenyu-common/src/main/java/org/apache/shenyu/common/dto/convert/rule/impl/DivideRuleHandle.java‎

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,32 @@ public class DivideRuleHandle implements RuleHandle {
7676
*/
7777
private StickySessionConfig sticky;
7878

79+
/**
80+
* Optional name of a configured upstream connection-pool group (see
81+
* {@code shenyu.httpclient.pool-groups}); null selects the default shared pool.
82+
* The name must reference a group that exists at gateway startup; an unknown
83+
* reference visibly fails the request instead of falling back silently.
84+
*/
85+
private String poolGroup;
86+
87+
/**
88+
* Gets the optional pool group name.
89+
*
90+
* @return the pool group name, or null for the default pool
91+
*/
92+
public String getPoolGroup() {
93+
return poolGroup;
94+
}
95+
96+
/**
97+
* Sets the optional pool group name.
98+
*
99+
* @param poolGroup the pool group name
100+
*/
101+
public void setPoolGroup(final String poolGroup) {
102+
this.poolGroup = poolGroup;
103+
}
104+
79105
/**
80106
* New instance divide rule handle.
81107
*
@@ -251,12 +277,12 @@ public boolean equals(final Object o) {
251277
&& requestMaxSize == that.requestMaxSize && Objects.equals(loadBalance, that.loadBalance)
252278
&& Objects.equals(retryStrategy, that.retryStrategy)
253279
&& Objects.equals(retryBackOffSpec, that.retryBackOffSpec) && Objects.equals(canary, that.canary)
254-
&& Objects.equals(sticky, that.sticky);
280+
&& Objects.equals(sticky, that.sticky) && Objects.equals(poolGroup, that.poolGroup);
255281
}
256282

257283
@Override
258284
public int hashCode() {
259-
return Objects.hash(loadBalance, retryStrategy, retryBackOffSpec, retry, timeout, headerMaxSize, requestMaxSize, canary, sticky);
285+
return Objects.hash(loadBalance, retryStrategy, retryBackOffSpec, retry, timeout, headerMaxSize, requestMaxSize, canary, sticky, poolGroup);
260286
}
261287

262288
@Override
@@ -283,6 +309,9 @@ public String toString() {
283309
+ canary
284310
+ ", sticky="
285311
+ sticky
312+
+ ", poolGroup='"
313+
+ poolGroup
314+
+ '\''
286315
+ '}';
287316
}
288317
}
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.shenyu.common.dto.convert.rule.impl;
19+
20+
import org.apache.shenyu.common.dto.convert.rule.canary.CanaryConfigValidator;
21+
import org.junit.jupiter.api.Test;
22+
23+
import static org.junit.jupiter.api.Assertions.assertEquals;
24+
import static org.junit.jupiter.api.Assertions.assertNotEquals;
25+
import static org.junit.jupiter.api.Assertions.assertNull;
26+
import static org.junit.jupiter.api.Assertions.assertTrue;
27+
28+
/**
29+
* Tests for the optional poolGroup field of {@link DivideRuleHandle}: it binds from the
30+
* handle JSON and stays null for every existing rule so the default pool is untouched.
31+
*/
32+
public final class DivideRuleHandlePoolGroupTest {
33+
34+
@Test
35+
public void poolGroupBindsFromHandleJson() {
36+
DivideRuleHandle handle = CanaryConfigValidator.parseHandle("{\"retry\":1,\"poolGroup\":\"bulk\"}");
37+
assertEquals("bulk", handle.getPoolGroup());
38+
assertEquals(1, handle.getRetry());
39+
}
40+
41+
@Test
42+
public void missingPoolGroupDefaultsToNull() {
43+
DivideRuleHandle handle = CanaryConfigValidator.parseHandle("{\"retry\":3}");
44+
assertNull(handle.getPoolGroup());
45+
}
46+
47+
@Test
48+
public void poolGroupParticipatesInEqualsHashCodeAndToString() {
49+
DivideRuleHandle bulk = new DivideRuleHandle();
50+
bulk.setPoolGroup("bulk");
51+
DivideRuleHandle interactive = new DivideRuleHandle();
52+
interactive.setPoolGroup("interactive");
53+
54+
assertNotEquals(bulk, interactive);
55+
assertNotEquals(bulk.hashCode(), interactive.hashCode());
56+
assertTrue(bulk.toString().contains("poolGroup='bulk'"));
57+
}
58+
}

0 commit comments

Comments
 (0)