|
| 1 | +<!-- |
| 2 | +Licensed to the Apache Software Foundation (ASF) under one or more |
| 3 | +contributor license agreements. See the NOTICE file distributed with |
| 4 | +this work for additional information regarding copyright ownership. |
| 5 | +The ASF licenses this file to You under the Apache License, Version 2.0 |
| 6 | +(the "License"); you may not use this file except in compliance with |
| 7 | +the License. You may obtain a copy of the License at |
| 8 | +
|
| 9 | + http://www.apache.org/licenses/LICENSE-2.0 |
| 10 | +
|
| 11 | +Unless required by applicable law or agreed to in writing, software |
| 12 | +distributed under the License is distributed on an "AS IS" BASIS, |
| 13 | +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| 14 | +See the License for the specific language governing permissions and |
| 15 | +limitations under the License. |
| 16 | +--> |
| 17 | + |
| 18 | +# Named upstream connection-pool groups |
| 19 | + |
| 20 | +## Overview |
| 21 | + |
| 22 | +A divide rule can select one of a finite set of named upstream connection-pool |
| 23 | +groups configured at gateway startup. Requests that use the same upstream origin |
| 24 | +but belong to different workloads can therefore use independent bounded |
| 25 | +connection and acquisition pools. A slow bulk workload cannot consume the |
| 26 | +connection/acquisition capacity reserved for an interactive workload merely |
| 27 | +because both call the same backend host. |
| 28 | + |
| 29 | +This is an opt-in feature. With no `shenyu.httpclient.pool-groups` configured, |
| 30 | +the gateway keeps its existing single shared pool and request behavior is |
| 31 | +unchanged. |
| 32 | + |
| 33 | +## Configuration |
| 34 | + |
| 35 | +Pool groups live under the existing `shenyu.httpclient` prefix, next to the |
| 36 | +shared `pool` settings. Only the `FIXED` pool type is accepted for a group; each |
| 37 | +group is bounded by its own connection limit and pending-acquire queue. |
| 38 | + |
| 39 | +```yaml |
| 40 | +shenyu: |
| 41 | + httpclient: |
| 42 | + pool-groups: |
| 43 | + - name: bulk |
| 44 | + type: FIXED |
| 45 | + maxConnections: 32 |
| 46 | + acquireTimeout: 500 |
| 47 | + maxPendingAcquires: 32 |
| 48 | + - name: interactive |
| 49 | + type: FIXED |
| 50 | + maxConnections: 256 |
| 51 | + acquireTimeout: 3000 |
| 52 | +``` |
| 53 | +
|
| 54 | +Rules for the group list: |
| 55 | +
|
| 56 | +- At most 16 groups. |
| 57 | +- `name` is required and unique (case-insensitive). It must not be |
| 58 | + `shenyu-proxy`, the name of the shared default pool. |
| 59 | +- `type` must be `FIXED`. |
| 60 | +- `maxConnections` must be strictly positive. |
| 61 | +- `acquireTimeout` must not be negative. |
| 62 | +- `maxPendingAcquires` must be omitted or strictly positive; `-1` and `0` are |
| 63 | + rejected for groups. The shared default pool keeps its historical behavior. |
| 64 | +- TLS, proxy, keep-alive, idle handlers, HTTP/2 negotiation and the shared |
| 65 | + Netty loop resources are inherited from `shenyu.httpclient`, exactly like the |
| 66 | + default client. |
| 67 | +- The group set is read only at startup. Adding, removing or changing a group |
| 68 | + requires a gateway restart. |
| 69 | + |
| 70 | +To select a group, set `poolGroup` in the divide rule handle: |
| 71 | + |
| 72 | +```json |
| 73 | +{ |
| 74 | + "loadBalance": "random", |
| 75 | + "retry": 3, |
| 76 | + "poolGroup": "bulk" |
| 77 | +} |
| 78 | +``` |
| 79 | + |
| 80 | +The dashboard form does not expose `poolGroup` yet; use the Admin rule API or |
| 81 | +the handle JSON directly. |
| 82 | + |
| 83 | +## Request behavior |
| 84 | + |
| 85 | +The divide plugin publishes the selected group on the exchange before the HTTP |
| 86 | +client runs. The Netty and WebClient strategies both resolve that group once per |
| 87 | +attempt and use the group's client. The selection is pinned for the whole |
| 88 | +request lifetime, including retry/failover attempts and the response body; a |
| 89 | +rule update affects future requests only. |
| 90 | + |
| 91 | +An unknown group name, a registry that has already been closed, or a rule that |
| 92 | +selects a group on a gateway where the group is not configured produces HTTP |
| 93 | +503 for that request. There is no silent fallback to another group or to the |
| 94 | +shared default pool. Requests without `poolGroup` always take the shared |
| 95 | +default client. |
| 96 | + |
| 97 | +Group selection is per gateway process. Different gateways may configure |
| 98 | +different group names; a rule that references a group must only be published to |
| 99 | +gateways that define it. The limits are per group provider and per origin |
| 100 | +inside that provider, not cluster-wide quotas and not a request rate limiter. |
| 101 | +With HTTP/2, one connection multiplexes concurrent streams, so the isolation is |
| 102 | +connection/acquisition capacity rather than one connection per RPC. |
| 103 | + |
| 104 | +## Metrics |
| 105 | + |
| 106 | +Set `metrics: true` on a group to publish its Reactor Netty provider meters: |
| 107 | + |
| 108 | +```yaml |
| 109 | + - name: bulk |
| 110 | + type: FIXED |
| 111 | + maxConnections: 32 |
| 112 | + metrics: true |
| 113 | +``` |
| 114 | + |
| 115 | +The meters are native Reactor Netty Micrometer meters registered in the global |
| 116 | +Micrometer registry, for example: |
| 117 | + |
| 118 | +- `reactor.netty.connection.provider.active.connections` |
| 119 | +- `reactor.netty.connection.provider.idle.connections` |
| 120 | +- `reactor.netty.connection.provider.pending.connections` |
| 121 | +- `reactor.netty.connection.provider.max.pending.connections` |
| 122 | +- `reactor.netty.connection.provider.pending.connections.time` |
| 123 | + |
| 124 | +The provider name (the group name) is carried in the meter tags, so the metric |
| 125 | +cardinality is bounded by the configured groups and discovered origins. This |
| 126 | +requires `micrometer-core` on the classpath; the shipped bootstrap gets it |
| 127 | +through Spring Boot Actuator. It is not exported by the ShenYu Prometheus |
| 128 | +`MetricsRegister` endpoint in this delivery. To scrape the meters, add a |
| 129 | +Micrometer registry (for example `micrometer-registry-prometheus`) and expose |
| 130 | +it through your Micrometer/Actuator setup. |
| 131 | + |
| 132 | +There is no separate Reactor Netty counter for a rejected acquisition. Use the |
| 133 | +pending queue depth, its configured maximum, and the pending-acquire timer's |
| 134 | +outcome to observe waiting and rejection behavior. |
| 135 | + |
| 136 | +## Upgrade, migration and rollback |
| 137 | + |
| 138 | +The feature is additive. Rolling upgrades should leave `pool-groups` absent on |
| 139 | +the new version until all gateways can serve the rules that reference a group; |
| 140 | +older gateways ignore an unknown `poolGroup` handle field. To roll back, remove |
| 141 | +`poolGroup` from the affected rule handles and restart the gateways without the |
| 142 | +`pool-groups` block. Removing the configuration restores the shared default pool |
| 143 | +and the previous behavior. |
| 144 | + |
| 145 | +## Fixed-workload isolation measurement |
| 146 | + |
| 147 | +`UpstreamPoolGroupIsolationTest` covers correctness. For a repeatable capacity |
| 148 | +measurement, the test sources also contain an opt-in benchmark that compares one |
| 149 | +shared provider with two independent providers against the same loopback origin |
| 150 | +(8 bulk clients on a 50 ms endpoint, 2 interactive clients on a fast endpoint, |
| 151 | +5 seconds per scenario): |
| 152 | + |
| 153 | +```bash |
| 154 | +mvn -pl shenyu-plugin/shenyu-plugin-httpclient -am test \ |
| 155 | + -Dtest=UpstreamPoolGroupIsolationBenchmark \ |
| 156 | + -Dshenyu.poolgroup.benchmark=true -DfailIfNoTests=false |
| 157 | +``` |
| 158 | + |
| 159 | +Example output on Windows 10, JDK 17.0.18, 16 cores (fixed loopback workload, |
| 160 | +not a universal p50/p99 claim): |
| 161 | + |
| 162 | +```text |
| 163 | +scenario,bulk_ok,bulk_fail,interactive_ok,interactive_fail,interactive_p50_ms,interactive_p99_ms |
| 164 | +shared,274,173293,85,82980,2,815 |
| 165 | +groups,338,8,28774,0,0,1 |
| 166 | +``` |
| 167 | + |
| 168 | +The shared scenario is one provider of 4 connections shared by both workloads; |
| 169 | +the groups scenario uses a bulk provider of 4 and an interactive provider of 8. |
| 170 | +The interactive workload is effectively starved in the shared scenario and |
| 171 | +unaffected in the grouped scenario, which is the fixed-workload evidence for |
| 172 | +the isolation boundary. Re-run the command on your deployment host before |
| 173 | +quoting latency numbers; results include loopback and JIT effects. |
| 174 | + |
| 175 | +## Troubleshooting |
| 176 | + |
| 177 | +| Symptom | Check | |
| 178 | +| --- | --- | |
| 179 | +| HTTP 503 with an unknown pool group message | The rule references a group not defined on this gateway, or the registry is shutting down. Check the exact group name and the gateway's startup configuration. | |
| 180 | +| Startup fails with a group validation message | Use `FIXED`, a positive `maxConnections`, an explicit unique name other than `shenyu-proxy`, and omit or use a positive `maxPendingAcquires`. | |
| 181 | +| Startup fails because a custom `HttpClient` bean is present | `pool-groups` owns its clients; remove the custom `HttpClient` bean or the group configuration. | |
| 182 | +| Metrics are missing | Set `metrics: true` on the group, ensure `micrometer-core` is present, and verify the meter is exported by your Micrometer registry. | |
0 commit comments