Repository navigation
Commit 775f017
feat(python-sdk): use pyqwest native HTTP/2 connection balancing (#1931)
## Summary
[curioswitch/pyqwest#246](curioswitch/pyqwest#246),
released in pyqwest 0.12.0, adds client-side load balancing to pyqwest's
connection pool. It spreads an origin's HTTP/2 streams across
connections by free stream capacity (taken from the server's
`SETTINGS_MAX_CONCURRENT_STREAMS`) and dials another connection only
once every existing one is full. `max_connections_per_address` caps the
number of connections per resolved address. Our Python-side
`ConnectionBalancer` / `BalancingTransport` did the same job by fanning
out over several pyqwest transports, so this PR removes them and uses
one native transport.
```python
# before: ConnectionRetryTransport(BalancingTransport(ConnectionBalancer(build)))
# after
ConnectionRetryTransport(
SyncHTTPTransport(..., max_connections_per_address=max_connections), # HTTPTransport for async
max_retries=connection_retries,
)
```
- The dependency moves to `pyqwest>=0.12.0,<0.13`. pyqwest 0.12.0
requires Python 3.11 or newer, so this PR is based on
#1974, whose commit `chore(python):
require Python 3.11` is included here. Merge #1974 first, then rebase
this branch onto main. The `code-interpreter-python` and
`desktop-python` lockfiles are relocked because they resolve `e2b` from
the workspace path.
- `E2B_MAX_CONNECTIONS` (default `200`) now maps to
`max_connections_per_address`. Once the cap is reached, a request waits
for a free stream on an existing connection. Previously it piled onto
the least-loaded connection, which had the same effect.
- `E2B_STREAMS_PER_CONNECTION` is removed. The per-connection stream
limit now comes from the server (the envd edge advertises 100).
- Removed `ConnectionBalancer`, `PooledConnection`, `request_origin`,
`BalancingTransport` and `_TrackedContent`. Slot accounting for a
response's lifetime is now done by pyqwest: it keeps the hyper-util
connection lease in the response extensions until the body completes or
is closed.
- `test_envd_stream_capacity.py` is rewritten to check the native pool
against the local HTTP/2 frame server, for both sync and async:
- With a 100-stream limit, waves of 80 streams use 1, then 2, then 3
connections, and freed streams are reused.
- With a cap of 2 connections × 10 streams, the 21st request waits until
a stream closes.
- The balancer unit tests are removed.
Link to Devin session:
https://app.devin.ai/sessions/81af1a093fda4f22a89e3c1e4d78047c
Open in Devin Desktop:
https://app.devin.ai/desktop/session/81af1a093fda4f22a89e3c1e4d78047c?variant=devin
Requested by: @mishushakov
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: [email protected] <[email protected]>
Co-authored-by: Mish Ushakov <[email protected]>1 parent 547decc commit 775f017
13 files changed
Lines changed: 341 additions & 840 deletions
File tree
- .changeset
- packages
- code-interpreter-python
- desktop-python
- python-sdk
- e2b/api
- client_async
- client_sync
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
68 | 68 | | |
69 | 69 | | |
70 | 70 | | |
71 | | - | |
| 71 | + | |
72 | 72 | | |
73 | | - | |
| 73 | + | |
74 | 74 | | |
75 | 75 | | |
76 | | - | |
| 76 | + | |
77 | 77 | | |
78 | 78 | | |
79 | 79 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
5 | 4 | | |
6 | 5 | | |
7 | 6 | | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | 7 | | |
13 | 8 | | |
14 | 9 | | |
15 | 10 | | |
16 | | - | |
17 | 11 | | |
18 | 12 | | |
19 | | - | |
| 13 | + | |
20 | 14 | | |
21 | 15 | | |
22 | 16 | | |
| |||
95 | 89 | | |
96 | 90 | | |
97 | 91 | | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | | - | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
108 | 98 | | |
109 | 99 | | |
110 | | - | |
111 | | - | |
112 | | - | |
113 | | - | |
114 | | - | |
115 | | - | |
116 | | - | |
117 | | - | |
118 | | - | |
119 | | - | |
120 | | - | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | | - | |
127 | | - | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
140 | | - | |
141 | | - | |
142 | | - | |
143 | | - | |
144 | | - | |
145 | | - | |
146 | | - | |
147 | | - | |
148 | | - | |
149 | | - | |
150 | | - | |
151 | | - | |
152 | | - | |
153 | | - | |
154 | | - | |
155 | | - | |
156 | | - | |
157 | | - | |
158 | | - | |
159 | | - | |
160 | | - | |
161 | | - | |
162 | | - | |
163 | | - | |
164 | | - | |
165 | | - | |
166 | | - | |
167 | | - | |
168 | | - | |
169 | | - | |
170 | 100 | | |
171 | 101 | | |
172 | 102 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
3 | | - | |
| 2 | + | |
4 | 3 | | |
5 | 4 | | |
6 | 5 | | |
| |||
11 | 10 | | |
12 | 11 | | |
13 | 12 | | |
14 | | - | |
15 | 13 | | |
16 | 14 | | |
| 15 | + | |
17 | 16 | | |
18 | 17 | | |
19 | 18 | | |
20 | 19 | | |
21 | | - | |
22 | 20 | | |
23 | 21 | | |
24 | 22 | | |
| |||
61 | 59 | | |
62 | 60 | | |
63 | 61 | | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | | - | |
80 | | - | |
81 | | - | |
82 | | - | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
87 | | - | |
88 | | - | |
89 | | - | |
90 | | - | |
91 | | - | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | | - | |
108 | | - | |
109 | | - | |
110 | | - | |
111 | | - | |
112 | | - | |
113 | | - | |
114 | | - | |
115 | | - | |
116 | | - | |
117 | | - | |
118 | | - | |
119 | | - | |
120 | | - | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | | - | |
127 | | - | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
140 | 62 | | |
141 | 63 | | |
142 | 64 | | |
143 | 65 | | |
144 | 66 | | |
145 | 67 | | |
146 | | - | |
| 68 | + | |
147 | 69 | | |
148 | 70 | | |
149 | 71 | | |
| |||
153 | 75 | | |
154 | 76 | | |
155 | 77 | | |
156 | | - | |
| 78 | + | |
157 | 79 | | |
158 | | - | |
| 80 | + | |
159 | 81 | | |
160 | 82 | | |
161 | 83 | | |
| |||
169 | 91 | | |
170 | 92 | | |
171 | 93 | | |
172 | | - | |
173 | | - | |
174 | | - | |
175 | | - | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
176 | 99 | | |
177 | 100 | | |
178 | 101 | | |
179 | 102 | | |
180 | 103 | | |
181 | | - | |
| 104 | + | |
182 | 105 | | |
183 | | - | |
| 106 | + | |
184 | 107 | | |
185 | 108 | | |
186 | 109 | | |
| |||
193 | 116 | | |
194 | 117 | | |
195 | 118 | | |
196 | | - | |
197 | | - | |
| 119 | + | |
| 120 | + | |
198 | 121 | | |
199 | 122 | | |
200 | | - | |
201 | | - | |
202 | | - | |
203 | | - | |
204 | | - | |
205 | | - | |
206 | | - | |
207 | | - | |
208 | | - | |
209 | | - | |
210 | | - | |
211 | | - | |
212 | | - | |
213 | | - | |
214 | | - | |
215 | | - | |
216 | 123 | | |
217 | 124 | | |
218 | 125 | | |
219 | | - | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
220 | 141 | | |
221 | 142 | | |
222 | 143 | | |
| |||
233 | 154 | | |
234 | 155 | | |
235 | 156 | | |
236 | | - | |
| 157 | + | |
237 | 158 | | |
238 | | - | |
| 159 | + | |
239 | 160 | | |
240 | 161 | | |
241 | 162 | | |
| |||
0 commit comments