Repository navigation
Commit 905ca61
authored
[serve] Prevent ASGI
See #44644 for the user-reported issue.
In #43713, I changed the behavior of the ASGI receive interface to not raise an exception but rather silently hang if the request was not present in the proxy. This was done under the assumption that the code calling receive would be subsequently cancelled anyways. However, this assumption was faulty for a few reasons:
- The user may ignore the CancelledError but still want to fetch ASGI messages.
- Cancellation is best-effort. Specifically, because we are currently using the run_coroutine_threadsafe / concurrent.futures.Executor interface, there is no way to guarantee that either the scheduling task is cancelled or the resulting object ref is cancelled. In rare cases, neither may happen (see asyncio.run_coroutine_threadsafe leaves underlying cancelled asyncio task running python/cpython#105836).
This PR mitigates the issue by returning an appropriate disconnect ASGI message when the proxy no longer has the request in scope.
Note that this is not a perfect solution: we may still drop the messages prior to the disconnect (e.g., the body of the HTTP request) if the replica does not fetch them prior to the client disconnecting.
- Note that this is not a regression from the current behavior and should only happen in practice if the client disconnects very quickly after initiating the request.
- As a follow-up, we should rework this codepath to something more robust. For example we could use a push-based model where the proxy instead pushes messages eagerly to the replica (this was originally very difficult to implement but should be possible now).
---------
Signed-off-by: Edward Oakes <[email protected]>receive() from hanging on disconnects (#44647)1 parent 7364705 commit 905ca61
3 files changed
Lines changed: 60 additions & 14 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
217 | 217 | | |
218 | 218 | | |
219 | 219 | | |
| 220 | + | |
220 | 221 | | |
221 | 222 | | |
222 | 223 | | |
| 224 | + | |
223 | 225 | | |
224 | 226 | | |
225 | 227 | | |
226 | 228 | | |
227 | 229 | | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
228 | 248 | | |
229 | 249 | | |
230 | 250 | | |
| |||
244 | 264 | | |
245 | 265 | | |
246 | 266 | | |
247 | | - | |
| 267 | + | |
248 | 268 | | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
249 | 272 | | |
250 | 273 | | |
251 | 274 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
351 | 351 | | |
352 | 352 | | |
353 | 353 | | |
| 354 | + | |
| 355 | + | |
354 | 356 | | |
355 | 357 | | |
356 | 358 | | |
| |||
942 | 944 | | |
943 | 945 | | |
944 | 946 | | |
| 947 | + | |
945 | 948 | | |
| 949 | + | |
946 | 950 | | |
947 | 951 | | |
948 | 952 | | |
949 | 953 | | |
950 | 954 | | |
951 | 955 | | |
952 | 956 | | |
953 | | - | |
| 957 | + | |
954 | 958 | | |
955 | 959 | | |
956 | 960 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
| 3 | + | |
3 | 4 | | |
4 | 5 | | |
5 | 6 | | |
| |||
65 | 66 | | |
66 | 67 | | |
67 | 68 | | |
68 | | - | |
69 | | - | |
70 | | - | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
71 | 75 | | |
72 | 76 | | |
73 | 77 | | |
| |||
80 | 84 | | |
81 | 85 | | |
82 | 86 | | |
83 | | - | |
| 87 | + | |
84 | 88 | | |
85 | 89 | | |
86 | 90 | | |
| |||
125 | 129 | | |
126 | 130 | | |
127 | 131 | | |
128 | | - | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
129 | 138 | | |
130 | 139 | | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
131 | 146 | | |
132 | 147 | | |
133 | 148 | | |
| |||
136 | 151 | | |
137 | 152 | | |
138 | 153 | | |
139 | | - | |
140 | | - | |
141 | | - | |
142 | | - | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
143 | 162 | | |
144 | 163 | | |
145 | 164 | | |
146 | 165 | | |
147 | 166 | | |
148 | 167 | | |
149 | | - | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
150 | 171 | | |
151 | 172 | | |
152 | 173 | | |
| |||
157 | 178 | | |
158 | 179 | | |
159 | 180 | | |
160 | | - | |
161 | | - | |
162 | 181 | | |
0 commit comments