You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
proposal: x/net/http2: support for WebSockets over HTTP/2 #49918
WebSockets over HTTP2 have been standardized as RFC 8441 and Firefox and Chromium supports that. Thus, I suggest that support is added for Websockets over HTTP2. The server should also be able to specify using HTTP/2 SETTINGS parameter that it supports Websockets over HTTP2.
Reacted by Ahmed W., andig, Eligio Mariño, DeedleFake, Antonio Ojea, Sam Batschelet, dozn, Mikhail Mazurskiy, Nikolay Tonev, Hannes and 10 more
Implement the SETTINGS_ENABLE_CONNECT_PROTOCOL SETTINGS
Requires adding an HTTP2 server option to enable the setting
Implement the The Extended CONNECT Method
This requires adding a new field to the requests
o A new pseudo-header field :protocol MAY be included on request
HEADERS indicating the desired protocol to be spoken on the tunnel
created by CONNECT. The pseudo-header field is single valued and
contains a value from the "Hypertext Transfer Protocol (HTTP)
Upgrade Token Registry" located at
<https://www.iana.org/assignments/http-upgrade-tokens/>
and adding the logic to use the field if SETTINGS_ENABLE_CONNECT_PROTOCOL has been received.
I think that this will only require adding a new option on the http2 server to enable the SETTINGS_ENABLE_CONNECT_PROTOCOL option and another one in the client to let the user know if that setting is enable.
Considering the gorilla/websocket library is currently unmaintained, which is causing a lot of concern in the community, I think this is a great opportunity to solidify and complete websocket support in the standard library
My main concern is that golang seems to avoid implementing the CONNECT semantics in the stdlib #22554 (comment), leaving the implementation to the users.
The requirement of the Extended CONNECT Method will require to add new CONNECT semantics (including a new pseudoheader) to the stdlib, I don't know what the golang team think about this 😄
I've started poking at this a bit and it seems that a resolution will require two patches: one to the x/net/http2 package to allow stream oriented interaction with an http2 stream, and then a followup change to x/net/websocket to add the actual websocket support. I've talked about this a bit on the go-nuts mailing list, but it really should be a public discussion, so I'm moving it here. Here's the most recent message I posted about a strawman patch I've put together:
I've got something working for exposing HTTP/2 streams as Stream struct that is spiritually an io.ReadWriteCloser on the client side, and using existing interfaces on the server side. I'm posting about it here to give people a chance to share initial thoughts before I write up something more formal. I've posted my changes on gerrit in case anyone wants to look at the details.
On the server side of things, it seems that the current interface is basically already adequate except for the fact that the server currently rejects any requests using non https/http schemes. The http.ResponseWriter that gets passed down into HTTP/2 handlers implements http.Flusher, allowing you to force writes out on to the wire, and you can just use the http.Request.Body as the reader for the stream. The right thing to do here seems relatively straightforward. We can do one of a few things:
Convert the scheme check to a regex check based on RFC3986 3.1. The spec does say in 8.1.2.3 that "All HTTP/2 requests MUST include exactly one valid value for the ":method", ":scheme", and ":path" pseudo-header fields, ... An HTTP request that omits mandatory pseudo-header fields is malformed (Section 8.1.2.6)." (emphasis mine), but in the same section it also says '":scheme" is not restricted to "http" and "https" schemed URIs.'. I think this should be read to mean that the only check we should be performing on scheme is that it parses correctly according to RFC3986 3.1. This is the option I think makes the most sense.
Just expand the allowlist in the check so that it includes wss and ws as well as http and https. This is a more conservative option and I think can be defended on the basis that the http library isn't generally expected to serve as a proxy, so a fixed list of allowed schemes is fine.
Drop the check entirely as I've done in my strawman patch. This seems wrong.
In terms of the client side of things, I'm relatively happy with the interface for the Stream struct I added. It re-uses the http.Request and http.Response types to let people set and read headers in a familiar way, and is basically just an unpacked version of the RoundTrip routine that keeps the request open after the stream gets initially created and only closes it when the user explicitly asks for a close. There are some things that seem clunky about it to me though:
It's hanging off the http2.Transport struct. As a user I would expect something like this on the http.Client, so maybe it is better to put it there somehow. That does run into the issue of this routine not being available for HTTP/1.1 though.
It might be better not to re-use the http.Request/http.Response types since we aren't really making requests and responses. Maybe a more unpacked API that directly references headers and trailers makes more sense.
I always feel weird about performing IO operations without passing a context down. Currently Stream uses the context attached to the request passed in to open the stream, which seems fine, but maybe a more explicit API would be better.
I just noticed https://datatracker.ietf.org/doc/html/rfc8441#section-5 saying that for HTTP/2 wss:// URIs should be translated to https:// and ws:// URIs should be translated to http:// URIs, so it seems that we may not need to make any changes to the server side of the HTTP/2 stack after all.
We also need a way to set/get the :protocol pseudo-header. Maybe just put it in the Header map?
Unless we put some level of websocket bootstrap support directly into net/http, a websocket client is going to need to know whether it's going to be speaking HTTP/1 or HTTP/2, since the initial protocol bootstrap differs between the two. This is probably okay.
Yeah, I think those are all good points. The suggestion you made in the CL works with no changes to http2 for a GET request, but runs into SETTINGS_ENABLE_CONNECT_PROTOCOL trouble when I try to use CONNECT with it.
I've started poking at SETTINGS_ENABLE_CONNECT_PROTOCOL support a bit. It seems like it should be easy enough to just advertise it in the initial settings frame on the server side and then set it as a flag on the ClientConn on the server side. We probably don't even need to expose it to users on the client side, we could just check the flag and return an error if they try to send a CONNECT message and the setting is not enabled.
Currently, the client seems to clobber the scheme and path when you try sending a CONNECT request, which the server doesn't much like, so I'm trying to figure that out. I'm a little worried about the compatibility of no longer clobbering those parts of the URI though, since users might have come to rely on that behavior. Do you know what the reason for clobbering it like this is?
The SETTINGS_ENABLE_CONNECT_PROTOCOL parameter changes the semantics of CONNECT.
ClientConn.encodeHeaders should handle CONNECT requests with a :protocol pseudo-header differently from ones without--in particular, it should return an error if the server didn't set SETTINGS_ENABLE_CONNECT_PROTOCOL, and it should include the :path and :scheme if the server did.
Ah, that makes sense. I had thought it was just an advertisement that the server knew how to handle the connect protocol, but a mode switch makes more sense. Then it probably sense to just add an option to http2.Server to tell it to send the ENABLE_CONNECT_PROTOCOL setting and have the client check the setting to determine if it ought to clobber the scheme and path.
Activity
aojea commentedon Dec 21, 2021
also related #27244
aojea commentedon Dec 21, 2021
Just for reference https://datatracker.ietf.org/doc/html/rfc8441 it seems this requires (please correct me if I'm wrong, not very familiar with the stdlib implemententation):
Requires adding an HTTP2 server option to enable the setting
This requires adding a new field to the requestsand adding the logic to use the field if SETTINGS_ENABLE_CONNECT_PROTOCOL has been received.
EDIT1 :/
go/src/net/http/request.go
Lines 107 to 109 in 90fb5a4
EDIT2, so this seems to be on purpose
#22554 (comment)
aojea commentedon Dec 22, 2021
I think that this will only require adding a new option on the http2 server to enable the SETTINGS_ENABLE_CONNECT_PROTOCOL option and another one in the client to let the user know if that setting is enable.
If I understand correctly how CONNECT works in golang, the rest of the logic can be done at a higher level in a similar way that is being done with https://github.com/golang/net/tree/master/websocket
JeremyLoy commentedon Jan 7, 2022
Considering the gorilla/websocket library is currently unmaintained, which is causing a lot of concern in the community, I think this is a great opportunity to solidify and complete websocket support in the standard library
aojea commentedon Jan 9, 2022
I did a prototype https://github.com/aojea/net/pull/1/files to get a better idea on what will be required to implement this.
My main concern is that golang seems to avoid implementing the CONNECT semantics in the stdlib #22554 (comment), leaving the implementation to the users.
The requirement of the
Extended CONNECT Methodwill require to add new CONNECT semantics (including a new pseudoheader) to the stdlib, I don't know what the golang team think about this 😄hexfusion commentedon Feb 21, 2022
How can we move this forward? I think many folks are interested in the implementation of rfc8441 it has been a few years[1].
@neild could you possibly help direct us on next steps?
[1] #32763
ethanpailes commentedon May 26, 2022
I've started poking at this a bit and it seems that a resolution will require two patches: one to the x/net/http2 package to allow stream oriented interaction with an http2 stream, and then a followup change to x/net/websocket to add the actual websocket support. I've talked about this a bit on the go-nuts mailing list, but it really should be a public discussion, so I'm moving it here. Here's the most recent message I posted about a strawman patch I've put together:
I've got something working for exposing HTTP/2 streams as
Streamstruct that is spiritually anio.ReadWriteCloseron the client side, and using existing interfaces on the server side. I'm posting about it here to give people a chance to share initial thoughts before I write up something more formal. I've posted my changes on gerrit in case anyone wants to look at the details.On the server side of things, it seems that the current interface is basically already adequate except for the fact that the server currently rejects any requests using non
https/httpschemes. Thehttp.ResponseWriterthat gets passed down into HTTP/2 handlers implementshttp.Flusher, allowing you to force writes out on to the wire, and you can just use thehttp.Request.Bodyas the reader for the stream. The right thing to do here seems relatively straightforward. We can do one of a few things:schemeis that it parses correctly according to RFC3986 3.1. This is the option I think makes the most sense.wssandwsas well ashttpandhttps. This is a more conservative option and I think can be defended on the basis that the http library isn't generally expected to serve as a proxy, so a fixed list of allowed schemes is fine.In terms of the client side of things, I'm relatively happy with the interface for the
Streamstruct I added. It re-uses thehttp.Requestandhttp.Responsetypes to let people set and read headers in a familiar way, and is basically just an unpacked version of the RoundTrip routine that keeps the request open after the stream gets initially created and only closes it when the user explicitly asks for a close. There are some things that seem clunky about it to me though:http2.Transportstruct. As a user I would expect something like this on thehttp.Client, so maybe it is better to put it there somehow. That does run into the issue of this routine not being available for HTTP/1.1 though.http.Request/http.Responsetypes since we aren't really making requests and responses. Maybe a more unpacked API that directly references headers and trailers makes more sense.Streamuses the context attached to the request passed in to open the stream, which seems fine, but maybe a more explicit API would be better.cc: @neild @ianlancetaylor
ethanpailes commentedon May 27, 2022
I just noticed https://datatracker.ietf.org/doc/html/rfc8441#section-5 saying that for HTTP/2
wss://URIs should be translated tohttps://andws://URIs should be translated tohttp://URIs, so it seems that we may not need to make any changes to the server side of the HTTP/2 stack after all.neild commentedon May 27, 2022
Commented on https://go.dev/cl/408835: Do we need a new stream-oriented
TransportAPI?RoundTripalready keeps the request stream open so long as the request or response body are still being written.We'd need to add support for
SETTINGS_ENABLE_CONNECT_PROTOCOL. That should be straightforward.We also need a way to set/get the
:protocolpseudo-header. Maybe just put it in theHeadermap?Unless we put some level of websocket bootstrap support directly into
net/http, a websocket client is going to need to know whether it's going to be speaking HTTP/1 or HTTP/2, since the initial protocol bootstrap differs between the two. This is probably okay.ethanpailes commentedon May 27, 2022
Yeah, I think those are all good points. The suggestion you made in the CL works with no changes to
http2for aGETrequest, but runs intoSETTINGS_ENABLE_CONNECT_PROTOCOLtrouble when I try to useCONNECTwith it.I've started poking at SETTINGS_ENABLE_CONNECT_PROTOCOL support a bit. It seems like it should be easy enough to just advertise it in the initial settings frame on the server side and then set it as a flag on the ClientConn on the server side. We probably don't even need to expose it to users on the client side, we could just check the flag and return an error if they try to send a CONNECT message and the setting is not enabled.
Currently, the client seems to clobber the scheme and path when you try sending a
CONNECTrequest, which the server doesn't much like, so I'm trying to figure that out. I'm a little worried about the compatibility of no longer clobbering those parts of the URI though, since users might have come to rely on that behavior. Do you know what the reason for clobbering it like this is?neild commentedon May 27, 2022
HTTP/2
CONNECTexplicitly doesn't allow the:schemeand:pathpseudo-headers:https://datatracker.ietf.org/doc/html/rfc7540#section-8.3
The
SETTINGS_ENABLE_CONNECT_PROTOCOLparameter changes the semantics ofCONNECT.ClientConn.encodeHeadersshould handleCONNECTrequests with a:protocolpseudo-header differently from ones without--in particular, it should return an error if the server didn't setSETTINGS_ENABLE_CONNECT_PROTOCOL, and it should include the:pathand:schemeif the server did.ethanpailes commentedon May 27, 2022
Ah, that makes sense. I had thought it was just an advertisement that the server knew how to handle the connect protocol, but a mode switch makes more sense. Then it probably sense to just add an option to
http2.Serverto tell it to send the ENABLE_CONNECT_PROTOCOL setting and have the client check the setting to determine if it ought to clobber the scheme and path.neild commentedon May 27, 2022
I'm not sure we even need an option; maybe just always send
SETTINGS_ENABLE_CONNECT_PROTOCOL.42 remaining items
mchandler-plato commentedon Dec 2, 2024
Sent you an email with details. Thanks!
gopherbot commentedon Dec 3, 2024
Change https://go.dev/cl/632899 mentions this issue:
websocket: add http2 support using extended connectgopherbot commentedon Jan 8, 2025
Change https://go.dev/cl/641475 mentions this issue:
http2: disable extended CONNECT by defaulthttp2: disable extended CONNECT by default
gopherbot commentedon Jan 17, 2025
Change https://go.dev/cl/642606 mentions this issue:
[internal-branch.go1.24-vendor] http2: disable extended CONNECT by default[internal-branch.go1.24-vendor] http2: disable extended CONNECT by de…
fix: disable HTTP/2 on origin so WebSocket endpoints work behind Clou…
re-apply: disable HTTP/2 on origin so WebSocket endpoints work behind…
fix: disable HTTP/2 on origin so WebSocket endpoints work behind Clou…
re-apply: disable HTTP/2 on origin so WebSocket endpoints work behind…