Natively support gRPC on the docker socket by dmcgowan · Pull Request #50744 · moby/moby

dmcgowan · 2025-08-15T05:58:29Z

relates to / addresses Replace or update the grpc endpoint #49836
relates to Replace or update the grpc endpoint buildkit#5927

Adds support for directly serving the gRPC server on the Docker socket. The requests can be easily detected from the content-type and use of HTTP 2. Explicitly adds the HTTP 2 protocol when using tcp+tls or unencrypted HTTP 2 when using a unix socket.

Only registers the buildkit service, which will allow deprecating the hijacking /session and /grpc endpoints. In the future, we can register more services using containerd style plugins.

- Human readable description for the release notes

Natively support gRPC on the listening socket

thaJeztah · 2025-08-15T08:39:56Z

integration/build/build_traces_test.go

-		client.WithSessionDialer(func(ctx context.Context, proto string, meta map[string][]string) (net.Conn, error) {
-			return c.DialHijack(ctx, "/session", proto, meta)
-		}),
-		client.WithContextDialer(func(ctx context.Context, _ string) (net.Conn, error) {
-			return c.DialHijack(ctx, "/grpc", "h2c", nil)
-		}),


Do we need to keep the /grpc and /session endpoints? IIUC removing these means that any existing version of buildx or the CLI will break?

Oh, ignore me; this is client-side 🙈

I guess clients should still try both to stay compatible, right?

docker/buildx#3369 handles both, I don't know if any other clients are using and if we can just safely deprecate those endpoints since using them requires a very delicate dance.

Basically any version of buildx that currently exists; i.e., we can't remove the old endpoints, as they're part of the API

Could we use HTTP 301 Moved Permanently for legacy clients to switch to the updated endpoint?

FWIW; this is only the test, not the endpoint being removed (🙈); so probably fine, but we can consider officially deprecating the old endpoints

thaJeztah · 2025-08-15T09:00:56Z

This failure looks potentially related;

=== RUN   TestDockerDaemonSuite/TestTLSVerify
    docker_cli_daemon_test.go:708: Daemon should not have started due to missing certs: exit status 2
        panic: runtime error: invalid memory address or nil pointer dereference
        [signal SIGSEGV: segmentation violation code=0x1 addr=0x70 pc=0x55aaeba7cbfa]

Details

=== RUN   TestDockerDaemonSuite/TestTLSVerify
    docker_cli_daemon_test.go:708: Daemon should not have started due to missing certs: exit status 2
        panic: runtime error: invalid memory address or nil pointer dereference
        [signal SIGSEGV: segmentation violation code=0x1 addr=0x70 pc=0x55aaeba7cbfa]
        
        goroutine 1 [running, locked to thread]:
        github.com/moby/moby/v2/daemon/command.newAPIServerTLSConfig(0xc0001db700?)
        	/go/src/github.com/docker/docker/daemon/command/daemon.go:822 +0x15a
        github.com/moby/moby/v2/daemon/command.newDaemonCLI(0xc0001db700)
        	/go/src/github.com/docker/docker/daemon/command/daemon.go:90 +0x33
        github.com/moby/moby/v2/daemon/command.newDaemonCommand.func1(0xc000576008, {0xc0002d2940?, 0x7?, 0x55aaebafb48e?})
        	/go/src/github.com/docker/docker/daemon/command/docker.go:35 +0x65
        github.com/spf13/cobra.(*Command).execute(0xc000576008, {0xc0000500f0, 0x1, 0x1})
        	/go/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:1015 +0xaaa
        github.com/spf13/cobra.(*Command).ExecuteC(0xc000576008)
        	/go/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:1148 +0x46f
        github.com/spf13/cobra.(*Command).Execute(...)
        	/go/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:1071
        github.com/spf13/cobra.(*Command).ExecuteContext(...)
        	/go/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:1064
        github.com/moby/moby/v2/daemon/command.daemonRunner.Run({0x55aaec77d9d8?}, {0x55aaec7acfd0, 0x55aaede48200})
        	/go/src/github.com/docker/docker/daemon/command/docker.go:111 +0x6e
        main.main()
        	/go/src/github.com/docker/docker/cmd/dockerd/main.go:38 +0x126
        
        goroutine 8 [select]:
        go.opencensus.io/stats/view.(*worker).start(0xc0004dab00)
        	/go/src/github.com/docker/docker/vendor/go.opencensus.io/stats/view/worker.go:292 +0x9f
        created by go.opencensus.io/stats/view.init.0 in goroutine 1
        	/go/src/github.com/docker/docker/vendor/go.opencensus.io/stats/view/worker.go:34 +0x8d
        
        goroutine 43 [select]:
        io.(*pipe).read(0xc000113980, {0xc000610000, 0x10000, 0x0?})
        	/usr/local/go/src/io/pipe.go:57 +0xa5
        io.(*PipeReader).Read(0xc0004d9e08?, {0xc000610000?, 0x55aaede4b880?, 0xc0004d9e60?})
        	/usr/local/go/src/io/pipe.go:134 +0x1a
        bufio.(*Scanner).Scan(0xc0004d9f28)
        	/usr/local/go/src/bufio/scan.go:219 +0x81e
        github.com/sirupsen/logrus.(*Entry).writerScanner(0xc0001acf50, 0xc000113980, 0xc0002d2550)
        	/go/src/github.com/docker/docker/vendor/github.com/sirupsen/logrus/writer.go:86 +0x11d
        created by github.com/sirupsen/logrus.(*Entry).WriterLevel in goroutine 1
        	/go/src/github.com/docker/docker/vendor/github.com/sirupsen/logrus/writer.go:57 +0x31f
        
        goroutine 42 [select]:
        io.(*pipe).read(0xc000113920, {0xc00058c000, 0x10000, 0x0?})
        	/usr/local/go/src/io/pipe.go:57 +0xa5
        io.(*PipeReader).Read(0xc0004d8e08?, {0xc00058c000?, 0xc0005cc000?, 0xc0004d8e60?})
        	/usr/local/go/src/io/pipe.go:134 +0x1a
        bufio.(*Scanner).Scan(0xc0004d8f28)
        	/usr/local/go/src/bufio/scan.go:219 +0x81e
        github.com/sirupsen/logrus.(*Entry).writerScanner(0xc0001acf50, 0xc000113920, 0xc0002d2540)
        	/go/src/github.com/docker/docker/vendor/github.com/sirupsen/logrus/writer.go:86 +0x11d
        created by github.com/sirupsen/logrus.(*Entry).WriterLevel in goroutine 1
        	/go/src/github.com/docker/docker/vendor/github.com/sirupsen/logrus/writer.go:57 +0x31f
        
        goroutine 44 [select]:
        io.(*pipe).read(0xc0001139e0, {0xc000680000, 0x10000, 0x0?})
        	/usr/local/go/src/io/pipe.go:57 +0xa5
        io.(*PipeReader).Read(0xc0004d4608?, {0xc000680000?, 0x55aaede4b880?, 0xc0004d4660?})
        	/usr/local/go/src/io/pipe.go:134 +0x1a
        bufio.(*Scanner).Scan(0xc000094f28)
        	/usr/local/go/src/bufio/scan.go:219 +0x81e
        github.com/sirupsen/logrus.(*Entry).writerScanner(0xc0001acf50, 0xc0001139e0, 0xc0002d2570)
        	/go/src/github.com/docker/docker/vendor/github.com/sirupsen/logrus/writer.go:86 +0x11d
        created by github.com/sirupsen/logrus.(*Entry).WriterLevel in goroutine 1
        	/go/src/github.com/docker/docker/vendor/github.com/sirupsen/logrus/writer.go:57 +0x31f

thaJeztah · 2025-08-15T09:01:06Z

Looks like this requires updating the github.com/moby/moby/v2 module to use go1.24 as minimum;

INFO [runner] linters took 2m14.274822457s with stages: goanalysis_metalinter: 2m14.21043666s 
daemon/command/daemon.go:313:14: stdversion: http.Protocols requires go1.24 or later (file is go1.23) (govet)
		httpServer.Protocols = &p
		           ^
daemon/command/daemon.go:318:14: stdversion: http.Protocols requires go1.24 or later (file is go1.23) (govet)
		httpServer.Protocols = &p
		           ^
daemon/command/httphandler.go:73:84: stdversion: http.Protocols requires go1.24 or later (file is go1.23) (govet)
func supportGRPC(ctx context.Context, lss []net.Listener, hasTLS bool) (bool, http.Protocols) {
                                                                                   ^
daemon/command/httphandler.go:74:13: stdversion: http.Protocols requires go1.24 or later (file is go1.23) (govet)
	var p http.Protocols
	           ^

From #49836 (comment), I saw that @corhere originally suggested some approach through golang.org/x/net, but not sure if that would allow working without go1.24;

It would be trivial for us to support HTTP/2 with Prior Knowledge – and also properly support the deprecated Upgrade: h2c mechanism in a way that actually complies with RFC 7540 Section 3.2 by wrapping the daemon's HTTP handler in https://pkg.go.dev/golang.org/x/[email protected]/http2/h2c

Also;

#49836 (comment)

I suspected the Upgrade connected more directly with a grpc-go server, and that's why it needed to be POST specifically to /grpc

It's more straightforward than that. We implemented the h2c upgrade incorrectly and added it as a route to the HTTP server mux instead of wrapping the HTTP server mux like we should have.

thaJeztah · 2025-08-15T09:04:24Z

I pushed a commit to update the Go version in go.mod so that CI can run with that change.

corhere · 2025-08-15T10:55:26Z

100% using golang.org/x/net like I suggested would not require bumping the go version. And I would still prefer that approach over open-coding it again since we got it wrong open-coding it the first time around.

dmcgowan · 2025-08-15T18:16:27Z

100% using golang.org/x/net like I suggested would not require bumping the go version.

1.23 is deprecated, the min version will get bumped as soon as we vendor in a project which bumps it (containerd 2.2 or anything that has a transitive Kubernetes dep)

And I would still prefer that approach over open-coding it again since we got it wrong open-coding it the first time around.

Can you explain this one further. I usually think the preferred solution is using the standard library as the Go maintainers have encouraged not using those lower level libraries. "This package is low-level and intended to be used directly by very few people", we aren't doing anything that special here. Not sure what "over open-coding it again" refers to or how it relates to the existing hijack approach, which I think we all agree isn't great.

Use the GRPC server when requests are for the grpc content type. Signed-off-by: Derek McGowan <[email protected]>

akerouanton · 2025-12-11T08:57:35Z

I’ve rebased this PR and removed the supportGRPC function, which previously determined whether gRPC and h2c were enabled. h2c is now always enabled.

When a TLS listener receives a connection, it returns an error if the handshake can’t be completed. As a result, h2c won’t be used for TLS listeners, but it will be used for non-TLS TCP connections and the UNIX socket.

In the previous iteration, gRPC was disabled when running the dev container with the API exposed over TCP. This is now resolved.

vvoland

LGTM

vvoland · 2025-12-11T11:37:56Z

daemon/command/daemon.go

 	})
-	httpServer.Handler = apiServer.CreateMux(ctx, routers...)
+	gs := newGRPCServer(ctx)
+	b.backend.RegisterGRPC(gs)


Note: This will implicitly expose the containerd content store via buildkit:

moby/vendor/github.com/moby/buildkit/control/control.go

Line 164 in c64b781

store := &roContentStore{c.opt.ContentStore.WithFallbackNS(c.opt.ContentStore.Namespace() + "_history")}

It's read-only so that's good, but we should make that explicitly registered by the Moby and not implicitly by Buildkit.

@copilot can you open a ticket for this

thanks for nothing @copilot! I did it myself: #51698

thaJeztah

LGTM

dmcgowan mentioned this pull request Aug 15, 2025

Add support for native grpc on the docker socket docker/buildx#3369

Open

thaJeztah reviewed Aug 15, 2025

View reviewed changes

thaJeztah added area/api API impact/api kind/enhancement Enhancements are not bugs or new features but can improve usability or performance. labels Aug 15, 2025

dmcgowan force-pushed the add-grpc-support branch 2 times, most recently from 0f2fc4a to 8756712 Compare August 15, 2025 18:55

thompson-shaun added this to 🔦 Maintainer spotlight Aug 21, 2025

github-project-automation bot moved this to New in 🔦 Maintainer spotlight Aug 21, 2025

thompson-shaun moved this from New to Open in 🔦 Maintainer spotlight Aug 21, 2025

thompson-shaun modified the milestones: 29.0.0, 30.0.0, 29.1.0 Aug 21, 2025

thaJeztah mentioned this pull request Aug 26, 2025

ImageBuildOptions lacks parity with compose/BuildOptions #50804

Closed

dmcgowan linked an issue Aug 26, 2025 that may be closed by this pull request

Proposal: c8d: expose contentstore API #44369

Open

thompson-shaun moved this from Open to Complete in 🔦 Maintainer spotlight Sep 18, 2025

dmcgowan force-pushed the add-grpc-support branch from 8756712 to c8eb19a Compare October 17, 2025 19:45

github-actions bot added the area/daemon Core Engine label Oct 17, 2025

thaJeztah added the impact/changelog label Oct 30, 2025

vvoland modified the milestones: 29.1.0, v-future Nov 26, 2025

Natively support GRPC on the docker socket

d210449

Use the GRPC server when requests are for the grpc content type. Signed-off-by: Derek McGowan <[email protected]>

akerouanton force-pushed the add-grpc-support branch from c8eb19a to d210449 Compare December 11, 2025 08:47

akerouanton marked this pull request as ready for review December 11, 2025 08:57

akerouanton modified the milestones: v-future, 29.2.0 Dec 11, 2025

vvoland approved these changes Dec 11, 2025

View reviewed changes

vvoland moved this from Complete to New in 🔦 Maintainer spotlight Dec 11, 2025

thaJeztah approved these changes Dec 11, 2025

View reviewed changes

akerouanton merged commit 3cba626 into moby:master Dec 12, 2025
259 of 260 checks passed

This was referenced Dec 12, 2025

Deprecate /grpc and /session endpoints #51696

Closed

Replace or update the grpc endpoint #49836

Closed

vvoland mentioned this pull request Dec 12, 2025

Explicitly expose content store gRPC service #51698

Open

rafikk mentioned this pull request Dec 12, 2025

Proposal: c8d: expose contentstore API #44369

Open

vvoland added kind/feature Functionality or other elements that the project doesn't currently have. Features are new and shiny and removed kind/enhancement Enhancements are not bugs or new features but can improve usability or performance. labels Dec 16, 2025

thaJeztah moved this from New to Complete in 🔦 Maintainer spotlight Dec 18, 2025

sssalrawahi9-coder approved these changes Feb 11, 2026

View reviewed changes

Conversation

dmcgowan commented Aug 15, 2025 • edited by vvoland Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented Aug 15, 2025

Uh oh!

thaJeztah commented Aug 15, 2025

Uh oh!

thaJeztah commented Aug 15, 2025

Uh oh!

corhere commented Aug 15, 2025

Uh oh!

dmcgowan commented Aug 15, 2025

Uh oh!

akerouanton commented Dec 11, 2025

Uh oh!

vvoland left a comment

Choose a reason for hiding this comment

Uh oh!

vvoland Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

dmcgowan commented Aug 15, 2025 •

edited by vvoland

Loading

vvoland Dec 11, 2025 •

edited

Loading