implement module compatibility check by thaJeztah · Pull Request #46941 · moby/moby

thaJeztah · 2023-12-15T13:08:55Z

This package imports all "importable" packages, i.e., packages that:

are not applications ("main")
are not internal
and that have non-test go-files

We do this to verify that our code can be consumed as a dependency in "module mode". When using a dependency that does not have a go.mod (i.e.; is not a "module"), go implicitly generates a go.mod. Lacking information from the dependency itself, it assumes "go1.16" language (see DefaultGoModVersion). Starting with Go1.21, go downgrades the language version used for such dependencies, which means that any language feature used that is not supported by go1.16 results in a compile error;

# github.com/docker/cli/cli/context/store
/go/pkg/mod/github.com/docker/[email protected]+incompatible/cli/context/store/storeconfig.go:6:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
/go/pkg/mod/github.com/docker/[email protected]+incompatible/cli/context/store/store.go:74:12: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)

These errors do NOT occur when using GOPATH mode, nor do they occur when using "pseudo module mode" (the "-mod=mod -modfile=vendor.mod" approach used in this repository).

As a workaround for this situation, we must include "//go:build" comments in any file that uses newer go-language features (such as the "any" type or the "min()", "max()" builtins).

From the go toolchain docs (https://go.dev/doc/toolchain):

The go line for each module sets the language version the compiler enforces
when compiling packages in that module. The language version can be changed
on a per-file basis by using a build constraint.

For example, a module containing code that uses the Go 1.21 language version
should have a go.mod file with a go line such as go 1.21 or go 1.21.3.
If a specific source file should be compiled only when using a newer Go
toolchain, adding //go:build go1.22 to that source file both ensures that
only Go 1.22 and newer toolchains will compile the file and also changes
the language version in that file to Go 1.22.

This file is a generated module that imports all packages provided in the repository, which replicates an external consumer using our code as a dependency in go-module mode, and verifies all files in those packages have the correct "//go:build " set.

To test this package:

make -C ./internal/gocompat/
GO111MODULE=off go generate .
go mod tidy
go test -v
# github.com/docker/docker/libnetwork/options
../../libnetwork/options/options.go:45:25: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/libnetwork/internal/setmatrix
../../libnetwork/internal/setmatrix/setmatrix.go:13:16: type parameter requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/internal/setmatrix/setmatrix.go:13:18: predeclared comparable requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/internal/setmatrix/setmatrix.go:14:20: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/internal/setmatrix/setmatrix.go:20:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/internal/setmatrix/setmatrix.go:31:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/internal/setmatrix/setmatrix.go:43:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/internal/setmatrix/setmatrix.go:59:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/internal/setmatrix/setmatrix.go:80:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/internal/setmatrix/setmatrix.go:93:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/internal/setmatrix/setmatrix.go:104:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/internal/setmatrix/setmatrix.go:104:10: too many errors
# github.com/docker/docker/libnetwork/config
../../libnetwork/config/config.go:35:47: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/config/config.go:47:41: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/config/config.go:63:55: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../libnetwork/config/config.go:95:63: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/testutil
../../testutil/helpers.go:80:9: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
# github.com/docker/docker/builder/builder-next/adapters/containerimage
../../builder/builder-next/adapters/containerimage/pull.go:72:4: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
../../builder/builder-next/adapters/containerimage/pull.go:200:19: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
FAIL	gocompat [build failed]
make: *** [Makefile:5: verify] Error 1

temporarily skip some packages

one, or combinations of these still makes it fail, but unfortunately
it doesn't print what file it is:

go test -v
# github.com/docker/docker/daemon
embedding interface element ~[]string requires go1.18 or later (-lang was set to go1.16; check go.mod)
FAIL	gocompat [build failed]

no external consumer should be importing the daemon-code, so skipping
checks for these (for now).

- What I did

- How I did it

- How to verify it

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

thaJeztah · 2023-12-15T15:08:06Z

+// // promoteNil converts a nil slice to an empty slice of that type.
+// // A non-nil slice is returned as is.
+// func promoteNil[S ~[]E, E any](s S) S {
+// 	if s == nil {
+// 		return S{}
+// 	}
+// 	return s
+// }


I wonder if there's a bug in go; even with //go:build 1.19 this fails. I also tried //go:build go1.20 (and 1.21) but same;

make -C ./internal/gocompat/ make: Entering directory '/go/src/github.com/docker/docker/internal/gocompat' GO111MODULE=off go generate . GO111MODULE=on go mod tidy GO111MODULE=on go test -v # github.com/docker/docker/daemon embedding interface element ~[]string requires go1.18 or later (-lang was set to go1.16; check go.mod) FAIL gocompat [build failed] make: *** [Makefile:3: verify] Error 1 make: Leaving directory '/go/src/github.com/docker/docker/internal/gocompat'

Some discussions related to that error (one was a bug in go);

embedding interface element requires go1.18 or later (-lang was set to go1.16; check go.mod) golangci/golangci-lint#3177

cmd/go: install -mod=readonly pkg@version uses -lang=go1.16 golang/go#54908

The lack of a position in the error message is suspicious. As is the error message: seems like it should only apply to embedded types in interfaces, though I could be wrong. I wonder if the type-checker is synthesizing a virtual interface type to represent the function's type constraint and neglecting to annotate it with the source position of the function, causing the type-checker to fall back to the default language version. ~~Is gotip also affected?~~ Only Go 1.21 is affected, not gotip.

It's a compiler bug. I have a minimal reproducer; will report upstream.

Oh, thanks!! 🙏

Yes, after trying all angles, I started to suspect it could be a bug, but I failed to create a minimal reproducer so far, so wanted to give it more tries during the weekend.

cmd/compile: //go:build file version ignored when calling generic fn which has related type params (reproduces with 1.21 and 1.20, not at tip) golang/go#64759

corhere · 2024-06-21T19:12:56Z

Good news! go vet will be able to perform the module compatibility check itself starting in Go 1.23.

vvoland · 2025-03-24T13:23:58Z

bump, WDYT about getting this one in @thaJeztah?

thaJeztah · 2025-03-25T12:59:56Z

Rebased; this should currently fail because this PR wasn't merged yet: Add missing go1.22 build constraints #49700

thaJeztah · 2025-03-25T13:52:39Z

Nice! It's failing, so it works;

go: downloading github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572
GO111MODULE=on go test -v
# github.com/docker/docker/internal/platform
../platform/platform_linux.go:16:20: implicit function instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod)
FAIL	gocompat [build failed]
make: *** [Makefile:3: verify] Error 1
make: Leaving directory '/go/src/github.com/docker/docker/internal/gocompat'
make: *** [Makefile:254: validate-gocompat] Error 2
Error: Process completed with exit code 2.

This package imports all "importable" packages, i.e., packages that: - are not applications ("main") - are not internal - and that have non-test go-files We do this to verify that our code can be consumed as a dependency in "module mode". When using a dependency that does not have a go.mod (i.e.; is not a "module"), go implicitly generates a go.mod. Lacking information from the dependency itself, it assumes "go1.16" language (see [DefaultGoModVersion]). Starting with Go1.21, go downgrades the language version used for such dependencies, which means that any language feature used that is not supported by go1.16 results in a compile error; # github.com/docker/cli/cli/context/store /go/pkg/mod/github.com/docker/[email protected]+incompatible/cli/context/store/storeconfig.go:6:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) /go/pkg/mod/github.com/docker/[email protected]+incompatible/cli/context/store/store.go:74:12: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) These errors do NOT occur when using GOPATH mode, nor do they occur when using "pseudo module mode" (the "-mod=mod -modfile=vendor.mod" approach used in this repository). As a workaround for this situation, we must include "//go:build" comments in any file that uses newer go-language features (such as the "any" type or the "min()", "max()" builtins). From the go toolchain docs (https://go.dev/doc/toolchain): > The go line for each module sets the language version the compiler enforces > when compiling packages in that module. The language version can be changed > on a per-file basis by using a build constraint. > > For example, a module containing code that uses the Go 1.21 language version > should have a go.mod file with a go line such as go 1.21 or go 1.21.3. > If a specific source file should be compiled only when using a newer Go > toolchain, adding //go:build go1.22 to that source file both ensures that > only Go 1.22 and newer toolchains will compile the file and also changes > the language version in that file to Go 1.22. This file is a generated module that imports all packages provided in the repository, which replicates an external consumer using our code as a dependency in go-module mode, and verifies all files in those packages have the correct "//go:build <go language version>" set. To test this package: make -C ./internal/gocompat/ GO111MODULE=off go generate . go mod tidy go test -v # github.com/docker/docker/libnetwork/options ../../libnetwork/options/options.go:45:25: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) # github.com/docker/docker/libnetwork/internal/setmatrix ../../libnetwork/internal/setmatrix/setmatrix.go:13:16: type parameter requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../libnetwork/internal/setmatrix/setmatrix.go:13:18: predeclared comparable requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../libnetwork/internal/setmatrix/setmatrix.go:14:20: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../libnetwork/internal/setmatrix/setmatrix.go:20:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../libnetwork/internal/setmatrix/setmatrix.go:31:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../libnetwork/internal/setmatrix/setmatrix.go:43:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../libnetwork/internal/setmatrix/setmatrix.go:59:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../libnetwork/internal/setmatrix/setmatrix.go:80:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../libnetwork/internal/setmatrix/setmatrix.go:93:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../libnetwork/internal/setmatrix/setmatrix.go:104:10: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../libnetwork/internal/setmatrix/setmatrix.go:104:10: too many errors # github.com/docker/docker/libnetwork/config ../../libnetwork/config/config.go:35:47: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../libnetwork/config/config.go:47:41: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../libnetwork/config/config.go:63:55: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../libnetwork/config/config.go:95:63: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) # github.com/docker/docker/testutil ../../testutil/helpers.go:80:9: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod) # github.com/docker/docker/builder/builder-next/adapters/containerimage ../../builder/builder-next/adapters/containerimage/pull.go:72:4: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod) ../../builder/builder-next/adapters/containerimage/pull.go:200:19: type instantiation requires go1.18 or later (-lang was set to go1.16; check go.mod) FAIL gocompat [build failed] make: *** [Makefile:5: verify] Error 1 [DefaultGoModVersion]: https://github.com/golang/go/blob/58c28ba286dd0e98fe4cca80f5d64bbcb824a685/src/cmd/go/internal/gover/version.go#L15-L24 [2]: https://go.dev/doc/toolchain Signed-off-by: Sebastiaan van Stijn <[email protected]> Signed-off-by: Paweł Gronowski <[email protected]>

Signed-off-by: Paweł Gronowski <[email protected]>

thaJeztah · 2025-03-25T15:11:44Z

Rebased after that PR was merged; this should go green now 👍

thaJeztah · 2025-03-25T19:46:17Z

And it's green 🎉

@vvoland PTAL

vvoland

LGTM, thanks!

thaJeztah force-pushed the govalidator branch 3 times, most recently from bb0b6ab to 2bee51f Compare December 15, 2023 13:12

thaJeztah mentioned this pull request Dec 15, 2023

add //go:build directives to prevent downgrading to go1.16 language #46942

Merged

thaJeztah self-assigned this Dec 15, 2023

thaJeztah force-pushed the govalidator branch from 2bee51f to 2d885ac Compare December 15, 2023 14:39

thaJeztah added status/2-code-review area/testing labels Dec 15, 2023

thaJeztah added this to the 25.0.0 milestone Dec 15, 2023

thaJeztah force-pushed the govalidator branch from 2d885ac to 8d2472d Compare December 15, 2023 15:05

thaJeztah commented Dec 15, 2023

View reviewed changes

thaJeztah mentioned this pull request Dec 15, 2023

ci: add module compatibility check docker/cli#4723

Merged

thaJeztah force-pushed the govalidator branch from 8d2472d to 717307b Compare December 15, 2023 19:33

corhere force-pushed the govalidator branch from 9ea3a57 to 7e25e34 Compare December 21, 2023 21:19

thaJeztah mentioned this pull request Jan 3, 2024

daemon: work around go1.21 compiler bug #47020

Merged

thaJeztah modified the milestones: 25.0.0, 26.0.0 Jan 19, 2024

thaJeztah force-pushed the govalidator branch from 7e25e34 to 975a784 Compare January 25, 2024 10:19

thaJeztah mentioned this pull request Jan 25, 2024

add more //go:build directives to prevent downgrading to go1.16 language #47213

Merged

vvoland modified the milestones: 26.0.0, v-future Feb 21, 2024

thaJeztah force-pushed the govalidator branch 2 times, most recently from 43c5202 to cf99976 Compare June 13, 2024 12:44

thaJeztah mentioned this pull request Jun 13, 2024

Add more go:build statements to prevent downgrading Go language version, and update to go1.21 #47973

Merged

thaJeztah force-pushed the govalidator branch from cf99976 to cc34fdf Compare June 13, 2024 13:05

thaJeztah added the area/go-sdk label Jun 18, 2024

thaJeztah force-pushed the govalidator branch 2 times, most recently from f62b708 to b16f1d2 Compare June 20, 2024 11:56

thaJeztah mentioned this pull request Aug 5, 2024

c8d/image: Simplify presentImages and better "platform not found" error #48276

Merged

thaJeztah force-pushed the govalidator branch from 9509b47 to a417ab6 Compare January 1, 2025 12:33

This was referenced Jan 1, 2025

daemon: add missing "//go:build" directive #49186

Merged

daemon: remove workaround for go1.21 compiler bug #49187

Merged

thaJeztah force-pushed the govalidator branch from a417ab6 to 36f68aa Compare January 18, 2025 15:57

vvoland force-pushed the govalidator branch 4 times, most recently from d8e4ee4 to b18d5b7 Compare February 6, 2025 18:10

vvoland added this to 🔦 Maintainer spotlight Mar 24, 2025

github-project-automation Bot moved this to Up next in 🔦 Maintainer spotlight Mar 24, 2025

thaJeztah force-pushed the govalidator branch from b18d5b7 to 8534f5a Compare March 25, 2025 12:59

thaJeztah changed the title ~~[WIP] implement module compatibility check~~ implement module compatibility check Mar 25, 2025

thaJeztah and others added 2 commits March 25, 2025 16:10

hack/validate: Add gocompat

a60603b

Signed-off-by: Paweł Gronowski <[email protected]>

thaJeztah force-pushed the govalidator branch from 8534f5a to a60603b Compare March 25, 2025 15:10

thaJeztah modified the milestones: v-future, 28.0.5 Mar 25, 2025

thaJeztah marked this pull request as ready for review March 25, 2025 15:11

thaJeztah requested a review from tianon as a code owner March 25, 2025 15:11

vvoland approved these changes Mar 26, 2025

View reviewed changes

vvoland merged commit 849c723 into moby:master Mar 26, 2025
150 checks passed

thaJeztah deleted the govalidator branch March 26, 2025 13:35

thompson-shaun moved this from Up next to Complete in 🔦 Maintainer spotlight Mar 27, 2025

thompson-shaun modified the milestones: 28.0.5, 28.1.0 Apr 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

implement module compatibility check#46941

implement module compatibility check#46941
vvoland merged 2 commits intomoby:masterfrom
thaJeztah:govalidator

thaJeztah commented Dec 15, 2023 •

edited

Loading

Uh oh!

thaJeztah Dec 15, 2023 •

edited

Loading

Uh oh!

corhere Dec 15, 2023 •

edited

Loading

Uh oh!

corhere Dec 15, 2023

Uh oh!

thaJeztah Dec 15, 2023

Uh oh!

corhere Dec 15, 2023

Uh oh!

corhere commented Jun 21, 2024

Uh oh!

vvoland commented Mar 24, 2025

Uh oh!

thaJeztah commented Mar 25, 2025

Uh oh!

thaJeztah commented Mar 25, 2025

Uh oh!

thaJeztah commented Mar 25, 2025

Uh oh!

thaJeztah commented Mar 25, 2025

Uh oh!

vvoland left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

thaJeztah commented Dec 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

temporarily skip some packages

Uh oh!

thaJeztah Dec 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

corhere Dec 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

corhere Dec 15, 2023

Choose a reason for hiding this comment

Uh oh!

thaJeztah Dec 15, 2023

Choose a reason for hiding this comment

Uh oh!

corhere Dec 15, 2023

Choose a reason for hiding this comment

Uh oh!

corhere commented Jun 21, 2024

Uh oh!

vvoland commented Mar 24, 2025

Uh oh!

thaJeztah commented Mar 25, 2025

Uh oh!

thaJeztah commented Mar 25, 2025

Uh oh!

thaJeztah commented Mar 25, 2025

Uh oh!

thaJeztah commented Mar 25, 2025

Uh oh!

vvoland left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

thaJeztah commented Dec 15, 2023 •

edited

Loading

thaJeztah Dec 15, 2023 •

edited

Loading

corhere Dec 15, 2023 •

edited

Loading