vendor: use go mod (rebase after removal of deprecated host-discovery) by thaJeztah · Pull Request #43101 · moby/moby

thaJeztah · 2021-12-22T17:07:06Z

this is a rebase of #43075 to see if it's easy to re-do the go mod changes if #42247 would be merged first

I made some changes in separate commits (could be squashed); using this draft PR to check CI.

thaJeztah · 2021-12-22T18:11:54Z

Hmm.. interesting; looks like vendor-check fails (diff in vendor.sum), but doesn't stop CI from running (so it keeps hanging on something);

[2021-12-22T17:40:44.561Z] patching file stat_unix.go
[2021-12-22T17:40:44.561Z] Hunk #1 succeeded at 9 (offset 1 line).
[2021-12-22T17:40:44.561Z] Hunk #2 succeeded at 17 (offset 1 line).
[2021-12-22T17:40:44.561Z] Hunk #3 succeeded at 25 (offset 1 line).
[2021-12-22T17:40:44.561Z] The result of go mod vendor differs
[2021-12-22T17:40:44.561Z] 
[2021-12-22T17:40:44.561Z]  M vendor.sum
[2021-12-22T17:40:44.561Z] 
[2021-12-22T17:40:44.561Z] Please vendor your package with hack/vendor.sh.
[2021-12-22T17:40:44.561Z] 
[2021-12-22T17:40:44.561Z] diff --git a/vendor.sum b/vendor.sum
[2021-12-22T17:40:44.561Z] index 2b8477107a..dc3747792d 100644
[2021-12-22T17:40:44.561Z] --- a/vendor.sum
[2021-12-22T17:40:44.561Z] +++ b/vendor.sum
[2021-12-22T17:40:44.561Z] @@ -123,6 +123,7 @@ github.com/coreos/etcd v3.3.25+incompatible/go.mod h
[2021-12-22T17:40:44.561Z] 1:uF7uidLiAD3TWHmW31ZFd/JWoc
[2021-12-22T17:40:44.561Z]  github.com/coreos/go-iptables v0.5.0/go.mod h1:/mVI274lEDI2ns62jHCDnCyBF9Iwsmek
[2021-12-22T17:40:44.561Z] av8Dbxlm1MU=
[2021-12-22T17:40:44.561Z]  github.com/coreos/go-oidc v2.1.0+incompatible/go.mod h1:CgnwVTmzoESiwO9qyAFEMiH
[2021-12-22T17:40:44.561Z] oZ1nMCKZlZ9V6mm3/LKc=
[2021-12-22T17:40:44.561Z]  github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3E
[2021-12-22T17:40:44.561Z] edlOD2RNk=
[2021-12-22T17:40:44.561Z] +github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+1
[2021-12-22T17:40:44.561Z] 53eP4D4r3EedlOD2RNk=
[2021-12-22T17:40:44.561Z]  github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7 h1:u9SHYsPQNyt5
[2021-12-22T17:40:44.561Z] tgDm3YN7+9dYrpK96E5wFilTFWIDZOM=
[2021-12-22T17:40:44.561Z]  github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7/go.mod h1:F5haX
[2021-12-22T17:40:44.561Z] 7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
[2021-12-22T17:40:44.561Z]  github.com/coreos/go-systemd/v22 v22.1.0/go.mod h1:xO0FLkIi5MaZafQlIrOotqXZ90ih
[2021-12-22T17:40:44.561Z] +1atmu1JpKERPPk=
[2021-12-22T17:40:44.561Z] @@ -205,7 +206,6 @@ github.com/go-openapi/spec v0.19.3/go.mod h1:FpwSN1k
[2021-12-22T17:40:44.561Z] sY1eteniUU7X0N/BgJ7a4WvBFVA8
[2021-12-22T17:40:44.561Z]  github.com/go-openapi/swag v0.19.2/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9g

thaJeztah · 2021-12-22T18:15:14Z

Probably because hack/vendor.sh doesn't run go mod tidy (and I may have forgotten to run it again 🤔)

thaJeztah · 2021-12-23T13:34:49Z

Hmm.. new failure?

=== RUN   TestFollowLogsHandleDecodeErr
time="2021-12-23T11:18:44Z" level=error msg="Log reader notified that it must re-open log file, some log data may not be streamed to the client." file=/tmp/TestFollowLogsHandleDecodeErr3941852975
    logfile_test.go:317: assertion failed: 5 (int) != 6 (dec.resetCount int)
--- FAIL: TestFollowLogsHandleDecodeErr (0.01s)

thaJeztah · 2022-01-06T21:30:46Z

+	github.com/containerd/console v1.0.2 // indirect
+	github.com/containerd/go-runc v1.0.0 // indirect
+	github.com/containerd/ttrpc v1.1.0 // indirect
+	github.com/coreos/etcd v3.3.25+incompatible // indirect


Note to self; I need to update this to the version that's used in vendor.conf (see #41791)

thaJeztah · 2022-01-06T21:32:21Z

+	github.com/docker/libkv v0.2.2-0.20211217103745-e480589147e3
+	github.com/docker/libtrust v0.0.0-20150526203908-9cbd2a1374f4
+	github.com/docker/swarmkit v1.12.1-0.20210726173615-3629f50980f6
+	github.com/fluent/fluent-logger-golang v1.8.0


Note to self: need to update this to v1.9.0 (see #43100)

errordeveloper · 2022-01-10T11:26:09Z

@thaJeztah thanks for rebasing and the additions!

thaJeztah · 2022-01-10T11:32:33Z

Weird failures on Windows 😞 (likely unrelated)

SUCCESS: Specified value was saved.
INFO: Downloading git...
INFO: Downloading go...
INFO: Downloading compiler 1 of 3...
INFO: Downloading compiler 2 of 3...
INFO: Downloading compiler 3 of 3...
INFO: Extracting git...
Move-Item : Access to the path 'C:\git-tmp\tools\usr' is denied.
At line:1 char:2427
+ ... tory C:\git | Out-Null; Move-Item C:\git-tmp\tools\* C:\git\.; Remove ...
+                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : WriteError: (C:\git-tmp\tools\usr:DirectoryInfo) 
    [Move-Item], IOException
    + FullyQualifiedErrorId : MoveDirectoryItemIOError,Microsoft.PowerShell.Commands.MoveItemCommand

errordeveloper · 2022-01-10T12:01:40Z

@thaJeztah I have seen similar Windows file access/modification races on my PR earlier... definitely unrelated.

thaJeztah · 2022-01-10T13:32:26Z

Looking mostly OK; Other failure on Windows 2022 is a known flaky; #43012

=== FAIL: github.com/docker/docker/integration-cli TestDockerSuite/TestSlowStdinClosing (1.08s)
    docker_cli_run_test.go:4177: assertion failed: error is not nil: exit status 127
    --- FAIL: TestDockerSuite/TestSlowStdinClosing (1.08s)

errordeveloper · 2022-01-12T11:08:02Z

@thaJeztah should this be marked "ready for review"?

- use `vendor.mod` instead of `go.mod` to avoid issues to do with use of CalVer, not SemVer - ensure most of the dependency versions do not change - only zookeeper client has to change (via docker/libkv#218) as previously used version is no longer maintained and has missing dependencies Signed-off-by: Sebastiaan van Stijn <[email protected]>

- all changes here are attributed to difference in behaviour between, namely: - resolution of secondary test dependencies - prunning of non-Go files Signed-off-by: Ilya Dmitrichenko <[email protected]> Signed-off-by: Sebastiaan van Stijn <[email protected]>

This allows the modules to be cached when vendoring modules from within the dev container. Signed-off-by: Sebastiaan van Stijn <[email protected]>

commit googleapis/google-cloud-go@ad4f932 removes some of the test-dependencies from cloud.google.com. only other relevant changes in vendored code are from this commit: googleapis/google-cloud-go@dccc6b4 Full diff: googleapis/google-cloud-go@v0.44.3...v0.59.0 Signed-off-by: Sebastiaan van Stijn <[email protected]>

thaJeztah · 2022-01-20T08:21:30Z

All green now! We rolled back the windows machines to the previous builds (gonna be using #43145 to track if we're able to find where the regression is in Windows; looks like there's issues with newer builds of Windows itself that cause CI to fail and to run severely slower, but we may need help from the Microsoft people on that, as it looks to be in the platform itself)

thaJeztah · 2022-01-20T08:22:38Z

Next steps will be to remove the replace rules, and to update dependencies (containerd, buildkit, swarmkit)

slonopotamus · 2022-01-24T08:09:22Z

So, is moby moving towards using Go modules in a proper way, with go.mod/go.sum?

thaJeztah · 2022-01-24T08:37:02Z

Ultimately, yes, but that requires quite some additional changes to make it work

thaJeztah mentioned this pull request Dec 22, 2021

vendor: use go mod #43075

Closed

thaJeztah force-pushed the go_mod_rebase branch 2 times, most recently from dc0d134 to 2e0b7ba Compare December 22, 2021 19:06

thaJeztah mentioned this pull request Jan 6, 2022

Convert "vendor.conf" to "go.mod" with minimal changes [DNM] #42489

Closed

thaJeztah force-pushed the go_mod_rebase branch from 3893251 to ec56ad8 Compare January 6, 2022 21:27

thaJeztah commented Jan 6, 2022

View reviewed changes

thaJeztah force-pushed the go_mod_rebase branch from ec56ad8 to c92a26f Compare January 10, 2022 11:13

errordeveloper approved these changes Jan 10, 2022

View reviewed changes

thaJeztah force-pushed the go_mod_rebase branch from c92a26f to 2c151c0 Compare January 12, 2022 17:10

thaJeztah marked this pull request as ready for review January 12, 2022 17:10

thaJeztah requested a review from tianon as a code owner January 12, 2022 17:10

AkihiroSuda approved these changes Jan 18, 2022

View reviewed changes

errordeveloper and others added 4 commits January 18, 2022 15:46

vendor: regenerate

e5d2811

- all changes here are attributed to difference in behaviour between, namely: - resolution of secondary test dependencies - prunning of non-Go files Signed-off-by: Ilya Dmitrichenko <[email protected]> Signed-off-by: Sebastiaan van Stijn <[email protected]>

Makefile: use volume for go mod cache when running dev-container

3bd7448

This allows the modules to be cached when vendoring modules from within the dev container. Signed-off-by: Sebastiaan van Stijn <[email protected]>

thaJeztah force-pushed the go_mod_rebase branch from 2c151c0 to ace8c78 Compare January 18, 2022 14:46

tonistiigi approved these changes Jan 18, 2022

View reviewed changes

thaJeztah merged commit 860db4f into moby:master Jan 20, 2022

thaJeztah deleted the go_mod_rebase branch January 20, 2022 08:21

thaJeztah added this to the 21.xx milestone Jan 20, 2022

thaJeztah added area/project kind/refactor PR's that refactor, or clean-up code labels Jan 20, 2022

crazy-max mentioned this pull request Jan 20, 2022

vendor: eliminate some of the replace directives moby/swarmkit#3048

Closed

thaJeztah mentioned this pull request Jan 20, 2022

update runc to v1.1.0 #43083

Merged

thaJeztah mentioned this pull request Feb 5, 2022

hack/vendor.sh: run "go mod tidy" before vendoring #43212

Merged

thaJeztah mentioned this pull request Feb 15, 2022

vendor.mod: reduce number of replace rules: update some dependencies #43236

Merged

Conversation

thaJeztah commented Dec 22, 2021

Uh oh!

thaJeztah commented Dec 22, 2021

Uh oh!

thaJeztah commented Dec 22, 2021

Uh oh!

thaJeztah commented Dec 23, 2021

Uh oh!

thaJeztah Jan 6, 2022

Choose a reason for hiding this comment

Uh oh!

thaJeztah Jan 6, 2022

Choose a reason for hiding this comment

Uh oh!

errordeveloper commented Jan 10, 2022

Uh oh!

thaJeztah commented Jan 10, 2022

Uh oh!

errordeveloper commented Jan 10, 2022

Uh oh!

thaJeztah commented Jan 10, 2022

Uh oh!

errordeveloper commented Jan 12, 2022

Uh oh!

thaJeztah commented Jan 20, 2022

Uh oh!

thaJeztah commented Jan 20, 2022

Uh oh!

slonopotamus commented Jan 24, 2022

Uh oh!

thaJeztah commented Jan 24, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants