Skip to content

bump containerd/cgroups 5fbad35c2a7e855762d3c60f2e474ffcad0d470a#40154

Merged
tiborvass merged 1 commit intomoby:masterfrom
thaJeztah:bump_cgroups
Oct 31, 2019
Merged

bump containerd/cgroups 5fbad35c2a7e855762d3c60f2e474ffcad0d470a#40154
tiborvass merged 1 commit intomoby:masterfrom
thaJeztah:bump_cgroups

Conversation

@thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Oct 30, 2019

@vikramhh vikramhh mentioned this pull request Oct 30, 2019
Closed
1 task
@thaJeztah
Copy link
Member Author

thaJeztah commented Oct 30, 2019 •
edited
Loading

Ah, fun; so this requires a more recent version of libcontainerd;

never mind; just local changes

libcontainerd/types/types_linux.go:16:11: undefined: cgroups.Metrics

@thaJeztah thaJeztah force-pushed the bump_cgroups branch 2 times, most recently from 8d1ace8 to 973afa2 Compare October 30, 2019 12:55
@thaJeztah
Copy link
Member Author

thaJeztah commented Oct 30, 2019

Failures on Windows RS1 is interesting; looks like we're trying to use a unix-socket on windows;
https://ci.docker.com/public/blue/rest/organizations/jenkins/pipelines/moby/branches/PR-40154/runs/5/nodes/52/log/?start=0

[2019-10-30T13:19:27.378Z] --- FAIL: TestInfoDebug (0.02s)
[2019-10-30T13:19:27.378Z]     info_test.go:75: [d757c903a4332] failed to start daemon with arguments [--containerd /var/run/docker/containerd/containerd.sock --data-root D:\gopath\src\github.com\docker\docker\bundles\tmp\TestInfoDebug\d757c903a4332\root --exec-root C:\windows\TEMP\dxr\d757c903a4332 --pidfile D:\gopath\src\github.com\docker\docker\bundles\tmp\TestInfoDebug\d757c903a4332\docker.pid --userland-proxy=true --containerd-namespace d757c903a4332 --containerd-plugins-namespace d757c903a4332p --host unix://C:\windows\TEMP\docker-integration\d757c903a4332.sock --storage-driver overlay2 --debug] : protocol not available
[2019-10-30T13:19:27.378Z] === RUN   TestInfoInsecureRegistries
[2019-10-30T13:19:27.378Z] --- FAIL: TestInfoInsecureRegistries (0.00s)
[2019-10-30T13:19:27.378Z]     info_test.go:100: [d3dab042d8737] failed to start daemon with arguments [--containerd /var/run/docker/containerd/containerd.sock --data-root D:\gopath\src\github.com\docker\docker\bundles\tmp\TestInfoInsecureRegistries\d3dab042d8737\root --exec-root C:\windows\TEMP\dxr\d3dab042d8737 --pidfile D:\gopath\src\github.com\docker\docker\bundles\tmp\TestInfoInsecureRegistries\d3dab042d8737\docker.pid --userland-proxy=true --containerd-namespace d3dab042d8737 --containerd-plugins-namespace d3dab042d8737p --host unix://C:\windows\TEMP\docker-integration\d3dab042d8737.sock --debug --storage-driver overlay2 --insecure-registry=192.168.1.0/24 --insecure-registry=insecurehost.com:5000] : protocol not available
[2019-10-30T13:19:27.378Z] === RUN   TestInfoRegistryMirrors
[2019-10-30T13:19:27.378Z] --- FAIL: TestInfoRegistryMirrors (0.01s)
[2019-10-30T13:19:27.378Z]     info_test.go:124: [de65b11b7ae5d] failed to start daemon with arguments [--containerd /var/run/docker/containerd/containerd.sock --data-root D:\gopath\src\github.com\docker\docker\bundles\tmp\TestInfoRegistryMirrors\de65b11b7ae5d\root --exec-root C:\windows\TEMP\dxr\de65b11b7ae5d --pidfile D:\gopath\src\github.com\docker\docker\bundles\tmp\TestInfoRegistryMirrors\de65b11b7ae5d\docker.pid --userland-proxy=true --containerd-namespace de65b11b7ae5d --containerd-plugins-namespace de65b11b7ae5dp --host unix://C:\windows\TEMP\docker-integration\de65b11b7ae5d.sock --debug --storage-driver overlay2 --registry-mirror=https://192.168.1.2 --registry-mirror=http://registry.mirror.com:5000] : protocol not available

--host unix://C:\windows\TEMP\docker-integration\d757c903a4332.sock
--storage-driver overlay2 --debug
protocol not available

Other failure is:

[2019-10-30T14:19:44.318Z]     --- FAIL: TestDockerSuite/TestBuildDockerignoreExceptions (59.01s)
[2019-10-30T14:19:44.318Z]         docker_cli_build_test.go:2351: assertion failed:
[2019-10-30T14:19:44.318Z]             Command:  d:\CI\PR-40154\5\binary\docker.exe build -t testbuilddockerignoreexceptions .
[2019-10-30T14:19:44.318Z]             ExitCode: 1
[2019-10-30T14:19:44.318Z]             Error:    exit status 1
[2019-10-30T14:19:44.318Z]             Stdout:   Sending build context to Docker daemon  8.704kB
[2019-10-30T14:19:44.318Z]
[2019-10-30T14:19:44.318Z]             Step 1/14 : FROM busybox
[2019-10-30T14:19:44.318Z]              ---> 0261edd4d45e
[2019-10-30T14:19:44.318Z]             Step 2/14 : ADD . /bla
[2019-10-30T14:19:44.318Z]              ---> e0f30eafb58e
[2019-10-30T14:19:44.318Z]             Step 3/14 : RUN sh -c "[[ -f /bla/src/x.go ]]"
[2019-10-30T14:19:44.318Z]              ---> Running in 3a6818356d2e
[2019-10-30T14:19:44.318Z]             Removing intermediate container 3a6818356d2e
[2019-10-30T14:19:44.318Z]              ---> 1f27a9e725ec
[2019-10-30T14:19:44.318Z]             Step 4/14 : RUN sh -c "[[ -f /bla/Makefile ]]"
[2019-10-30T14:19:44.318Z]              ---> Running in becac854582c
[2019-10-30T14:19:44.318Z]             Removing intermediate container becac854582c
[2019-10-30T14:19:44.318Z]              ---> 7de50716cf78
[2019-10-30T14:19:44.318Z]             Step 5/14 : RUN sh -c "[[ ! -e /bla/src/_vendor ]]"
[2019-10-30T14:19:44.318Z]              ---> Running in e093121ac05d
[2019-10-30T14:19:44.318Z]             Removing intermediate container e093121ac05d
[2019-10-30T14:19:44.318Z]              ---> 4c096849cc23
[2019-10-30T14:19:44.318Z]             Step 6/14 : RUN sh -c "[[ ! -e /bla/.gitignore ]]"
[2019-10-30T14:19:44.318Z]              ---> Running in 2b75521a3e8c
[2019-10-30T14:19:44.318Z]             Removing intermediate container 2b75521a3e8c
[2019-10-30T14:19:44.318Z]              ---> f3d5aaf6c5ff
[2019-10-30T14:19:44.318Z]             Step 7/14 : RUN sh -c "[[ ! -e /bla/README.md ]]"
[2019-10-30T14:19:44.318Z]              ---> Running in 69716f78fc7f
[2019-10-30T14:19:44.318Z]             Removing intermediate container 69716f78fc7f
[2019-10-30T14:19:44.318Z]              ---> 78ba8722b7d5
[2019-10-30T14:19:44.318Z]             Step 8/14 : RUN sh -c "[[  -e /bla/dir/dir/foo ]]"
[2019-10-30T14:19:44.318Z]              ---> Running in 2461d8c9be92
[2019-10-30T14:19:44.318Z]             Removing intermediate container 2461d8c9be92
[2019-10-30T14:19:44.318Z]              ---> 770346c0fc32
[2019-10-30T14:19:44.318Z]             Step 9/14 : RUN sh -c "[[ ! -e /bla/dir/foo1 ]]"
[2019-10-30T14:19:44.318Z]              ---> Running in 223a625bd942
[2019-10-30T14:19:44.318Z]             Removing intermediate container 223a625bd942
[2019-10-30T14:19:44.318Z]              ---> 5b73811b072a
[2019-10-30T14:19:44.318Z]             Step 10/14 : RUN sh -c "[[ -f /bla/dir/e ]]"
[2019-10-30T14:19:44.318Z]              ---> Running in 6d26bd279f1a
[2019-10-30T14:19:44.318Z]             Removing intermediate container 6d26bd279f1a
[2019-10-30T14:19:44.318Z]              ---> 3d4e6b8b197e
[2019-10-30T14:19:44.318Z]             Step 11/14 : RUN sh -c "[[ -f /bla/dir/e-dir/foo ]]"
[2019-10-30T14:19:44.318Z]              ---> Running in c6ca7aa4cbc4
[2019-10-30T14:19:44.318Z]
[2019-10-30T14:19:44.319Z]             Stderr:   re-exec error: exit status 1: output: time="2019-10-30T13:34:53Z" level=error msg="hcsshim::ImportLayer - failed failed in Win32: There is not enough space on the disk. (0x70)" error="hcsshim::ImportLayer - failed failed in Win32: There is not enough space on the disk. (0x70)" importFolderPath="D:\\CI\\PR-40154\\5\\daemon\\tmp\\hcs262570275" path="\\\\?\\D:\\CI\\PR-40154\\5\\daemon\\windowsfilter\\552078b33187a91163be0570d75c4780a6c0ebb46442fc4118a86b1d9fca90cd"
[2019-10-30T14:19:44.319Z]             hcsshim::ImportLayer - failed failed in Win32: There is not enough space on the disk. (0x70)
[2019-10-30T14:19:44.319Z]
[2019-10-30T14:19:44.319Z]
[2019-10-30T14:19:44.319Z]             Failures:
[2019-10-30T14:19:44.319Z]             ExitCode was 1 expected 0
[2019-10-30T14:19:44.319Z]             Expected no error

Could be the host having issues;

hcsshim::ImportLayer - failed failed in Win32: There is not enough space on the disk. (0x70)

@thaJeztah
Copy link
Member Author

thaJeztah commented Oct 30, 2019

Looks like we have an issue; it's failing on Windows RS5 as well, but Jenkins doesn't report the failure;
https://ci.docker.com/public/job/moby/job/PR-40154/5/execution/node/176/log/?consoleFull

00:14:14.623  --- FAIL: TestInfoDebug (0.01s)
00:14:14.623      info_test.go:75: [dbe75bf7729f3] failed to start daemon with arguments [--containerd /var/run/docker/containerd/containerd.sock --data-root D:\gopath\src\github.com\docker\docker\bundles\tmp\TestInfoDebug\dbe75bf7729f3\root --exec-root C:\windows\TEMP\dxr\dbe75bf7729f3 --pidfile D:\gopath\src\github.com\docker\docker\bundles\tmp\TestInfoDebug\dbe75bf7729f3\docker.pid --userland-proxy=true --containerd-namespace dbe75bf7729f3 --containerd-plugins-namespace dbe75bf7729f3p --host unix://C:\windows\TEMP\docker-integration\dbe75bf7729f3.sock --storage-driver overlay2 --debug] : protocol not available
00:14:14.623  === RUN   TestInfoInsecureRegistries
00:14:14.623  --- FAIL: TestInfoInsecureRegistries (0.00s)
00:14:14.623      info_test.go:100: [d3c745c16a39c] failed to start daemon with arguments [--containerd /var/run/docker/containerd/containerd.sock --data-root D:\gopath\src\github.com\docker\docker\bundles\tmp\TestInfoInsecureRegistries\d3c745c16a39c\root --exec-root C:\windows\TEMP\dxr\d3c745c16a39c --pidfile D:\gopath\src\github.com\docker\docker\bundles\tmp\TestInfoInsecureRegistries\d3c745c16a39c\docker.pid --userland-proxy=true --containerd-namespace d3c745c16a39c --containerd-plugins-namespace d3c745c16a39cp --host unix://C:\windows\TEMP\docker-integration\d3c745c16a39c.sock --debug --storage-driver overlay2 --insecure-registry=192.168.1.0/24 --insecure-registry=insecurehost.com:5000] : protocol not available
00:14:14.623  === RUN   TestInfoRegistryMirrors
00:14:14.623  --- FAIL: TestInfoRegistryMirrors (0.01s)
00:14:14.623      info_test.go:124: [d277126ad0515] failed to start daemon with arguments [--containerd /var/run/docker/containerd/containerd.sock --data-root D:\gopath\src\github.com\docker\docker\bundles\tmp\TestInfoRegistryMirrors\d277126ad0515\root --exec-root C:\windows\TEMP\dxr\d277126ad0515 --pidfile D:\gopath\src\github.com\docker\docker\bundles\tmp\TestInfoRegistryMirrors\d277126ad0515\docker.pid --userland-proxy=true --containerd-namespace d277126ad0515 --containerd-plugins-namespace d277126ad0515p --host unix://C:\windows\TEMP\docker-integration\d277126ad0515.sock --debug --storage-driver overlay2 --registry-mirror=https://192.168.1.2 --registry-mirror=http://registry.mirror.com:5000] : protocol not available

@thaJeztah
Copy link
Member Author

thaJeztah commented Oct 30, 2019 •
edited
Loading

Opened #40155 to skip for now, but we need to look into this, and especially Windows RS5 not reporting failures

related: #40069 and #39576

@ddebroy @vikramhh @andrewhsu

@andrewhsu andrewhsu mentioned this pull request Oct 30, 2019
Merged
@thaJeztah
bump containerd/cgroups 5fbad35c2a7e855762d3c60f2e474ffcad0d470a
27552ce 
full diff: containerd/cgroups@c4b9ac5...5fbad35

- containerd/cgroups#82 Add go module support
- containerd/cgroups#96 Move metrics proto package to stats/v1
- containerd/cgroups#97 Allow overriding the default /proc folder in blkioController
- containerd/cgroups#98 Allows ignoring memory modules
- containerd/cgroups#99 Add Go 1.13 to Travis
- containerd/cgroups#100 stats/v1: export per-cgroup stats

Signed-off-by: Sebastiaan van Stijn <[email protected]>
@thaJeztah
Copy link
Member Author

thaJeztah commented Oct 31, 2019

ping @dmcgowan @jterry75 @kolyshkin PTAL

@vikramhh
Copy link

vikramhh commented Oct 31, 2019

@thaJeztah - Windows RS5 not reporting failures has been root caused and I have a fix in mind. Can we take in #39998 before that though - will avoid some duplication of work as the fix will need to be reworked again for #39998.

@vikramhh
Copy link

vikramhh commented Oct 31, 2019

LGTM

@tiborvass tiborvass merged commit 9232e10 into moby:master Oct 31, 2019
kolyshkin
kolyshkin reviewed Oct 31, 2019
View reviewed changes
daemon/daemon_unix.go
"time"

containerd_cgroups "github.com/containerd/cgroups"
v1 "github.com/containerd/cgroups/stats/v1"
Copy link
Contributor

@kolyshkin kolyshkin Oct 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is kinda weird to call this v1. maybe just cgroups?

Copy link
Member

@AkihiroSuda AkihiroSuda Nov 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"statsv1" (v2 will be added soon)

Copy link
Member Author

@thaJeztah thaJeztah Nov 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened #40167 to rename to statsV1

@thaJeztah thaJeztah deleted the bump_cgroups branch October 31, 2019 17:46
kolyshkin
kolyshkin reviewed Oct 31, 2019
View reviewed changes
libcontainerd/types/types_linux.go
"time"

"github.com/containerd/cgroups"
v1 "github.com/containerd/cgroups/stats/v1"
Copy link
Contributor

@kolyshkin kolyshkin Oct 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@thaJeztah thaJeztah mentioned this pull request Nov 1, 2019
Merged
@vikramhh vikramhh mentioned this pull request Nov 26, 2019
Merged
1 task
@thaJeztah thaJeztah mentioned this pull request Dec 3, 2019
Merged
@thaJeztah thaJeztah added this to the 20.03.0 milestone Apr 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kolyshkin kolyshkin kolyshkin left review comments

@AkihiroSuda AkihiroSuda AkihiroSuda approved these changes

@tiborvass tiborvass tiborvass approved these changes

Assignees

No one assigned

Labels

area/runtime Runtime process/cherry-picked status/2-code-review

Projects

None yet

Milestone

20.10.0

Development

Successfully merging this pull request may close these issues.

5 participants

@thaJeztah @vikramhh @tiborvass @kolyshkin @AkihiroSuda