Build-deb hack for systemd tasksmax #21628

clnperez · 2016-03-29T20:21:40Z

This uncomments the TasksMax line in the docker.service file if it's a known release that uses a systemd that supports it.

Verify by:

docker run --rm -it --privileged -v pwd:/go/src/github.com/docker/docker docker ./hack/make.sh dynbinary build-deb
cd bundles/latest/build-deb/
cd into one of the distros that built
untar the docker bundle
check that the TasksMax value is either commented or uncommented in the docker.service file under bundles/latest/build-deb//docker/contrib/init/systemd/

Verify more quickly by deleting some of the releases under contrib/builder/deb/amd64/ (e.g. all but ubuntu-wily and ubuntu-xenial, so you can check that wily is still commented out but xenial isn't).

Since we can't just add the TasksMax value into the docker.service
file, we can add it in at buildtime.

See /pull/21491 for some background.

Signed-off-by: Christy Perez [email protected]

thaJeztah · 2016-03-30T00:47:28Z

hack/make/build-deb

will this lead to -unsupported being added to the version string? https://github.com/docker/docker/blob/cf91a1be45fc4f1d4e3338610f921d665d406b04/hack/make.sh#L84-L94

It isn't going to change anything about the version string. It just conditionally adds a line to Dockerfile.build

But this changes of a Dockerfie will result of a dirty working directory as there will be a changes not commited. I think @thaJeztah is right on this, this probably result of a -unsupported build.

So, this whole script is actually creating the Dockerfile.build. I just re-ran to make sure that that unsupported message doesn't show up (just as a sanity check) and it didn't. Are you guys seeing it?

This is all good folks.

Thanks @tiborvass 😻

Thanks for checking! Sorry, I wasn't sure, and thought I'd leave a comment before we overlook it 👍

clnperez · 2016-03-30T16:30:56Z

I just rebased since PR #21491 (which added #TasksMax=infinity) got merged.

tiborvass · 2016-03-30T17:03:24Z

LGTM

tianon · 2016-03-30T17:06:20Z

hack/make/build-deb

Since these and everything after them will support it, would it be better to swap this list and list the ones that don't have the newer systemd? (ie, only update this list when we remove unsupported distros, not for every new one we add?)

Good point 👍

Yes, that is a good idea. That way, we've got a list, and we don't have to change it to pick up the value for any new release. And if an old release is removed (edit: from contrib/builder/deb/), we don't have to change the list then either. Testing this now. Thanks @tianon!

clnperez · 2016-03-30T22:08:00Z

All tested, and only the two not in the list get the sed line in their Dockerfile.build. PTAL

thaJeztah · 2016-03-30T22:20:01Z

hack/make/build-deb

typo: s/releses/releases/

thaJeztah · 2016-03-30T22:46:47Z

ping @tianon PTAL ❤️

tianon · 2016-03-30T22:52:04Z

LGTM

clnperez · 2016-04-04T15:08:47Z

@thaJeztah, does this one need a re-LGTM since it's been through a couple of edits?

thaJeztah · 2016-04-04T18:40:43Z

@clnperez let me check with @tiborvass

tiborvass · 2016-04-04T18:45:14Z

hack/make/build-deb

why escape the / ?

A remnant of a previous mistake in my sed command. Fixed, thanks!

Since we can't use the TasksMax value in the docker.service file by default, we can uncomment it at buildtime. See moby/pull/21491 for some background. Signed-off-by: Christy Perez <[email protected]>

tianon · 2016-04-07T16:18:09Z

LGTM

tiborvass · 2016-04-07T16:30:41Z

LGTM

StefanScherer · 2016-06-15T20:40:25Z

hack/make/build-deb

+		debian-jessie|debian-wheezy|ubuntu-precise|ubuntu-trusty|ubuntu-wily)
+			;;
+		*)
+			echo "RUN sed -i -- 's/#TasksMax=infinity/TasksMax=infinity/' contrib/init/systemd/docker.service" >> "$DEST/$version/Dockerfile.build"


Just found out that this leads to an armhf deb package for raspbian-jessie with binaries marked as "unsupported". The armhf debian-jessie is fine without that "unsupported" marker.
I'll make a PR for line 92 to make #21895 work again.

i wonder if you couldnt just do the sed in the raspbian-jessie dockerfile instead.

I also thought this should be "deep" enough. But the git commit check seems to run afterwards.

This is the relevant output of a make deb building current rc1 on armhf:

Step 14 : RUN sed -i -- 's/#TasksMax=infinity/TasksMax=infinity/' contrib/init/systemd/docker.service ---> Running in 708e5b0729ec ---> fd894af223c8 Removing intermediate container 708e5b0729ec Step 15 : RUN dpkg-buildpackage -uc -us -I.git ---> Running in eb84d00712e7 dpkg-buildpackage: source package docker-engine dpkg-buildpackage: source version 1.12.0~rc1-0~jessie dpkg-buildpackage: source distribution jessie dpkg-buildpackage: source changed by Docker <[email protected]> dpkg-source -I.git --before-build docker dpkg-buildpackage: host architecture armhf debian/rules clean dh clean --with=bash-completion --with=systemd dh_testdir dh_auto_clean dh_clean dpkg-source -I.git -b docker dpkg-source: warning: no source format specified in debian/source/format, see dpkg-source(1) dpkg-source: warning: source directory 'docker' is not <sourcepackage>-<upstreamversion> 'docker-engine-1.12.0~rc1' dpkg-source: info: using source format `1.0' dpkg-source: info: building docker-engine in docker-engine_1.12.0~rc1-0~jessie.tar.gz dpkg-source: info: building docker-engine in docker-engine_1.12.0~rc1-0~jessie.dsc debian/rules build dh build --with=bash-completion --with=systemd dh_testdir dh_auto_configure debian/rules override_dh_auto_build make[1]: Entering directory '/usr/src/docker' ./hack/make.sh dynbinary # WARNING! I don't seem to be running in a Docker container. # The result of this command might be an incorrect build, and will not be # officially supported. # # Try this instead: make all # #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ # GITCOMMIT = 1f136c1-unsupported # The version you are building is listed as unsupported because # there are some files in the git repository that are in an uncommited state. # Commit these changes, or add to .gitignore to remove the -unsupported from the version. # Here is the current list: M contrib/init/systemd/docker.service #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ bundles/1.12.0-rc1 already exists. Removing.

If we can come up with a simple check for whether our current-environment systemd is new enough, we could just add the check and the sed directly to "override_dh_installinit" inside "debian/rules", and perform it directly on the installed files in "debian/docker-engine/usr/.../docker.service" instead of on the file within the repository (or perform the substitution on the file directly before the current "dh_installinit" line on the file in the repository, which occurs after the binary is built).

But this sed is wrong for any platform, right?
Is it possible to revert this PR then?

....and also this isn't in 1.11.2

Sorry for one more message, but I finally got the build-deb script to finish without crashing b/c of some flaky repo, and it did build docker-engine_1.12.0~~rc1-0~~stretch_amd64.deb.

That said, do what you think makes the most sense. But for some reason it's only affecting the raspbian builds.

If we can use @tianon's approach #21628 (comment), perhaps it's better (but I don't know how that should be implemented 😊)

Sorry to disturb once more. I've done a make deb with the 1.12.0-rc2 tag and checked both debian-jessie and debian-stretch for amd64.

The docker binary for debian-stretch shows the "unsupported" string and the debian-jessie binary is fine:

root@00b88863593e:/tmp/jessie/usr/bin# ./docker --version Docker version 1.12.0-rc2, build 906eacd root@00b88863593e:/tmp/stretch/usr/bin# ./docker --version Docker version 1.12.0-rc2, build 906eacd-unsupported

So how should we proceed with fix for raspbian? I would like to send a PR to revert this one as it does not seem to work as intended as the sed changes the source tree and the binaries in the deb package are marked as unsupported.

I also installed the test package 1.12.0-rc2 in an ubuntu/xenial64 Vagrant box with curl -fsSL https://test.docker.com/ | sh and it also shows the unsupported tag:

+ sh -c docker version Client: Version: 1.12.0-rc2 API version: 1.24 Go version: go1.6.2 Git commit: 906eacd-unsupported Built: Fri Jun 17 21:21:56 2016 OS/Arch: linux/amd64 Server: Version: 1.12.0-rc2 API version: 1.24 Go version: go1.6.2 Git commit: 906eacd-unsupported Built: Fri Jun 17 21:21:56 2016 OS/Arch: linux/amd64 If you would like to use Docker as a non-root user, you should now consider adding your user to the "docker" group with something like: sudo usermod -aG docker your-user Remember that you will have to log out and back in for this to take effect! root@ubuntu-xenial:/home/ubuntu#

The same steps with a debian/jessie64 Vagrant box shows clean binaries.
So it seems that this is not only a raspbian problem.

GordonTheTurtle added the status/0-triage label Mar 29, 2016

clnperez mentioned this pull request Mar 29, 2016

docker.service: don't limit tasks #21491

Merged

calavera added status/1-design-review status/2-code-review and removed status/0-triage status/1-design-review labels Mar 30, 2016

thaJeztah reviewed Mar 30, 2016
View reviewed changes

clnperez force-pushed the systemd-tasksmax-workaround branch from 0b7a541 to 95a8d75 Compare March 30, 2016 16:27

tianon reviewed Mar 30, 2016
View reviewed changes

clnperez force-pushed the systemd-tasksmax-workaround branch from 95a8d75 to 15e1486 Compare March 30, 2016 21:33

thaJeztah reviewed Mar 30, 2016
View reviewed changes

clnperez force-pushed the systemd-tasksmax-workaround branch from 15e1486 to 7ad362d Compare March 30, 2016 22:40

tiborvass reviewed Apr 4, 2016
View reviewed changes

Build-deb hack for systemd tasksmax

2b849e0

Since we can't use the TasksMax value in the docker.service file by default, we can uncomment it at buildtime. See moby/pull/21491 for some background. Signed-off-by: Christy Perez <[email protected]>

clnperez force-pushed the systemd-tasksmax-workaround branch from 7ad362d to 2b849e0 Compare April 7, 2016 16:16

tiborvass merged commit e82830e into moby:master Apr 7, 2016

thaJeztah mentioned this pull request May 13, 2016

docker 1.12 daemon crashed after launched 40+ busybox container #22690

Closed

StefanScherer reviewed Jun 15, 2016
View reviewed changes

StefanScherer mentioned this pull request Jun 29, 2016

Revert tasksmax workaround to avoid unsupported bins #24100

Merged

thaJeztah mentioned this pull request Jun 30, 2016

Make "TasksMax" setting dependent on systemd version #24197

Closed

thaJeztah mentioned this pull request Jul 13, 2016

Unable to run 50 containers - rpc error: code = 2 desc = "oci runtime error..." #24536

Closed

clnperez deleted the systemd-tasksmax-workaround branch March 29, 2017 20:30

thaJeztah mentioned this pull request Aug 31, 2018

Update new systemd unit file with changes from upstream docker/docker-ce-packaging#179

Merged

seemethere mentioned this pull request Sep 4, 2018

[18.09] Update new systemd unit file with changes from upstream docker/docker-ce-packaging#182

Merged

Build-deb hack for systemd tasksmax #21628

Build-deb hack for systemd tasksmax #21628

Uh oh!

Conversation

clnperez commented Mar 29, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

clnperez commented Mar 30, 2016

Uh oh!

tiborvass commented Mar 30, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

clnperez commented Mar 30, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented Mar 30, 2016

Uh oh!

tianon commented Mar 30, 2016

Uh oh!

clnperez commented Apr 4, 2016

Uh oh!

thaJeztah commented Apr 4, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tianon commented Apr 7, 2016

Uh oh!

tiborvass commented Apr 7, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tianon Jun 15, 2016 via email

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!