Skip to content

Dockerfile: install criu from binary repo #41739

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kolyshkin wants to merge 1 commit into from
Closed

Dockerfile: install criu from binary repo #41739

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 5 additions & 18 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,23 +23,10 @@ ARG DEBIAN_FRONTEND
# Install dependency packages specific to criu
RUN --mount=type=cache,sharing=locked,id=moby-criu-aptlib,target=/var/lib/apt \
--mount=type=cache,sharing=locked,id=moby-criu-aptcache,target=/var/cache/apt \
apt-get update && apt-get install -y --no-install-recommends \
libcap-dev \
libnet-dev \
libnl-3-dev \
libprotobuf-c-dev \
libprotobuf-dev \
protobuf-c-compiler \
protobuf-compiler \
python-protobuf

# Install CRIU for checkpoint/restore support
ARG CRIU_VERSION=3.14
RUN mkdir -p /usr/src/criu \
&& curl -sSL https://github.com/checkpoint-restore/criu/archive/v${CRIU_VERSION}.tar.gz | tar -C /usr/src/criu/ -xz --strip-components=1 \
&& cd /usr/src/criu \
&& make \
&& make PREFIX=/build/ install-criu
echo 'deb https://download.opensuse.org/repositories/devel:/tools:/criu/Debian_10/ /' > /etc/apt/sources.list.d/criu.list \
&& curl -fsSL https://download.opensuse.org/repositories/devel:/tools:/criu/Debian_10/Release.key | apt-key add - \
Comment on lines +26 to +27
Copy link
Copy Markdown
Member

@thaJeztah thaJeztah Feb 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somewhat confusing the the "debian" packages are on a "opensuse" repository 😅

Do we need to replace the use of apt-key add here @tianon ? (recalling I was in the middle of reviewing docker/docs#11990)

Copy link
Copy Markdown
Member

@tianon tianon Feb 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't immediately "need" to, but we will soon and definitely should.

If we don't actually care about the provenance, we can just download the file straight to /etc/apt/trusted.gpg.d/something.gpg.asc (probably criu.gpg.asc) and it will work in this instance (and be at least as secure as piping to apt-key add -...)

Copy link
Copy Markdown
Member

@thaJeztah thaJeztah May 18, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I gave it a quick try, and that works; tried to use ADD instead of curl (so that cache would be invalidated would the key change), but ran into a bug, LOL (opened moby/buildkit#2114)

ADD --chmod=0644 https://download.opensuse.org/repositories/devel:/tools:/criu/Debian_10/Release.key /etc/apt/trusted.gpg.d/criu.gpg.asc
# FIXME: workaround for https://github.com/moby/buildkit/issues/2114
RUN chmod 0644 /etc/apt/trusted.gpg.d/criu.gpg.asc
RUN --mount=type=cache,sharing=locked,id=moby-criu-aptlib,target=/var/lib/apt \
    --mount=type=cache,sharing=locked,id=moby-criu-aptcache,target=/var/cache/apt \
        echo 'deb https://download.opensuse.org/repositories/devel:/tools:/criu/Debian_10/ /' > /etc/apt/sources.list.d/criu.list \
        && apt-get update \
        && apt-get install -y --no-install-recommends criu \
        && install -D /usr/sbin/criu /build/

Let me know what you think @tianon @kolyshkin

Copy link
Copy Markdown
Member

@thaJeztah thaJeztah May 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, let me add install -D /usr/sbin/criu /build/ so that we can revert the change to the COPY --from as well

&& apt-get update \
&& apt-get install -y --no-install-recommends criu

FROM base AS registry
WORKDIR /go/src/github.com/docker/distribution
Expand Down Expand Up @@ -300,7 +287,7 @@ COPY --from=swagger /build/ /usr/local/bin/
COPY --from=tomlv /build/ /usr/local/bin/
COPY --from=tini /build/ /usr/local/bin/
COPY --from=registry /build/ /usr/local/bin/
COPY --from=criu /build/ /usr/local/
COPY --from=criu /usr/sbin/criu /usr/local/bin
Copy link
Copy Markdown
Member

@thaJeztah thaJeztah Feb 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess apt-get install does not have a --prefix option (unless compiling form source) 😓 (would've been nice if we could keep the same paths)

Copy link
Copy Markdown
Member

@tianon tianon May 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could do something cute like criu="$(which criu)"; ln "$criu" /build/ after the apt-get install :trollface:

Copy link
Copy Markdown
Member

@thaJeztah thaJeztah May 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that we don't run the intermediate criu stage, and only use it to download the package, I took a different approach in https://github.com/moby/moby/pull/41739/files#r634478731

Copy link
Copy Markdown
Member

@cpuguy83 cpuguy83 May 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can literally have images in ghcr or hub for all of these things we currently build in the Dockerfile..
Speed things up a lot.

Copy link
Copy Markdown
Member

@thaJeztah thaJeztah May 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But then we'd have to maintain (multi-arch) images for each of those stages?

Copy link
Copy Markdown
Member

@cpuguy83 cpuguy83 May 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't be terribly difficult.
We only need to add the versions we pull in here.

COPY --from=vndr /build/ /usr/local/bin/
COPY --from=gotestsum /build/ /usr/local/bin/
COPY --from=golangci_lint /build/ /usr/local/bin/
Expand Down