Skip to content

misc: remaining v13 todos#3370

Merged
georglauterbach merged 11 commits intomasterfrom
remaining-v13-todos/1
May 29, 2023
Merged

misc: remaining v13 todos#3370
georglauterbach merged 11 commits intomasterfrom
remaining-v13-todos/1

Conversation

@georglauterbach
Copy link
Copy Markdown
Member

@georglauterbach georglauterbach commented May 26, 2023

Description

Review commit by commit.

PS: No, I don't have the time to do this in separate PRs

Fixes #3355
Fixes #2545
Partly closes #3289

Type of change

  • Improvement (non-breaking change that does improve existing functionality)
  • This change partly is a documentation update

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (README.md or the documentation under docs/)
  • If necessary I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

See #3289

These have little effect at the moment, and will only be useful in
conjunction with `set -eEu`, but I want to have them in v13.0.0 as this
was part of the initial todo issue.
@georglauterbach georglauterbach added area/scripts kind/improvement Improve an existing feature, configuration file or the documentation area/documentation area/configuration (file) kind/update Update an existing feature, configuration file or the documentation labels May 26, 2023
@georglauterbach georglauterbach added this to the v13.0.0 milestone May 26, 2023
@georglauterbach georglauterbach self-assigned this May 26, 2023
Copy link
Copy Markdown
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tackling these 😀

Comment thread docs/content/faq.md Outdated
Comment thread docs/content/faq.md Outdated
Comment thread docs/content/faq.md Outdated
Comment thread Dockerfile
COPY --link --chown=200 --from=docker.io/clamav/clamav:latest /var/lib/clamav /var/lib/clamav

RUN <<EOF
chown root:root /var /var/lib
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the related commit message lacks context, this review comment can provide some context.

From TODO item in this comment:

/var/lib/clamav has an ownership all the way back to /var with clamav.

  • Either from the Dockerfile chown, or potentially the mail-state startup script.
  • I've not investigated, just noticed it.

We could add some context inline too:

Suggested change
chown root:root /var /var/lib
# COPY --chown=200 should not affect parents, restore:
chown root:root /var /var/lib

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the additional context! I already had a quick look yesterday, but couldn't figure out, what problem is going to be solved here or what/when this broke.

@georglauterbach Did you track down the cause?

11.3.1 seems to be the last release with correct ownership:

docker run --rm -it mailserver/docker-mailserver:11.3.1 bash -c 'ls -ld /var /var/lib'
drwxr-xr-x 1 root root 60 Dec 18 19:55 /var
drwxr-xr-x 1 root root 60 Dec 18 19:55 /var/lib

Current :edge:

docker run --rm -it mailserver/docker-mailserver:edge bash -c 'ls -ld /var /var/lib'
drwxr-xr-x 1 clamav clamav 60 May  2 22:14 /var
drwxr-xr-x 1 clamav clamav 60 May  2 22:14 /var/lib

But, when I locally build :edge myself I get this:

git clone --recurse-submodules https://github.com/docker-mailserver/docker-mailserver
cd docker-mailserver
docker build -t test .
docker run --rm -it test bash -c 'ls -ld /var /var/lib'
drwxr-xr-x 1 root root 60 May 22 00:00 /var
drwxr-xr-x 1 root root 60 May 27 11:10 /var/lib

That's pretty weird 🙄

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought this was due to the COPY --link, and maybe there is a bug in the version that's currently on the GitHub runner's Ubuntu 22.04?

Copy link
Copy Markdown
Member

@casperklein casperklein May 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO we shouldn't use buggy features when not absolutely necessary, if that means workarounds are a requirement..

@polarathene

# COPY --chown=200 should not affect parents, restore:

Can you confirm, that the problem is related to --link? Then we should extend the comment? Otherwise it should be fine.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You misunderstood - I am not certain, and --link is required AFAIK.

I was just guessing about why you don't observe the issue while the GitHub runner does.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and I think it is a bug only applicable to getting the image via pull

I am not sure if I understand correctly. You mean, that dockerhub messes with the image content after upload? I do upload my local build, to be pulled from my server. So I think it's definitely something happening during the build.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do upload my local build, to be pulled from my server.

You've pushed an image you built with buildx container driver to DockerHub, and then pulled it without any cache? And the issue is not present?

If you cannot reproduce what our CI build is pushing to DockerHub, and get that same pull behaviour I may be mistaken and I'll update the bug report. Fairly certain that my assumption is correct, but I haven't tried verifying.

You mean, that dockerhub messes with the image content after upload?

I am suspecting this yes. When we push to DockerHub we include additional metadata via a manifest that supports the cross-platform tagging.

If you build the image locally, it's added into the image store already built with the ClamAV linked layer IIRC. But when you push this to the registry, that linked layer is excluded from our DockerHub image, it's only metadata to pull in the layer from the clamav image digest at build time, and from the issues I referenced in the bug report, this is apparently optimized as treating that content layer as FROM scratch, where /var/lib does not exist.

Since we combined --link with --chown, the behaviour of --chownwhen parent dirs do not exist is to create them and use that chown ownership, whereas link behaved a bit differently I think (or always root?). This causes the parent dirs to be owned byclamav` during this image pull from the registry, which you would not experience with local image build.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After some more testing, I can reproduce it locally - no dockerhub involved.

you built with buildx container driver

After verifying, I can say I did not 🤷 I used the buildx default docker endpoint, the permission were good. If I repeat the build with a new builder instance (test) using docker-container as endpoint, /var and /var/lib are owned by clamav.


# docker buildx ls
NAME/NODE      DRIVER/ENDPOINT             STATUS   BUILDKIT                              PLATFORMS
test *         docker-container
  test0        unix:///var/run/docker.sock running  v0.11.6                               linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386
default        docker
  default      default                     running  v0.11.7-0.20230525183624-798ad6b0ce9f linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, I forgot to build with buildx via --driver docker-container too 😅

Thanks for confirming that! I'll go update the bug report.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll in the meantime go ahead and merge this PR :) When the linked issue is resolved upstream, we can remove the chmod again.

Copy link
Copy Markdown
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PS: No, I don't have the time to do this in separate PRs

Fair enough! 😛

Comment thread docs/content/faq.md
Comment thread Dockerfile Outdated
Copy link
Copy Markdown
Member

@casperklein casperklein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should ditch the buggy --link feature from the COPY statement or adjust the comment. Otherwise LGTM.

Copy link
Copy Markdown
Member

@polarathene polarathene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarified inline comment about workaround with link to upstream bug report 👍

Comment thread Dockerfile Outdated
@github-actions
Copy link
Copy Markdown
Contributor

Documentation preview for this PR is ready! 🎉

Built with commit: 286bfcb

@georglauterbach georglauterbach merged commit 6a4fac6 into master May 29, 2023
@georglauterbach georglauterbach deleted the remaining-v13-todos/1 branch May 29, 2023 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/configuration (file) area/documentation area/scripts kind/improvement Improve an existing feature, configuration file or the documentation kind/update Update an existing feature, configuration file or the documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: v13 sieve instructions are no longer valid other: open TODOs [POSTFIX] User unknown in local recipient table

3 participants