Skip to content

Dovecot: compile fts_xapian from source to match Dovecot ABI#3373

Merged
georglauterbach merged 18 commits intodocker-mailserver:masterfrom
tbutter:fts_xapian
Jun 1, 2023
Merged

Dovecot: compile fts_xapian from source to match Dovecot ABI#3373
georglauterbach merged 18 commits intodocker-mailserver:masterfrom
tbutter:fts_xapian

Conversation

@tbutter
Copy link
Copy Markdown
Contributor

@tbutter tbutter commented May 28, 2023

Description

compile fts_xapian from src to match dovecot abi version

Fixes #3247

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (README.md or the documentation under docs/)
  • If necessary I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@georglauterbach
Copy link
Copy Markdown
Member

How much larger will the final image be with these changes?

@tbutter
Copy link
Copy Markdown
Contributor Author

tbutter commented May 28, 2023

How much larger will the final image be with these changes?

My local build is 870MB vs. latest release 616MB.
The issue with the community builds of dovecot is that any plugin needs to be built again with that version.

The only way I see to avoid image bloat is to split the docker build: First install dovecot from community, compile fts_xapian, build a .deb.

Then next step the full build copying only the dep from the first build.

But that would be a larger change.

Probably uninstalling the build dep packages will help a bit. I'll try that.

@georglauterbach
Copy link
Copy Markdown
Member

georglauterbach commented May 28, 2023

Good idea! Because 870MB is definitely too large. The multi-stage build may be the best option, but it'd be nice if you tinker a bit around 👍🏼

@tbutter
Copy link
Copy Markdown
Contributor Author

tbutter commented May 28, 2023

Good idea! Because 870MB is definitely too large. The multi-stage build may be the best option, but it'd be nice if you tinker a bit around 👍🏼

removing deps brings it down to 700MB

@polarathene
Copy link
Copy Markdown
Member

polarathene commented May 29, 2023

Perhaps put it behind a build flag? 80MB is a lot of extra weight for a feature not many are using AFAIK? (definitely better than 870MB at least) Not ideal since it requires users that do want the feature to build the image themselves though..

With Fedora and OpenSUSE this is easier at least, but I'm not that familiar with Debian. Would there be any way to reduce the image weight further?

  • The situation might change with the upcoming Debian Bookworm release next month? (Dovecot community build could be made opt-in if it's new enough, and thus the two may be compatible by default meeting most users needs?)
  • We could defer the build to a separate stage and build / tag releases with that (but I don't think this one feature warrants that, plus it'd affect our own limited CI cache storage)
  • Changing base image has been discussed in the past which may also resolve the issue (eg: Fedora releases more frequently and isn't holding back package updates like how Bookworm will still be stuck on PostSRSd <2), however most active maintainers are in favour of sticking with Debian.

One other concern with compiling is the CI ARM builds would probably suffer a fair bit from that until Github offers actual ARM runners instead of emulating ARM via QEMU. Hopefully with build cache that shouldn't be too frequent of a problem.

@casperklein
Copy link
Copy Markdown
Member

casperklein commented May 29, 2023

  • fts-xapian-1.5.5 (dir)
  • dovecot-fts-xapian.tar.gz (file)

Those can be deleted after compilation?

Are the *-dev packages needed after compilation?

@tbutter
Copy link
Copy Markdown
Contributor Author

tbutter commented May 29, 2023

the new patch brings it down to 619MB, but it changes the Dockerfile structure. Is this an acceptable solution?

Copy link
Copy Markdown
Member

@casperklein casperklein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some minor suggestions, otherwise looks good 👍

Comment thread target/scripts/build/packages.sh Outdated
Comment thread target/scripts/build/packages.sh Outdated
Comment thread target/scripts/build/compile.sh
Copy link
Copy Markdown
Member

@georglauterbach georglauterbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍🏼 Very nice!

@georglauterbach georglauterbach changed the title compile fts_xapian from src to match dovecot abi Dovecot: compile fts_xapian from source to match Dovecot ABI May 29, 2023
casperklein
casperklein previously approved these changes May 29, 2023
@georglauterbach
Copy link
Copy Markdown
Member

@casperklein you can go ahead and merge this when all is well and you approve

Comment thread Dockerfile Outdated
Comment thread target/scripts/build/packages.sh Outdated
Comment thread target/scripts/build/packages.sh
@casperklein casperklein dismissed stale reviews from georglauterbach and polarathene via c806edc May 30, 2023 20:16
casperklein
casperklein previously approved these changes May 30, 2023
@casperklein
Copy link
Copy Markdown
Member

Is the dovecot-dev package needed after compilation?

@tbutter
Copy link
Copy Markdown
Contributor Author

tbutter commented May 30, 2023

Is the dovecot-dev package needed after compilation?

It contains the headers and /usr/share/dovecot/dovecot-abi - i am not 100% sure if the latter is needed. But it is just about 300kb.

@casperklein
Copy link
Copy Markdown
Member

Just for the record, ~3MB, not 300kb 😉

root@mail:/# aptitude show dovecot-dev
Package: dovecot-dev
Version: 2:2.3.20-3+debian11
Architecture: amd64
Uncompressed Size: 3431 k

polarathene
polarathene previously approved these changes May 31, 2023
@tbutter
Copy link
Copy Markdown
Contributor Author

tbutter commented May 31, 2023

Just for the record, ~3MB, not 300kb 😉

oh, you are right. i looked up the wrong version. let me test if it works without the -dev.

@tbutter tbutter dismissed stale reviews from polarathene and casperklein via f1604f8 May 31, 2023 05:17
@tbutter
Copy link
Copy Markdown
Contributor Author

tbutter commented May 31, 2023

Is the dovecot-dev package needed after compilation?

works without the -dev package. i changed the build. image size 619MB->618MB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

other: fts-xapian-plugin does not work after upgrading dms to 12.0

4 participants