Use zlib-ng instead of zlib#8500
Conversation
|
zlib-ng v2.2.2 x86_64-compat was not released because of CI test error. It'll be released at next version again. Anyway, building with CMake is also a fine solution. I'll follow Pillow devs' decision and close my PR #8495 when it's decided. |
|
Here's a chart of the write times for different compression levels and images:
Summary: zlib-ng is much faster. And file sizes:
Summary: zlib-ng has same file sizes for compression level 0 and bigger files for compression level 1, but if you care about size, any higher compression level gives more or less the same file size (and zlib-ng is faster). |
|
I've now checked that the Windows arm64 wheels pass the test suite on an MacBook Pro M2 Max (tested Python versions: 3.9.10, 3.10.11, 3.11.6, 3.12.0, 3.13.0). Benchmark results comparing the Python 3.13 wheels of the 11.0.0 release and the pull request build (click to expand) |
| "license": "LICENSE.md", | ||
| "patch": { | ||
| r"CMakeLists.txt": { | ||
| "set_target_properties(zlib PROPERTIES OUTPUT_NAME zlibstatic${{SUFFIX}})": "set_target_properties(zlib PROPERTIES OUTPUT_NAME zlib)", # noqa: E501 |
There was a problem hiding this comment.
If zlib-ng creates zlibstatic.lib by default, then wouldn't it seem helpful to detect that in setup.py, in case a user had built it themselves? nulano#44
There was a problem hiding this comment.
Oh, is the concern that zlib-ng might have been built without --zlib-compat?
| ], | ||
| "headers": [r"z*.h"], | ||
| "libs": [r"*.lib"], | ||
| "libs": [r"zlib.lib"], |
There was a problem hiding this comment.
| "libs": [r"zlib.lib"], | |
| "libs": ["zlib.lib"], |
|
Here's the results of running the benchmark on macOS 15.1 M2 with Python 3.13. Baseline: recent wheel from Test: recent wheel from this PR: https://github.com/python-pillow/Pillow/actions/runs/12328198837?pr=8500 Results# Image.__version__ = '11.1.0.dev0'
# Image.core.zlib_version = '1.3.1'
# Testing PATH = 'Tests/images/hopper.png' with 1000 repetitions
# read PNG: time = 0.121070 (sec)
# write PNG: time = 0.277860 (sec); size = 49353 bytes; compress_level = 0
# write PNG: time = 1.378973 (sec); size = 32403 bytes; compress_level = 1
# write PNG: time = 1.463843 (sec); size = 32156 bytes; compress_level = 2
# write PNG: time = 1.946944 (sec); size = 31792 bytes; compress_level = 3
# write PNG: time = 2.628619 (sec); size = 30715 bytes; compress_level = 4
# write PNG: time = 2.989835 (sec); size = 30529 bytes; compress_level = 5
# write PNG: time = 3.582502 (sec); size = 30343 bytes; compress_level = 6
# write PNG: time = 4.346017 (sec); size = 30277 bytes; compress_level = 7
# write PNG: time = 6.963706 (sec); size = 30183 bytes; compress_level = 8
# write PNG: time = 11.201260 (sec); size = 30166 bytes; compress_level = 9
# Image.__version__ = '11.1.0.dev0'
# Image.core.zlib_version = '1.3.1.zlib-ng'
# Testing PATH = 'Tests/images/hopper.png' with 1000 repetitions
# read PNG: time = 0.121355 (sec)
# write PNG: time = 0.243460 (sec); size = 49353 bytes; compress_level = 0
# write PNG: time = 0.484412 (sec); size = 40354 bytes; compress_level = 1
# write PNG: time = 0.582722 (sec); size = 31458 bytes; compress_level = 2
# write PNG: time = 0.647406 (sec); size = 31096 bytes; compress_level = 3
# write PNG: time = 0.849081 (sec); size = 30836 bytes; compress_level = 4
# write PNG: time = 0.699703 (sec); size = 30771 bytes; compress_level = 5
# write PNG: time = 0.916293 (sec); size = 30649 bytes; compress_level = 6
# write PNG: time = 1.426837 (sec); size = 30217 bytes; compress_level = 7
# write PNG: time = 1.855467 (sec); size = 30175 bytes; compress_level = 8
# write PNG: time = 2.772019 (sec); size = 30166 bytes; compress_level = 9
# Image.__version__ = '11.1.0.dev0'
# Image.core.zlib_version = '1.3.1'
# Testing PATH = 'Tests/images/effect_spread.png' with 1000 repetitions
# read PNG: time = 0.064432 (sec)
# write PNG: time = 0.271332 (sec); size = 49353 bytes; compress_level = 0
# write PNG: time = 1.590392 (sec); size = 42199 bytes; compress_level = 1
# write PNG: time = 1.749554 (sec); size = 42096 bytes; compress_level = 2
# write PNG: time = 2.188817 (sec); size = 41977 bytes; compress_level = 3
# write PNG: time = 2.496155 (sec); size = 41011 bytes; compress_level = 4
# write PNG: time = 2.278722 (sec); size = 40969 bytes; compress_level = 5
# write PNG: time = 2.514241 (sec); size = 40903 bytes; compress_level = 6
# write PNG: time = 2.503093 (sec); size = 40896 bytes; compress_level = 7
# write PNG: time = 2.616062 (sec); size = 40889 bytes; compress_level = 8
# write PNG: time = 2.614784 (sec); size = 40889 bytes; compress_level = 9
# Image.__version__ = '11.1.0.dev0'
# Image.core.zlib_version = '1.3.1.zlib-ng'
# Testing PATH = 'Tests/images/effect_spread.png' with 1000 repetitions
# read PNG: time = 0.062720 (sec)
# write PNG: time = 0.180120 (sec); size = 49353 bytes; compress_level = 0
# write PNG: time = 0.400339 (sec); size = 48241 bytes; compress_level = 1
# write PNG: time = 0.682549 (sec); size = 41287 bytes; compress_level = 2
# write PNG: time = 0.724561 (sec); size = 41168 bytes; compress_level = 3
# write PNG: time = 0.743060 (sec); size = 41134 bytes; compress_level = 4
# write PNG: time = 0.756581 (sec); size = 41120 bytes; compress_level = 5
# write PNG: time = 0.818945 (sec); size = 41118 bytes; compress_level = 6
# write PNG: time = 0.872731 (sec); size = 40895 bytes; compress_level = 7
# write PNG: time = 0.824981 (sec); size = 40889 bytes; compress_level = 8
# write PNG: time = 1.366639 (sec); size = 40889 bytes; compress_level = 9Again, a similar trend for time and identical numbers for size:
|
hugovk
left a comment
There was a problem hiding this comment.
Thank you!
Let's also include this in the release notes.
|
I've created #8599 for the release notes. |
| "license": "README", | ||
| "license_pattern": "Copyright notice:\n\n(.+)$", | ||
| "url": f"https://github.com/zlib-ng/zlib-ng/archive/refs/tags/{V['ZLIBNG']}.zip", | ||
| "filename": f"zlib-ng-{V['ZLIBNG']}.zip", |
There was a problem hiding this comment.
I've created #8601 to change this to .tar.gz, so that it matches wheels-dependencies.sh






Alternative to #8495 .
Changes proposed in this pull request:
Replace zlib with zlib-ng built with CMake in Windows builds. Building our own zlib-ng allows us to use the latest release since the zlib-ng 2.2.2 release is missing the x86_64 Windows build.
Add a flag to
PIL.featuresfor zlib-ng.python3 -m PIL.reportnow looks like this:I took inspiration from #8495 (comment) to make a benchmark with multiple compression values. It seems that zlib-ng usually produces a slightly larger file but in much less time during compression.
Script and results (click to expand)
Looking at the build logs, zlib-ng seems to have been properly detected by webp, libtiff, libpng, and freetype.
TODO: