qcow2 compression (zlib, zstd) #7462

ekorenevsky · 2025-11-06T23:48:42Z

Add support of reading and writing compressed clusters.
Support zlib and zstd compressions.

rbradford

Great PR - really nice structure.

block/Cargo.toml

block/src/qcow/decoder.rs

block/src/qcow/mod.rs

ekorenevsky · 2025-11-08T00:27:53Z

Had to squash all commits together due to impossibility to keep Cargo.lock, block/Cargo.toml, block/src/qcow/decoder.rs in sync if they were modified in different commits. The PR is not big, so I think this is not an issue.

rbradford

It would be great if we could unit test with a real compressed data - or alternatively do a boot test in the integration test suite with a compressed qcow2 file (we could just compress one of the existing files.)

block/src/qcow/decoder.rs

block/src/qcow/mod.rs

ekorenevsky · 2025-11-16T21:38:34Z

I succeeded to boot qcow2-compresed Ubuntu image. qcow2-backing also works well.

# create base snapshot, it will be immutable
qemu-img convert -c -O qcow2 ubuntu-raw.img ubuntu-compressed-frozen.qcow2
chmod -w ubuntu-compressed-frozen.qcow2

# create mutable layer, backed by immutable base snapshot
qemu-img create -b ubuntu-compressed-frozen.qcow2 -F qcow2 -f qcow2 ubuntu-compressed.qcow2


# boot
cloud-hypervisor/target/debug/cloud-hypervisor --kernel /path/to/linux/arch/x86/boot/bzImage --disk path=ubuntu-compressed.qcow2 --cmdline 'console=tty0 console=ttyS0 root=/dev/vda1 debug loglevel=8 rw net.ifnames=0' --memory size=2048M

Hope tests will work well when test qcow2 images are compressed. Will try a bit later...

phip1611

Generally LGTM, thanks for working on this! I left a few remarks

block/src/qcow/decoder.rs

block/src/qcow/mod.rs

phip1611

Please don't use anyhow here - we want to get rid of it in most places in Cloud Hypervisor, not strengthen the dependency to it :) Apart from that, changes are looking good.

block/src/qcow/decoder.rs

phip1611

Thanks for addressing my comments. LGTM now! Please fix CI

ekorenevsky · 2025-11-20T20:58:43Z

getting a weird error at CI

https://github.com/cloud-hypervisor/cloud-hypervisor/actions/runs/19551012517/job/55982246959?pr=7462

+ cargo test --lib --bins --target aarch64-unknown-linux-musl --release --workspace
    Updating crates.io index
error: failed to get `igvm` as a dependency of package `hypervisor v0.1.0 (/cloud-hypervisor/hypervisor)`

Caused by:
  failed to load source for dependency `igvm`

likebreath · 2025-11-20T21:12:00Z

getting a weird error at CI

https://github.com/cloud-hypervisor/cloud-hypervisor/actions/runs/19551012517/job/55982246959?pr=7462
+ cargo test --lib --bins --target aarch64-unknown-linux-musl --release --workspace
    Updating crates.io index
error: failed to get `igvm` as a dependency of package `hypervisor v0.1.0 (/cloud-hypervisor/hypervisor)`

Caused by:
  failed to load source for dependency `igvm`

Looks like the recurring GitHub network flake we've seen today. I just restarted the worker.

rbradford

Just those two changes and then we can 🚀 this.

block/src/qcow/decoder.rs

scripts/run_integration_tests_aarch64.sh

rbradford · 2025-11-23T09:41:50Z

Great ! lgtm- can you rebase this to drop the merge commit and then we can land it.

Add support of reading and writing compressed clusters. Support zlib and zstd compressions. L2 cache: store entire L2 entries, not only standard cluster addresses. Read path. Offsets of compressed clusters cannot be determined, therefore replace QcowFile.file_offset_read() with QcowFile.file_read(). This method reads the cluster, decompresses it if necessary and returns the data to the caller. Write path. QcowFile.file_offset_write(): since writing to compressed clusters is not generally possible, allocate a new standard (non-compressed) cluster if compressed L2 entry is encountered; then decompress compressed cluster into new cluster; then return offset inside new cluster to the caller. Processing of standard clusters is not changed. Signed-off-by: Eugene Korenevsky <[email protected]>

Add tests: - zlib: test_virtio_block_qcow2_zlib() - zstd: test_virtio_block_qcow2_zstd() Both these tests use zlib- and zstd-compressed images as OS image. Modify test_virtio_block_qcow2_backing_file() test: it is practical to test qcow2 file-backing with compression, so use zlib-compressed image as a backing file. Signed-off-by: Eugene Korenevsky <[email protected]>

Signed-off-by: Eugene Korenevsky <[email protected]>

ekorenevsky · 2025-11-23T15:22:53Z

Great ! lgtm- can you rebase this to drop the merge commit and then we can land it.

Rebased exactly these reviewed commits on top of main. It is safe to merge.
Some jobs at CI were canceled for unknown reason.

rbradford

🚀

ekorenevsky requested a review from a team as a code owner November 6, 2025 23:48

rbradford reviewed Nov 7, 2025

View reviewed changes

block/Cargo.toml Show resolved Hide resolved

block/src/qcow/decoder.rs Outdated Show resolved Hide resolved

block/src/qcow/decoder.rs Outdated Show resolved Hide resolved

block/src/qcow/mod.rs Show resolved Hide resolved

block/src/qcow/mod.rs Show resolved Hide resolved

ekorenevsky force-pushed the qcow2-compression branch 5 times, most recently from b2f39d7 to a151f8a Compare November 8, 2025 00:23

rbradford reviewed Nov 8, 2025

View reviewed changes

block/src/qcow/decoder.rs Outdated Show resolved Hide resolved

block/src/qcow/mod.rs Show resolved Hide resolved

ekorenevsky force-pushed the qcow2-compression branch from a151f8a to 37e2688 Compare November 16, 2025 21:06

ekorenevsky force-pushed the qcow2-compression branch from bac1c85 to d4a91a2 Compare November 17, 2025 23:17

phip1611 requested changes Nov 18, 2025

View reviewed changes

block/src/qcow/decoder.rs Outdated Show resolved Hide resolved

block/src/qcow/decoder.rs Outdated Show resolved Hide resolved

block/src/qcow/mod.rs Outdated Show resolved Hide resolved

block/src/qcow/mod.rs Outdated Show resolved Hide resolved

ekorenevsky force-pushed the qcow2-compression branch from d4a91a2 to 5c856b8 Compare November 18, 2025 22:26

ekorenevsky requested a review from phip1611 November 18, 2025 23:22

phip1611 requested changes Nov 20, 2025

View reviewed changes

block/src/qcow/decoder.rs Outdated Show resolved Hide resolved

ekorenevsky force-pushed the qcow2-compression branch from 5c856b8 to 2ded770 Compare November 20, 2025 18:19

ekorenevsky requested a review from phip1611 November 20, 2025 18:20

phip1611 approved these changes Nov 20, 2025

View reviewed changes

ekorenevsky force-pushed the qcow2-compression branch 2 times, most recently from 8e98d97 to bf3a856 Compare November 20, 2025 20:53

liuw approved these changes Nov 21, 2025

View reviewed changes

rbradford reviewed Nov 22, 2025

View reviewed changes

block/src/qcow/decoder.rs Outdated Show resolved Hide resolved

scripts/run_integration_tests_aarch64.sh Outdated Show resolved Hide resolved

ekorenevsky force-pushed the qcow2-compression branch from bf3a856 to 39c97ad Compare November 22, 2025 22:23

ekorenevsky force-pushed the qcow2-compression branch 3 times, most recently from 1601bb0 to 4787c0f Compare November 23, 2025 11:57

ekorenevsky added 3 commits November 23, 2025 17:12

block: qcow: switch qcow2 tests from focal to jammy qcow2 images

dc44ecd

Signed-off-by: Eugene Korenevsky <[email protected]>

ekorenevsky force-pushed the qcow2-compression branch from 4787c0f to dc44ecd Compare November 23, 2025 14:13

rbradford approved these changes Nov 24, 2025

View reviewed changes

rbradford added this pull request to the merge queue Nov 24, 2025

Merged via the queue into cloud-hypervisor:main with commit e6d31a3 Nov 24, 2025
42 of 45 checks passed

likebreath added this to Cloud Hypervisor Roadmap Dec 11, 2025

github-project-automation bot moved this to ✅ Done in Cloud Hypervisor Roadmap Dec 11, 2025

qcow2 compression (zlib, zstd) #7462

qcow2 compression (zlib, zstd) #7462

Uh oh!

Conversation

ekorenevsky commented Nov 6, 2025

Uh oh!

rbradford left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ekorenevsky commented Nov 8, 2025

Uh oh!

rbradford left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ekorenevsky commented Nov 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

phip1611 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

phip1611 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

phip1611 left a comment

Choose a reason for hiding this comment

Uh oh!

ekorenevsky commented Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

likebreath commented Nov 20, 2025

Uh oh!

rbradford left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rbradford commented Nov 23, 2025

Uh oh!

ekorenevsky commented Nov 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rbradford left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ekorenevsky commented Nov 16, 2025 •

edited

Loading

phip1611 left a comment •

edited

Loading

ekorenevsky commented Nov 20, 2025 •

edited

Loading

ekorenevsky commented Nov 23, 2025 •

edited

Loading