Skip to content

[release/1.7] Add blockfile snapshotter#8986

Merged
dmcgowan merged 13 commits intocontainerd:release/1.7from
dcantah:1.7-blockfile
Aug 21, 2023
Merged

[release/1.7] Add blockfile snapshotter#8986
dmcgowan merged 13 commits intocontainerd:release/1.7from
dcantah:1.7-blockfile

Conversation

@dcantah
Copy link
Copy Markdown
Member

@dcantah dcantah commented Aug 18, 2023

Unclear if we want this in 1.7, but given it's mostly entirely new code, regression chances seem low. This brings in every commit that touched the snapshots/blockfile directory since it was added as a few patches fixed CI flakiness and general usability.

  1. 07cc24b
  2. 6740d77
  3. 8472a40
  4. 8d732c6
  5. fd39769
  6. 99b61ae
  7. 6daeceb
  8. b823b4f
  9. cf9c78d
  10. 1c2977d
  11. 5437ef7
  12. 30a1630
  13. c9eebe3

Original Description:

A simple generic snapshotter using block files. It is designed to be cross-platform similar to the generic native snapshotter.

It can be paired with a differ to unpack in environments that do not support mounting.

  • Loopback mounts can be used for local mounts on Linux
  • VM runtimes can utilize virtio-blk rather than virtio-fs
  • Base scratch image can be provided with any configuration rather than relying on snapshotter to generate. Snapshot just disabled if no scratch image provided.
  • Copying blockfiles is very efficient on filesystems which support reflink/CoW/clone

Similar to #3130

dmcgowan and others added 10 commits August 18, 2023 03:10
Signed-off-by: Derek McGowan <[email protected]>
(cherry picked from commit 0c386e2)
Signed-off-by: Danny Canter <[email protected]>
Signed-off-by: Derek McGowan <[email protected]>
(cherry picked from commit 798dba9)
Signed-off-by: Danny Canter <[email protected]>
Signed-off-by: Derek McGowan <[email protected]>
(cherry picked from commit b729962)
Signed-off-by: Danny Canter <[email protected]>
Signed-off-by: Derek McGowan <[email protected]>
(cherry picked from commit 1be571b)
Signed-off-by: Danny Canter <[email protected]>
Signed-off-by: Derek McGowan <[email protected]>
(cherry picked from commit 7beaa5e)
Signed-off-by: Danny Canter <[email protected]>
Signed-off-by: Wei Fu <[email protected]>
(cherry picked from commit bb82f36)
Signed-off-by: Danny Canter <[email protected]>
Signed-off-by: Cardy.Tang <[email protected]>
(cherry picked from commit 9702d4a)
Signed-off-by: Danny Canter <[email protected]>
* Use direct-io mode to reduce IO.

* Add testViewHook helper to recovery the backing file since the ext4
  might need writable permission to handle recovery. If the backing file
  needs recovery and it's for View snapshot, the readonly mount will
  cause error.

* Use 8 MiB as capacity to reduce the IO.

Signed-off-by: Wei Fu <[email protected]>
(cherry picked from commit 7de95cb)
Signed-off-by: Danny Canter <[email protected]>
Signed-off-by: Wei Fu <[email protected]>
(cherry picked from commit a207b43)
Signed-off-by: Danny Canter <[email protected]>
Modify the loopback size in the blockfile snapshotter test setup.
Set the loopback size to 16MB when the page size is greater than 4096.

Signed-off-by: James Jenkins <[email protected]>
(cherry picked from commit 5c480d9)
Signed-off-by: Danny Canter <[email protected]>
fuweid added 3 commits August 18, 2023 05:23
Signed-off-by: Wei Fu <[email protected]>
(cherry picked from commit 72b7d16)
Signed-off-by: Danny Canter <[email protected]>
Signed-off-by: Wei Fu <[email protected]>
(cherry picked from commit 6dfb16f)
Signed-off-by: Danny Canter <[email protected]>
Pin it with 1e0d26eb2381594984ee80989c9c229dbd930d9f

Signed-off-by: Wei Fu <[email protected]>
(cherry picked from commit 59b0b39)
Signed-off-by: Danny Canter <[email protected]>
@dcantah
Copy link
Copy Markdown
Member Author

dcantah commented Aug 18, 2023

/retest

@dmcgowan
Copy link
Copy Markdown
Member

We don't normally add new features, I'm OK getting this in since it is mostly experimental with no impact on existing users. I can see the value in testing alongside the sandbox work in 1.7 though. So +1 from me.

Copy link
Copy Markdown
Member

@fuweid fuweid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dmcgowan dmcgowan merged commit bb2b8bc into containerd:release/1.7 Aug 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants