snapshots/overlay: add overlay cleanup#1833
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1833 +/- ##
==========================================
- Coverage 45.37% 45.25% -0.13%
==========================================
Files 96 96
Lines 9447 9511 +64
==========================================
+ Hits 4287 4304 +17
- Misses 4449 4491 +42
- Partials 711 716 +5
Continue to review full report at Codecov.
|
Is extending the API to include |
|
Moving this off of 1.0.1 as this seems like a larger change. |
dmcgowan
force-pushed
the
snapshot-gc-3rd-phase
branch
from
c36d3fd to
28de574
Compare
Updates overlay remove to simply remove the reference, adds a cleanup method for discarding the directory. Updates snapshot create to setup the directory structure while in the transaction, to prevent cleanup from removing directories which are part of a create. Signed-off-by: Derek McGowan <[email protected]>
dmcgowan
force-pushed
the
snapshot-gc-3rd-phase
branch
from
28de574 to
de54478
Compare
Allow configuring the overlay snapshotter to synchronously or asynchronously do cleanup. When the driver is integrated into a garbage collection system, the asynchronous cleanup can reduce the time of removal and allow the longer disk cleanup to be handled without locking the snapshotter. Signed-off-by: Derek McGowan <[email protected]>
dmcgowan
force-pushed
the
snapshot-gc-3rd-phase
branch
from
de54478 to
1fd2b57
Compare
dmcgowan
changed the title
|
Remove WIP tag, added options to for configuring the snapshotter, but the registered plugin will use the asynchronous mode |
| return nil, err | ||
| } | ||
|
|
||
| defer t.Rollback() |
There was a problem hiding this comment.
is this correct to always rollback?
There was a problem hiding this comment.
Yes, readonly transactions must always be rolled back.
In this case, even though the function is cleaning up, it is not making any alterations to the database, only to the disk. While the readonly transaction is open it figured out which directories must be removed, then after the transaction is done (rolled back in this case), it will perform the more expensive directory removals.
|
LGTM |
|
Discussed the possibility with @tonistiigi of also running the cleanup in |
|
LGTM |
5 participants
Updates overlay remove to simply remove the reference and adds a cleanup method for discarding the directory. Updates snapshot create to setup the directory structure while in the transaction to prevent cleanup from removing directories which are part of a create.
This effectively adds a 3rd phase to the garbage collection which completes all the on disk snapshot cleanup. During this disk cleanup the snapshotter will be unlocked to allow for the management of snapshots. The garbage collection scheduler will still wait for the completion of this 3rd phase, however it will not consider it when calculating pause time.
Added options to overlay snapshotter to allow configuring the asynchronous removal of keeping remove synchronous