Skip to content

borg2: "borg transfer" cmd (and also getting rid of legacy)#6703

Merged
ThomasWaldmann merged 14 commits intoborgbackup:borg2from
ThomasWaldmann:r2r-transfer
Jun 9, 2022
Merged

borg2: "borg transfer" cmd (and also getting rid of legacy)#6703
ThomasWaldmann merged 14 commits intoborgbackup:borg2from
ThomasWaldmann:r2r-transfer

Conversation

@ThomasWaldmann
Copy link
Member

No description provided.

this is somehow similar to borg recreate,
but with different focus and way simpler:

not changing compression algo
not changing chunking
not excluding files inside an archive by path match
only dealing with complete archives

but:
different src and dst repo
only reading each chunk once
keeping the compressed payload (no decompression/recompression effort)
--dry-run can be used before and afterwards to check
also: remove attic bug support code from borg check.

borg transfer removes the acl key. we do not run borg check on old repos.
hack: see the docstring of ZLIB_legacy class.

New clean ZLIB class that works as every other compressor.

ZLIB ID 0x0500, ZLIB_legacy ID 0x.8..
Item.hlid: same id, same hardlink (xxh64 digest)
Item.hardlink_master: not used for new archives any more
Item.source: not used for hardlink slaves any more
…gbackup#2323

Timestamp scales to 64 or 96bit serialization formats, that should be enough for everybody.

We use this in archived items and also in the files cache.
borg now has the chunks list in every item with content.
due to the symmetric way how borg now deals with hardlinks using
item.hlid, processing gets much simpler.

but some places where borg deals with other "sources" of hardlinks
still need to do some hardlink management:
borg uses the HardLinkManager there now (which is not much more
than a dict, but keeps documentation at one place and avoids some
code duplication we had before).

item.hlid is computed via hardlink_id function.

support hardlinked symlinks, fixes borgbackup#2379
as we use item.hlid now to group hardlinks together,
there is no conflict with the item.source usage for
symlink targets any more.

2nd+ hardlinks now add to the files count as did the 1st one.
for borg, now all hardlinks are created equal.
so any hardlink item with chunks now adds to the "file" count.

ItemFormatter: support {hlid} instead of {source} for hardlinks
now all hardlinked regular file items have chunks.
@ThomasWaldmann ThomasWaldmann changed the title borg transfer (and also getting rid of legacy) borg2: "borg transfer" cmd (and also getting rid of legacy) May 18, 2022
but still be able to read v1 archives
for borg transfer.
adapt borg transfer, transferred chunks are set to compression level "unknown".
the inner payload of ObfuscateSize chunks are compressed chunks and need
the same zlib fix and level patching as non-obfuscated compressed chunks.
@ThomasWaldmann ThomasWaldmann merged commit 86fe8bd into borgbackup:borg2 Jun 9, 2022
@ThomasWaldmann ThomasWaldmann deleted the r2r-transfer branch June 9, 2022 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants