Skip to content

Measure how much the chunk cache changes in big real life repositories. #4827

@textshell

Description

@textshell

While discussing ways to avoid cache sync on irc @ThomasWaldmann came up with a simple idea how to use current borg to measure how much transfers a simple chunk based "store cache into the repository" approach would take.

The basic idea is to see if using mostly existing code to store the chunk cache into the repository can produce good enough results to further investigate.

The basic idea is to take a repository with a fairly large chunk cache that is only accessed from a single location and to use borg to backup just the chunk cache of that repository to a local dummy repository. Using --stats gives an indication of the amount of traffic that saving the cache would add. And also how much data would be needed to transfer for a different location to catch up to the current cache. (catching up with multiple backups in one go might or might not need less data transfer)

It would be interesting to see this with some different chunker parameters. I would expect a fixed size chunking setting to be the prime target, but of course all chunker setting can be tested.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions