daemon/c8d: Limit concurrent downloads/uploads #95
Conversation
| // TODO(vvoland): use the i.downloadLimiter directly | ||
| opts = append(opts, containerd.WithMaxConcurrentDownloads(i.limits.MaxConcurrentDownloads)) |
There was a problem hiding this comment.
For pulls, the maximum downloads is currently respected across one pull - not the whole engine. Pull creates its own semaphore.Weighted inside the private fetch function which makes it impossible to share one limiter for all pulls.
Possibly at some point we would need to have our own implementation of Pull anyway, so maybe it's fine for now? I also have a quick patch PoC for containerd that would allow to pass the semaphore.Weighted directly. So I could try to clean it up a bit and make a it PR for it upstream, WDYT?
There was a problem hiding this comment.
This is fine, from the help: Set the max concurrent downloads for each pull (default 3)
There was a problem hiding this comment.
The help text is misleading then - limit is per whole engine.
| opts = i.applySnapshotterOpts(opts, ref) | ||
|
|
||
| if i.limits.MaxConcurrentDownloads > 0 { | ||
| // TODO(vvoland): use the i.downloadLimiter directly |
There was a problem hiding this comment.
Intent here is to use the same semaphore.Weighted for all pulls - to limit the downloads across whole engine and not one pull.
vvoland
force-pushed
the
c8d
branch
2 times, most recently
from
5e88685 to
ac7a31e
Compare
vvoland
force-pushed
the
c8d
branch
2 times, most recently
from
13b2b08 to
ff5b724
Compare
vvoland
force-pushed
the
c8d-upload-limit
branch
from
c4eca7c to
085bd47
Compare
Signed-off-by: Paweł Gronowski <[email protected]>
vvoland
force-pushed
the
c8d-upload-limit
branch
from
085bd47 to
9ff087d
Compare
4 participants
Respect the maximum concurrent downloads/upload daemon configuration.
Signed-off-by: Paweł Gronowski [email protected]
- What I did
- How I did it
- How to verify it
- Description for the changelog
- A picture of a cute animal (not mandatory but encouraged)