Skip to content

Comments

fix: throttle concurrent loading#6441

Closed
satlank wants to merge 1 commit intoapache:mainfrom
satlank:fixNoBackpressureOnConcurrent
Closed

fix: throttle concurrent loading#6441
satlank wants to merge 1 commit intoapache:mainfrom
satlank:fixNoBackpressureOnConcurrent

Conversation

@satlank
Copy link

@satlank satlank commented Jul 23, 2025

Which issue does this PR close?

Closes #6440.

Rationale for this change

Applying back-pressure on concurrent reads to not create tasks faster than they are processed. The example in the documentation of the completed_but_unretrieved fields reads to be that the intention was that if concurrency is 10 and completed_but_unretrieved is 3, then 7 tasks can be going. I.e. the Addition should have been a Substraction.

What changes are included in this PR?

Don't run out of system resources when doing concurrent reads by applying back-pressure.

Are there any user-facing changes?

@satlank satlank requested a review from Xuanwo as a code owner July 23, 2025 15:05
@dosubot dosubot bot added size:XS This PR changes 0-9 lines, ignoring generated files. releases-note/fix The PR fixes a bug or has a title that begins with "fix" labels Jul 23, 2025
@Xuanwo Xuanwo closed this in #6449 Jul 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases-note/fix The PR fixes a bug or has a title that begins with "fix" size:XS This PR changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: no back-pressure on concurrent reads with chunks

1 participant