Feature: Allow stream custom maxsize per batch#2063
Merged
mangalaman93 merged 5 commits intodgraph-io:mainfrom Oct 25, 2024
Merged
Feature: Allow stream custom maxsize per batch#2063mangalaman93 merged 5 commits intodgraph-io:mainfrom
mangalaman93 merged 5 commits intodgraph-io:mainfrom
Conversation
|
|
✅ Deploy Preview for badger-docs canceled.
|
716d7b0 to
acba512
Compare
Contributor
|
@simon28082 the linter is not happy. Could you check that please? and also pull in latest changes from the |
Contributor
Author
|
@mangalaman93 will do it |
Change maxSize type
Contributor
Author
|
@mangalaman93 I done it, please review |
mangalaman93
requested changes
Oct 18, 2024
mangalaman93
previously approved these changes
Oct 21, 2024
| // Send the batch immediately if it already exceeds the maximum allowed size. | ||
| // If the size of the batch exceeds maxStreamSize, break from the loop to | ||
| // avoid creating a batch that is so big that certain limits are reached. | ||
| if batch.LenNoPadding() > int(maxStreamSize) { |
Contributor
There was a problem hiding this comment.
could you try upgrading the action version in .github/workflows/ci-golang-lint.yml:29 to
uses: golangci/[email protected]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
I'm trying to read the data using badger and use grpc for node sync, but I can't change the max cut size of stream because it's restricted hardcode
Solution
Add a property to Stream to allow custom slice size