This repository was archived by the owner on Mar 5, 2026. It is now read-only.
feat: enable write retry and nack pending writes on reconnect#443
Merged
alvarowolfx merged 26 commits intogoogleapis:mainfrom May 3, 2024
Merged
feat: enable write retry and nack pending writes on reconnect#443alvarowolfx merged 26 commits intogoogleapis:mainfrom
alvarowolfx merged 26 commits intogoogleapis:mainfrom
Conversation
product-auto-label
Bot
added
size: m
alvarowolfx
added
the
owlbot:run
gcf-owl-bot
Bot
removed
the
owlbot:run
yirutang
reviewed
Apr 22, 2024
yirutang
reviewed
Apr 23, 2024
product-auto-label
Bot
added
size: l
shollyman
reviewed
Apr 23, 2024
yirutang
reviewed
Apr 23, 2024
yirutang
reviewed
Apr 23, 2024
| it('should manage to send data in scenario where every 10 request drops the connection', async () => { | ||
| bqWriteClient.initialize(); | ||
| const client = new WriterClient(); | ||
| client.enableWriteRetries(true); |
There was a problem hiding this comment.
You should also be able to test the nonretried behavior.
yirutang
reviewed
Apr 24, 2024
yirutang
reviewed
Apr 24, 2024
yirutang
reviewed
Apr 24, 2024
alvarowolfx
changed the title
alvarowolfx
added
kokoro:force-run
gcf-owl-bot
Bot
removed
the
owlbot:run
yoshi-kokoro
removed
the
kokoro:force-run
yirutang
reviewed
Apr 26, 2024
shollyman
reviewed
Apr 29, 2024
yirutang
reviewed
Apr 29, 2024
alvarowolfx
added
the
owlbot:run
gcf-owl-bot
Bot
removed
the
owlbot:run
alvarowolfx
added
the
owlbot:run
gcf-owl-bot
Bot
removed
the
owlbot:run
alvarowolfx
added
the
owlbot:run
gcf-owl-bot
Bot
removed
the
owlbot:run
feywind
approved these changes
Apr 30, 2024
| this.trace('data arrived', response); | ||
| const pw = this.getNextPendingWrite(); | ||
| if (!pw) { | ||
| this.trace('data arrived', response, this._pendingWrites.length); |
There was a problem hiding this comment.
This might be a place that could benefit from the logger I'm working on elsewhere. (Probably said that already, but specifically here...)
leahecole
reviewed
May 1, 2024
| this._open = false; | ||
| this._retrySettings = { | ||
| enableWriteRetries: false, | ||
| maxRetryAttempts: 4, |
Contributor
There was a problem hiding this comment.
Why these values? Can they be overridden?
Contributor
Author
There was a problem hiding this comment.
they can be overwritten via the enableWriteRetries and setMaxRetryAttempts methods. The default values came from the Go ManagedWriter - https://github.com/googleapis/google-cloud-go/blob/5a8c88956ea11eaf3e8aa8cffadfca06bf58c51d/bigquery/storage/managedwriter/retry.go#L31
alvarowolfx
commented
May 2, 2024
Contributor
Author
alvarowolfx
left a comment
alvarowolfx
left a comment
There was a problem hiding this comment.
answered some question and pushed updates to the PR cc @leahecole @feywind
| this._open = false; | ||
| this._retrySettings = { | ||
| enableWriteRetries: false, | ||
| maxRetryAttempts: 4, |
Contributor
Author
There was a problem hiding this comment.
they can be overwritten via the enableWriteRetries and setMaxRetryAttempts methods. The default values came from the Go ManagedWriter - https://github.com/googleapis/google-cloud-go/blob/5a8c88956ea11eaf3e8aa8cffadfca06bf58c51d/bigquery/storage/managedwriter/retry.go#L31
leahecole
reviewed
May 2, 2024
alvarowolfx
added
the
owlbot:run
gcf-owl-bot
Bot
removed
the
owlbot:run
This was referenced May 6, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fix an issue that can happen when an error is returned by the service side and we trigger a reconnect, in some cases where a in-flight/pending write is still waiting for response can be left without any ack/nack.
Towards internal b/329875851