Skip to content

Bigtable: Expose an API to allow users to create their own batches for MutateRows#3129

Merged
garrettjonesgoogle merged 6 commits intogoogleapis:masterfrom
igorbernstein2:batching-raw
Apr 25, 2018
Merged

Bigtable: Expose an API to allow users to create their own batches for MutateRows#3129
garrettjonesgoogle merged 6 commits intogoogleapis:masterfrom
igorbernstein2:batching-raw

Conversation

@igorbernstein2
Copy link
Copy Markdown
Contributor

@igorbernstein2 igorbernstein2 commented Apr 5, 2018

This has 2 objectives:

  1. Expose enough primitives for users to workaround the lack of batch flushing.
    This would allow me to experiment with @pongad's batching work and unblock integration with apache beam

  2. Unblock a transition path from the existing bigtable-client-core.
    The current plan for a future transition from bigtable-client-core is to implement the BigtableDataClient using this veneer. That interface sits below batching in the current client. To enable this transition path, I need to expose the manual batching primitive

@igorbernstein2
Expose the api to allow users to create their own batches for MutateR…
4efcde6 
…ows.

This has 2 objectives:
1. Expose enough primitives for users to workaround the lack of batch flushing.
This would allow me to experiment with pongad's batching work and unblock integration with apache beam

2. Unblock a transition path from the existing bigtable-client-core.
The current plan for a future transition from bigtable-client-core is to implement the BigtableDataClient using this veneer. That interface sits below batching in the current client. To enable this transition path, I need to expose the manual batching primitive
@igorbernstein2 igorbernstein2 requested a review from pongad as a code owner April 5, 2018 20:15
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Apr 5, 2018
@igorbernstein2
Copy link
Copy Markdown
Contributor Author

igorbernstein2 commented Apr 12, 2018

@pongad @garrettjonesgoogle PTAL

@pongad
Copy link
Copy Markdown
Contributor

pongad commented Apr 12, 2018

The code LGTM. IIUC, we're exposing createBulkMutateRowsCallable so that we can batch manually is that right?

@igorbernstein2
Copy link
Copy Markdown
Contributor Author

igorbernstein2 commented Apr 12, 2018

Yep. Manual batching is needed to transition the current client to veneer and I plan to leave that feature around for advanced use cases

garrettjonesgoogle
garrettjonesgoogle reviewed Apr 18, 2018
View reviewed changes
Comment thread ...-cloud-bigtable/src/test/java/com/google/cloud/bigtable/data/v2/models/BulkMutationTest.java Outdated
.setColumnQualifier(
ByteString.copyFromUtf8("fake-qualifier3"))
.setTimestampMicros(3_000)
.setValue(ByteString.copyFromUtf8("fake-value3")))))

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

Comment thread ...java/com/google/cloud/bigtable/data/v2/stub/mutaterows/BulkMutateRowsUserFacingCallable.java Outdated
import com.google.cloud.bigtable.data.v2.models.BulkMutation;

/**
* A callable to convert userfacing {@link BulkMutation}s to {@link MutateRowsRequest}s.

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

Comment thread google-cloud-bigtable/src/main/java/com/google/cloud/bigtable/data/v2/models/BulkMutation.java
* <p>This class is meant for manual batching, while {@link BulkMutationBatcher} is meant for
* automatic batching with flow control.
*/
public final class BulkMutation implements Serializable {

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

Comment thread ...table/src/main/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStubSettings.java
public BatchingCallSettings<RowMutation, Void> mutateRowsSettings() {
return mutateRowsSettings;
/** Returns the object with the settings used for manually batched calls to MutateRows. */
public BatchingCallSettings<RowMutation, Void> bulkMutateRowsSettings() {

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

@igorbernstein2
Copy link
Copy Markdown
Contributor Author

igorbernstein2 commented Apr 20, 2018

All feedback should be addressed, PTAL

garrettjonesgoogle
garrettjonesgoogle approved these changes Apr 25, 2018
View reviewed changes
Copy link
Copy Markdown
Contributor

@garrettjonesgoogle garrettjonesgoogle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after you address my requested comment

Comment thread ...table/src/main/java/com/google/cloud/bigtable/data/v2/stub/EnhancedBigtableStubSettings.java
public BatchingCallSettings<RowMutation, Void> mutateRowsSettings() {
return mutateRowsSettings;
/** Returns the object with the settings used for manually batched calls to MutateRows. */
public BatchingCallSettings<RowMutation, Void> bulkMutateRowsSettings() {

This comment was marked as spam.

Sign in to view

@igorbernstein2
Copy link
Copy Markdown
Contributor Author

igorbernstein2 commented Apr 25, 2018

Updated, please merge when ready

@garrettjonesgoogle garrettjonesgoogle merged commit 60b1d27 into googleapis:master Apr 25, 2018
chingor13 pushed a commit that referenced this pull request Mar 24, 2026
@diegomarquezp @gcf-owl-bot
feat: enable hermetic library generation (#3129)
f0d5dfb 
* feat: enable hermetic library generation

* fix config yaml syntax

* do not map runners home folder

* use copyright update comittish

* correct proto_path

* update protoc

* preserve pr_description

* update gapic_generator_version to 2.41.0

* infer image tag from config yaml

* correct workflow name

* update config scripts and yamls

* remove old update_googleapis_committish workflow

* sync config structure with that of google-cloud-java

* remove quotes from config yamls

* fix typo in update_generation_config.yaml

* correct

* quote codeowners_team in generation config

* update generator version

* fix path to hermetic_library_generation

* remove escaped newlines

* fixes to hermetic_library_generation

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
chingor13 pushed a commit that referenced this pull request Mar 30, 2026
@renovate-bot
deps: update sdk-platform-java dependencies (#3129)
bb30d4d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pongad pongad Awaiting requested review from pongad

1 more reviewer

@garrettjonesgoogle garrettjonesgoogle garrettjonesgoogle approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@igorbernstein2 @pongad @garrettjonesgoogle @googlebot