merge pubsub-hp into master#1593
Conversation
… the master branch
* merge from master * make pubsub high perf compile * make AckDeadlineRenewerTest pass
* use fewer guava classes in public surface - SubscriberImpl used to expose AbstractService. We just make this package-private. - SubscriberStats defines its own Stats type.
* make AutoValue work in PubSub Use error_prone_annotations so we don't clobber classpath with Java8 classes. Migrate SubscriberStats and PublisherStats to AutoValue
…ss to use containment in favor of inheritance
* Update * Fixing a race condition on shutdown
Some of our tests seem to still be flaking with timeout, eg https://travis-ci.org/GoogleCloudPlatform/google-cloud-java/jobs/185655553 Making the test not quiet should give us an idea of what's timing out.
* delete pull-related methods from PubSub Instead, provide a way to create the Subscriber object. * also delete modifyAckDeadline * google-java-format changed lines
This commit tries to deflake failures reported in https://ci.appveyor.com/project/garrettjonesgoogle/google-cloud-java/build/57 . The failure seems to come from `FakeSubscriberServiceImpl::sendError`. When calling `getAndAdvanceCurrentStream`, we waited for there to be a subscription by waiting on the `subscriptionInitialized` variable. The variable is notified before `addOpenedStream` is called though, and so it's possible that `getAndAdvanceCurrentStream` will see a list without any opened stream. The fix is to wait for at least one stream to be registered first.
* delete publish-related methods from PubSub
* make pubsub work with new gax update documentation links too
* Delete the handwritten Message type
* delete handwritten PushConfig * update code samples
* Delete handwritten PubSub layer * make Publisher/Subscriber builders public * also delete PubSubRpc
* fix doc annotations
* clean up Publisher and Subscriber surface
- PUBSUB_API_ADDRESS and PUBSUB_API_SCOPE are removed.
We now use {Publisher,Subscriber}Settings instead.
- Default settings are moved into the Builder class,
so that they have package-private visibility.
We can make them public later if we want to.
- Subscriber doc is fixed so that it is not chopped off mid-sentence.
- Subscriber doc mistakenly referred to the Publisher. This is now
fixed.
This bug manifests as testBundleAcks deadlocking. The test 1. publishes a series of messages to a mock server, 2. waits to receive them back, 3. and acknowledges them. For performance reasons, the client does not immediately send the acknowledgement request to the server. Instead, it sends acks in batch every 100ms. Using a fake clock, the test advances the time by 100ms, sending the ack-batch, then verify that the mock server receives the acks. The bug is in step 2. The test thread waits by waiting on a CountDownLatch, which is counted down by GRPC thread calling receiveMessage(). However, the method decrements the latch before acking the message. On a bad day, the test thread can wake up, advance the clock, and send the ack-batch before the GRPC thread could add the message to the batch. The test then waits for the server to receive an ack it never sent, deadlocking the test. The fix is for receiveMessage() to ack the message before decrementing the counter.
* Update version to 0.8.1-SNAPSHOT (#1467) Also, update versions in README to 0.8.0 * Add link to Maven Central for maven-central badge. (#1468) Used to link to the image, which wasn't super useful. * fix more races in pubsub tests Previously BlockingProcessStreamReader has a terminate() method, used to tell the Reader to stop reading from the emulator process. This causes an inter-process race. If the Reader stops before reading emulator's output, the emulator process will hang as it tries to write to stdout/stderr as there's no one to read from the other side of the pipe. Since there is no way to safely stop the Reader, this commit deletes the method and its associated test. Additionally, the timeout for LocalSystemTest is increased to 3 minutes, since the emulator, somehow, consistently takes just longer than a minute to shut down. * Regenerating SPI layer (#1501) * Converting Error Reporting and Monitoring to use resource name types * Removing formatX/parseX methods from pubsub, converting usage of the same to resource name types * New methods in Logging and PubSub * Updating grpc dependency to 1.0.3 (#1504) * Release 0.8.1 (#1512) * Fix code snippet (wrong method name) in README.md Original code snippet in _"Querying data"_ section: `..while (!queryResponse.jobComplete()) {..` This results in a compile error: _"Cannot resolve method jobComplete()"_ The correct method is `jobCompleted()` * Updating version in README files. [ci skip] * Update version to 0.8.2-alpha-SNAPSHOT * Allow path in URIs passed to newFileSystem (#1470) * This makes it easier for users who start with a URI describing a full path to get a FileSystem that can work with that path, since they no longer have to needlessly remove the path from the URI. Note that Oracle's description of newFileSystem [1] puts no restriction on the passed URI. [1] https://docs.oracle.com/javase/7/docs/api/java/nio/file/FileSystems.html#newFileSystem(java.net.URI,%20java.util.Map) * Preventing logging re-entrance at FINE level (#1523) * Preventing logging re-entrance at FINE level Also: * Reducing the scope of synchronized blocks * Removing logger exclusions except for Http2FrameLogger * Add a PathMatcher for CloudStorageFileSystem (#1469) Add a test, as well. We reuse the default PathMatcher since it does a fine job of globbing files. * Set timestamp from LogRecord (#1533) * Initialize the default MonitoredResource from a GAE environment (#1535) * BigQuery: Add support to FormatOptions for AVRO #1441 Added new constant in FormatOptions, and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required. * Reverting changed commited by mistake before review. * BigQuery: Add support to FormatOptions for AVRO #1441 Added new constant in FormatOptions and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required. * use RpcFuture and remove old BundlingSettings (#1572) * use RpcFuture and remove old BundlingSettings * Release 0.8.2 Note: This version was accidentally released to Sonatype because of experimenting with deployment commands, combined with the fact that autoReleaseAfterClose is set to true. Since releases can't be taken back, we might as well own up to the release and push the code forward. * Updating version in README files. [ci skip] * Fixing javadoc error in GaeFlexLoggingEnhancer (#1582) * get tests to compile and pass
This is the same race as #1495 but in sendStreamResponse rather than sendError.
|
Take care of any items I mentioned that are trivial to fix, and file issues for ones that are not trivial & fix them after merging. |
| @@ -1,5 +1,5 @@ | |||
| /* | |||
| * Copyright 2016 Google Inc. All Rights Reserved. | |||
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
| </dependency> | ||
| <dependency> | ||
| <groupId>com.google.guava</groupId> | ||
| <artifactId>guava</artifactId> |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
| </dependency> | ||
| <dependency> | ||
| <groupId>joda-time</groupId> | ||
| <artifactId>joda-time</artifactId> |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
| <dependency> | ||
| <groupId>log4j</groupId> | ||
| <artifactId>log4j</artifactId> | ||
| <version>1.2.17</version> |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
| <artifactId>mockito-all</artifactId> | ||
| <version>1.9.5</version> | ||
| <scope>test</scope> | ||
| </dependency> |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
| <artifactId>os-maven-plugin</artifactId> | ||
| <version>1.4.0.Final</version> | ||
| </extension> | ||
| </extensions> |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
| /** | ||
| * A barrier kind of object that helps to keep track and synchronously wait on pending messages. | ||
| */ | ||
| class MessagesWaiter { |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
|
@garrettjonesgoogle PTAL |
|
LGTM |
…to v3.17.0 (#1593) * chore(deps): update dependency com.google.cloud:google-cloud-logging to v3.17.0 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Joe Wang <[email protected]>
…to v3.17.0 (googleapis#1593) * chore(deps): update dependency com.google.cloud:google-cloud-logging to v3.17.0 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Joe Wang <[email protected]>
…jdbc to v2.17.1 (googleapis#1593) [](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [com.google.cloud:google-cloud-spanner-jdbc](https://togithub.com/googleapis/java-spanner-jdbc) | `2.17.0` -> `2.17.1` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>googleapis/java-spanner-jdbc (com.google.cloud:google-cloud-spanner-jdbc)</summary> ### [`v2.17.1`](https://togithub.com/googleapis/java-spanner-jdbc/blob/HEAD/CHANGELOG.md#2171-2024-04-30) [Compare Source](https://togithub.com/googleapis/java-spanner-jdbc/compare/v2.17.0...v2.17.1) ##### Dependencies - Update dependency com.google.cloud:google-cloud-spanner-bom to v6.65.1 ([#&googleapis#8203;1588](https://togithub.com/googleapis/java-spanner-jdbc/issues/1588)) ([0a3960d](https://togithub.com/googleapis/java-spanner-jdbc/commit/0a3960d3e8312465852d43e1dc5e6c4081ef1b67)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/googleapis/java-spanner-jdbc). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zMzEuMCIsInVwZGF0ZWRJblZlciI6IjM3LjMzMS4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
No description provided.