Skip to content
This repository was archived by the owner on Mar 9, 2026. It is now read-only.

feat: add use_topic_schema for Cloud Storage Subscriptions#1154

Merged
parthea merged 19 commits intomainfrom
owl-bot-copy
Jul 6, 2024
Merged

feat: add use_topic_schema for Cloud Storage Subscriptions#1154
parthea merged 19 commits intomainfrom
owl-bot-copy

Conversation

@gcf-owl-bot
Copy link
Copy Markdown
Contributor

@gcf-owl-bot gcf-owl-bot bot commented Apr 29, 2024

  • Regenerate this pull request now.

PiperOrigin-RevId: 649277844

Source-Link: googleapis/googleapis@fd7efa2

Source-Link: https://github.com/googleapis/googleapis-gen/commit/50be251329d8db5b555626ebd4886721f547d3cc
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNTBiZTI1MTMyOWQ4ZGI1YjU1NTYyNmViZDQ4ODY3MjFmNTQ3ZDNjYyJ9

BEGIN_NESTED_COMMIT
feat: add use_topic_schema for Cloud Storage Subscriptions
PiperOrigin-RevId: 643385701

BEGIN_COMMIT_OVERRIDE
feat: add use_topic_schema for Cloud Storage Subscriptions
feat: add service_account_email for export subscriptions
END_COMMIT_OVERRIDE

Source-Link: googleapis/googleapis@f29a82a

Source-Link: https://github.com/googleapis/googleapis-gen/commit/f8f3f60ad2ab4e81fe024368ad0430900174e82b
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZjhmM2Y2MGFkMmFiNGU4MWZlMDI0MzY4YWQwNDMwOTAwMTc0ZTgyYiJ9
END_NESTED_COMMIT
BEGIN_NESTED_COMMIT
chore: Update gapic-generator-python to v1.18.0
PiperOrigin-RevId: 638650618

Source-Link: googleapis/googleapis@6330f03

Source-Link: https://github.com/googleapis/googleapis-gen/commit/44fa4f1979dc45c1778fd7caf13f8e61c6d1cae8
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNDRmYTRmMTk3OWRjNDVjMTc3OGZkN2NhZjEzZjhlNjFjNmQxY2FlOCJ9
END_NESTED_COMMIT
BEGIN_NESTED_COMMIT
feat: add service_account_email for export subscriptions
PiperOrigin-RevId: 638641437

Source-Link: googleapis/googleapis@a3016a8

Source-Link: https://github.com/googleapis/googleapis-gen/commit/2001c330aee62a9d4e8a9852d8e5bf2662818ad4
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMjAwMWMzMzBhZWU2MmE5ZDRlOGE5ODUyZDhlNWJmMjY2MjgxOGFkNCJ9
END_NESTED_COMMIT
BEGIN_NESTED_COMMIT
chore: Update gapic-generator-python to v1.17.1
PiperOrigin-RevId: 629071173

Source-Link: googleapis/googleapis@4afa392

Source-Link: https://github.com/googleapis/googleapis-gen/commit/16dbbb4d0457db5e61ac9f99b0d52a46154455ac
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMTZkYmJiNGQwNDU3ZGI1ZTYxYWM5Zjk5YjBkNTJhNDYxNTQ0NTVhYyJ9
END_NESTED_COMMIT

PiperOrigin-RevId: 629071173

Source-Link: googleapis/googleapis@4afa392

Source-Link: googleapis/googleapis-gen@16dbbb4
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMTZkYmJiNGQwNDU3ZGI1ZTYxYWM5Zjk5YjBkNTJhNDYxNTQ0NTVhYyJ9
@trusted-contributions-gcf trusted-contributions-gcf bot added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels Apr 29, 2024
@product-auto-label product-auto-label bot added the size: xl Pull request size is extra large. label Apr 29, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Apr 29, 2024
@product-auto-label product-auto-label bot added the api: pubsub Issues related to the googleapis/python-pubsub API. label Apr 29, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Apr 29, 2024
@trusted-contributions-gcf trusted-contributions-gcf bot added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels May 30, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 30, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 30, 2024
PiperOrigin-RevId: 638641437

Source-Link: googleapis/googleapis@a3016a8

Source-Link: googleapis/googleapis-gen@2001c33
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMjAwMWMzMzBhZWU2MmE5ZDRlOGE5ODUyZDhlNWJmMjY2MjgxOGFkNCJ9
@gcf-owl-bot gcf-owl-bot bot changed the title chore: Update gapic-generator-python to v1.17.1 feat: add service_account_email for export subscriptions May 30, 2024
@trusted-contributions-gcf trusted-contributions-gcf bot added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels May 30, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 30, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 30, 2024
PiperOrigin-RevId: 638650618

Source-Link: googleapis/googleapis@6330f03

Source-Link: googleapis/googleapis-gen@44fa4f1
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNDRmYTRmMTk3OWRjNDVjMTc3OGZkN2NhZjEzZjhlNjFjNmQxY2FlOCJ9
@gcf-owl-bot gcf-owl-bot bot changed the title feat: add service_account_email for export subscriptions chore: Update gapic-generator-python to v1.18.0 May 30, 2024
@trusted-contributions-gcf trusted-contributions-gcf bot added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels May 30, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 30, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 30, 2024
@trusted-contributions-gcf trusted-contributions-gcf bot added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels May 30, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 30, 2024
@trusted-contributions-gcf trusted-contributions-gcf bot added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels Jul 2, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 2, 2024
@trusted-contributions-gcf trusted-contributions-gcf bot added the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 2, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 2, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 2, 2024
@mukund-ananthu mukund-ananthu self-requested a review July 2, 2024 17:44
@mukund-ananthu
Copy link
Copy Markdown
Contributor

cc: @parthea Could we rope in the code generator library owners for GAPIC for them to fix the code generator libraries to ensure the code doesn't break unit tests like they are now:

self = <google.api_core.grpc_helpers_async._WrappedUnaryUnaryCall object at 0x7fea08ad9e50>

    def __await__(self):
        try:
>           response = yield from self._call.__await__()

.nox/unit-3-8/lib/python3.8/site-packages/google/api_core/grpc_helpers_async.py:81: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_AioCall of RPC that terminated with:
	status = The request is missing a valid API key.
	details = "StatusCode.PERMIS...essage:"The request is missing a valid API key.", grpc_status:7, created_time:"2024-07-02T17:[44](https://github.com/googleapis/python-pubsub/actions/runs/9765604292/job/26956648919?pr=1154#step:5:45):25.648799599+00:00"}"
>

    def __await__(self) -> Generator[Any, None, ResponseType]:
        """Wait till the ongoing RPC request finishes."""
        try:
            response = yield from self._call_response
        except asyncio.CancelledError:
            # Even if we caught all other CancelledError, there is still
            # this corner case. If the application cancels immediately after
            # the Call object is created, we will observe this
            # `CancelledError`.
            if not self.cancelled():
                self.cancel()
            raise
    
        # NOTE(lidiz) If we raise RpcError in the task, and users doesn't
        # 'await' on it. AsyncIO will log 'Task exception was never retrieved'.
        # Instead, if we move the exception raising here, the spam stops.
        # Unfortunately, there can only be one 'yield from' in '__await__'. So,
        # we need to access the private instance variable.
        if response is cygrpc.EOF:
            if self._cython_call.is_locally_cancelled():
                raise asyncio.CancelledError()
            else:
>               raise _create_rpc_error(
                    self._cython_call._initial_metadata,
                    self._cython_call._status,
                )
E               grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with:
E               	status = StatusCode.PERMISSION_DENIED
E               	details = "The request is missing a valid API key."
E               	debug_error_string = "UNKNOWN:Error received from peer ipv4:172.253.122.95:443 {grpc_message:"The request is missing a valid API key.", grpc_status:7, created_time:"2024-07-02T17:44:25.6[48](https://github.com/googleapis/python-pubsub/actions/runs/9765604292/job/26956648919?pr=1154#step:5:49)799599+00:00"}"
E               >

.nox/unit-3-8/lib/python3.8/site-packages/grpc/aio/_call.py:318: AioRpcError

The above exception was the direct cause of the following exception:

creds = <google.auth.credentials.AnonymousCredentials object at 0x7fea0667f340>

    @pytest.mark.asyncio
    async def test_sync_pull_warning_if_return_immediately_async(creds):
        from google.pubsub_v1.services.subscriber.async_client import SubscriberAsyncClient
    
        client = SubscriberAsyncClient(credentials=creds)
        subscription_path = "projects/foo/subscriptions/bar"
    
        patcher = mock.patch(
            "google.pubsub_v1.services.subscriber.async_client.gapic_v1.method_async.wrap_method",
            new=mock.AsyncMock,
        )
    
        with patcher, pytest.warns(
            DeprecationWarning,
            match="The return_immediately flag is deprecated and should be set to False",
        ) as warned:
>           await client.pull(subscription=subscription_path, return_immediately=True)

tests/unit/pubsub_v1/subscriber/test_subscriber_client.py:311: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
google/pubsub_v1/services/subscriber/async_client.py:1289: in pull
    response = await rpc(
.nox/unit-3-8/lib/python3.8/site-packages/google/api_core/retry_async.py:223: in retry_wrapped_func
    return await retry_target(
.nox/unit-3-8/lib/python3.8/site-packages/google/api_core/retry_async.py:121: in retry_target
    return await asyncio.wait_for(
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/asyncio/tasks.py:[49](https://github.com/googleapis/python-pubsub/actions/runs/9765604292/job/26956648919?pr=1154#step:5:50)4: in wait_for
    return fut.result()
/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/asyncio/tasks.py:695: in _wrap_awaitable
    return (yield from awaitable.__await__())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <google.api_core.grpc_helpers_async._WrappedUnaryUnaryCall object at 0x7fea08ad9e[50](https://github.com/googleapis/python-pubsub/actions/runs/9765604292/job/26956648919?pr=1154#step:5:51)>

    def __await__(self):
        try:
            response = yield from self._call.__await__()
            return response
        except grpc.RpcError as rpc_error:
>           raise exceptions.from_grpc_error(rpc_error) from rpc_error
E           google.api_core.exceptions.PermissionDenied: 403 The request is missing a valid API key.

.nox/unit-3-8/lib/python3.8/site-packages/google/api_core/grpc_helpers_async.py:84: PermissionDenied
- generated xml file: /home/runner/work/python-pubsub/python-pubsub/unit_3.8_sponge_log.xml -
=========================== short test summary info ============================
FAILED tests/unit/pubsub_v1/subscriber/test_subscriber_client.py::test_sync_pull_warning_if_return_immediately_async - google.api_core.exceptions.PermissionDenied: 403 The request is missing a valid API key.
1 failed, 16[52](https://github.com/googleapis/python-pubsub/actions/runs/9765604292/job/26956648919?pr=1154#step:5:53) passed in 21.49s
nox > Command py.test --quiet --junitxml=unit_3.8_sponge_log.xml --cov=google/cloud --cov=tests/unit --cov-append --cov-config=.coveragerc --cov-report= --cov-fail-under=0 tests/unit failed with exit code 1
nox > Session unit-3.8 failed.

@parthea
Copy link
Copy Markdown
Contributor

parthea commented Jul 2, 2024

Following up on #1154 (comment), the test that is failing is part of the handwritten code. We can follow up on a long term solution via googleapis/google-cloud-python#16295. In the meantime, the following change fixes the issue.

(py3112env) partheniou@partheniou-vm-3:~/git/python-pubsub$ git diff
diff --git a/tests/unit/pubsub_v1/subscriber/test_subscriber_client.py b/tests/unit/pubsub_v1/subscriber/test_subscriber_client.py
index a09d85b..5cd2cae 100644
--- a/tests/unit/pubsub_v1/subscriber/test_subscriber_client.py
+++ b/tests/unit/pubsub_v1/subscriber/test_subscriber_client.py
@@ -298,10 +298,8 @@ async def test_sync_pull_warning_if_return_immediately_async(creds):
 
     client = SubscriberAsyncClient(credentials=creds)
     subscription_path = "projects/foo/subscriptions/bar"
-
-    patcher = mock.patch(
-        "google.pubsub_v1.services.subscriber.async_client.gapic_v1.method_async.wrap_method",
-        new=mock.AsyncMock,
+    patcher = mock.patch.object(
+        type(client.transport.pull), "__call__", new_callable=mock.AsyncMock,
     )
 
     with patcher, pytest.warns(

@trusted-contributions-gcf trusted-contributions-gcf bot added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels Jul 2, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 2, 2024
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 2, 2024
@trusted-contributions-gcf trusted-contributions-gcf bot added kokoro:force-run Add this label to force Kokoro to re-run the tests. owlbot:run Add this label to trigger the Owlbot post processor. labels Jul 2, 2024
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 2, 2024
@snippet-bot
Copy link
Copy Markdown

snippet-bot bot commented Jul 4, 2024

No region tags are edited in this PR.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

api: pubsub Issues related to the googleapis/python-pubsub API. owl-bot-copy size: xl Pull request size is extra large.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants