#260: break datastore cycles via factories #268

tseaver · 2014-10-20T19:53:44Z

My take on breaking the import cycles in 'gcloud.datastore' by adding a "factory registry" for the classes normally imported.

Note that some classes have more than one factory (class methods like'from_protobuf' and 'from_path'): each is registered under a separate name.

Also drops 'Dataset.query' and 'Dataset.entity' convenience APIs, pointing instead to constructing Query and Entity directly.

Addresses #260 (for 'gcloud.datastore').

gcloud/datastore/__init__.py

coveralls · 2014-10-20T20:31:15Z

Coverage remained the same when pulling 6a77c1a on tseaver:260-break_datastore_cycles_via_factories into 99e4d53 on GoogleCloudPlatform:master.

coveralls · 2014-10-20T20:36:26Z

Coverage remained the same when pulling 6a77c1a on tseaver:260-break_datastore_cycles_via_factories into 99e4d53 on GoogleCloudPlatform:master.

gcloud/datastore/_helpers.py

dhermes · 2014-10-20T22:30:37Z

I think this concept is a little too Pythonic.

In other words, cyclic imports are more desirable to me than a module global which is mutable.

Having _FACTORIES change at run time makes the codebase impossible to reason about statically as well.

tseaver · 2014-10-20T23:14:40Z

@dhermes I don't agree about the "mutable global": modules themselves get mutated during import. Note that the _FACTORIES bit (originally _factories, until pylint's bitching made me changed it) gets changed only as the registering module is imported (except for tests). Cyclical imports are a sign of much worse muddiness than a simple, controlled registry.

coveralls · 2014-10-20T23:22:36Z

Coverage remained the same when pulling cb2352f on tseaver:260-break_datastore_cycles_via_factories into 99e4d53 on GoogleCloudPlatform:master.

dhermes · 2014-10-21T01:11:47Z

Potential compromise:

Can we add very clear docs to _helpers and give references to all the places we expect a class to be registered?

dhermes · 2014-10-21T01:12:33Z

I also realized that the "run time" complaints are contradictory since a import inside a method/function happens at run time.

tseaver · 2014-10-21T04:12:09Z

1305e81 encapsulates the registry as an object with 'register', 'get', and 'invoke' methods. It also ensures that the relevant modules get imported early (to ensure that the registry is populated).

coveralls · 2014-10-21T04:22:56Z

Coverage remained the same when pulling 1305e81 on tseaver:260-break_datastore_cycles_via_factories into 99e4d53 on GoogleCloudPlatform:master.

gcloud/datastore/_helpers.py

coveralls · 2014-10-21T17:22:54Z

Coverage remained the same when pulling fd9cd02 on tseaver:260-break_datastore_cycles_via_factories into 99e4d53 on GoogleCloudPlatform:master.

coveralls · 2014-10-21T17:28:58Z

Coverage remained the same when pulling fd9cd02 on tseaver:260-break_datastore_cycles_via_factories into 99e4d53 on GoogleCloudPlatform:master.

coveralls · 2014-10-22T18:34:00Z

Coverage remained the same when pulling ccc79fd on tseaver:260-break_datastore_cycles_via_factories into c2a38fb on GoogleCloudPlatform:master.

Point instead to constructing Query and Entity directly. Addresses #260, but only partially, for gcloud.datastore.

Work toward breaking cycles based on import-for-construction.

Note that some classes have more than one factory (class methodsd like 'from_protobuf' and 'from_path'). Each is registered under a separate name. Addresses #260 (for gcloud.datastore).

Incorporates feedback from @dhermees.

@dhermes

Incorporate feedback from @dhermes.

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <[email protected]>

- [ ] Regenerate this pull request now. docs: clarified wording around Cloud Storage usage PiperOrigin-RevId: 433282831 Source-Link: googleapis/googleapis@0e87dc7 Source-Link: googleapis/googleapis-gen@1928631 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMTkyODYzMWYzZWU5MTQ4OGZlMDdiM2Q4MTg4YTUyZDEwODYyNDUzYSJ9

* chore: Update gapic-generator-python to v1.8.4 PiperOrigin-RevId: 507808936 Source-Link: googleapis/googleapis@64cf849 Source-Link: googleapis/googleapis-gen@53c48ca Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNTNjNDhjYWMxNTNkM2IzN2YzZDJjMmRlYzQ4MzBjZmQ5MWVjNDE1MyJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

#268)

chore: relocate owl bot post processor

…le (#268) Source-Link: googleapis/synthtool@a7ed11e Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:6e7328583be8edd3ba8f35311c76a1ecbc823010279ccb6ab46b7a76e25eafcc Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

* chore: Update gapic-generator-python to v1.8.5 PiperOrigin-RevId: 511892190 Source-Link: googleapis/googleapis@a45d9c0 Source-Link: googleapis/googleapis-gen@1907294 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMTkwNzI5NGIxZDgzNjVlYTI0ZjhjNWYyZTA1OWE2NDEyNGM0ZWQzYiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <[email protected]>

fix(deps): require proto-plus >= 1.22.0

* chore: upgrade gapic-generator-java, gax-java and gapic-generator-python PiperOrigin-RevId: 423842556 Source-Link: googleapis/googleapis@a616ca0 Source-Link: googleapis/googleapis-gen@29b938c Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMjliOTM4YzU4YzFlNTFkMDE5ZjJlZTUzOWQ1NWRjMGEzYzg2YTkwNSJ9 * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

#268)

Source-Link: googleapis/synthtool@f15cc72 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:bc5eed3804aec2f05fad42aacf973821d9500c174015341f721a984a0825b6fd

Source-Link: googleapis/synthtool@993985f Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:1894490910e891a385484514b22eb5133578897eb5b3c380e6d8ad475c6647cd

* ci(python): run lint / unit tests / docs as GH actions Source-Link: googleapis/synthtool@57be0cd Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:ed1f9983d5a935a89fe8085e8bb97d94e41015252c5b6c9771257cf8624367e6 * add commit to trigger gh actions * work around template bug Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <[email protected]>

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

Source-Link: https://togithub.com/googleapis/synthtool/commit/d0f51a0c2a9a6bcca86911eabea9e484baadf64b Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:240b5bcc2bafd450912d2da2be15e62bc6de2cf839823ae4bf94d4f392b451dc

@busunkim96

- [ ] Regenerate this pull request now. chore: fix docstring for first attribute of protos committer: @busunkim96 PiperOrigin-RevId: 401271153 Source-Link: googleapis/googleapis@787f8c9 Source-Link: googleapis/googleapis-gen@81decff Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiODFkZWNmZmU5ZmM3MjM5NmE4MTUzZTc1NmQxZDY3YTZlZWNmZDYyMCJ9

Source-Link: googleapis/synthtool@50db768 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:e09366bdf0fd9c8976592988390b24d53583dd9f002d476934da43725adbb978

Source-Link: https://togithub.com/googleapis/synthtool/commit/352b9d4c068ce7c05908172af128b294073bf53c Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:3e3800bb100af5d7f9e810d48212b37812c1856d20ffeafb99ebe66461b61fc7

* chore: remove pb2 file generation during post processing * fix: remove obsolete proto files

Follow-up to googleapis/python-bigquery-storage#225 I noticed we didn't need several of these samples now that BQ Storage API is used by default.

…p/templates/python_library/.kokoro (#268) Source-Link: googleapis/synthtool@e13b22b Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:97b671488ad548ef783a452a9e1276ac10f144d5ae56d98cc4bf77ba504082b4 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

Source-Link: https://togithub.com/googleapis/synthtool/commit/26c7505b2f76981ec1707b851e1595c8c06e90fc Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f946c75373c2b0040e8e318c5e85d0cf46bc6e61d0a01f3ef94d8de974ac6790

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

Source-Link: googleapis/synthtool@3551acd Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:230f7fe8a0d2ed81a519cfc15c6bb11c5b46b9fb449b8b1219b3771bcb520ad2 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

) * fix: Changed `client_info` import `google.api_core.client_info` -> `google.api_core.gapic_v1.client_info` * Add examples for all document initialization options (w/tests) Fixes #266

tseaver added api: datastore Issues related to the Datastore API. hygiene labels Oct 20, 2014

dhermes reviewed Oct 20, 2014
View reviewed changes

gcloud/datastore/__init__.py Outdated

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

tseaver mentioned this pull request Oct 20, 2014

Disentangle import cycles #260

Closed

dhermes reviewed Oct 20, 2014
View reviewed changes

gcloud/datastore/_helpers.py Outdated

This comment was marked as spam.

Sign in to view

dhermes reviewed Oct 21, 2014
View reviewed changes

gcloud/datastore/_helpers.py

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

tseaver added 8 commits October 22, 2014 15:34

Drop 'Dataset.query' and 'Dataset.entity' convenience APIs.

1036f85

Point instead to constructing Query and Entity directly. Addresses #260, but only partially, for gcloud.datastore.

Add a factory registry to datastore._helpers.

1c432a9

Work toward breaking cycles based on import-for-construction.

Break import cycles via a factory registry.

8d05122

Note that some classes have more than one factory (class methodsd like 'from_protobuf' and 'from_path'). Each is registered under a separate name. Addresses #260 (for gcloud.datastore).

Tidy local imports, remove remaining cycles.

3c1d516

Fix build: invoke correct factory when unmarshalling key from pb.

cf29c90

Reorder 'Query.__init__' paramters to match Entity.

31f60d8

Rename '_query_factory' helper -> '_get_factory' to avoid confustion.

3a62db9

Incorporates feedback from @dhermees.

Avoid relative imports.

daf889c

Incorporate feedback from @dhermes.

parthea added a commit that referenced this pull request Jun 4, 2023

chore(main): release 2.9.2 (#268)

a528192

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <[email protected]>

parthea pushed a commit that referenced this pull request Sep 20, 2023

chore(deps): update dependency google-cloud-video-transcoder to v1.6.0 (

06fa9b4

#268)

parthea pushed a commit that referenced this pull request Sep 22, 2023

build: switch to release-please for tagging (#268)

ed5ed6d

parthea pushed a commit that referenced this pull request Sep 22, 2023

chore: relocate owl bot post processor (#268)

87db1f9

chore: relocate owl bot post processor

parthea added a commit that referenced this pull request Sep 22, 2023

fix(deps): allow protobuf < 5.0.0 (#268)

e2d77e0

fix(deps): require proto-plus >= 1.22.0

parthea pushed a commit that referenced this pull request Oct 21, 2023

chore(deps): update dependency google-cloud-resource-manager to v1.1.0 (

51bb6f9

#268)

parthea pushed a commit that referenced this pull request Oct 21, 2023

chore: release 3.5.0 (#268)

e480db3

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

parthea pushed a commit that referenced this pull request Oct 21, 2023

chore(deps): update dependency google-analytics-admin to v0.11.0 (#268)

c755066

parthea pushed a commit that referenced this pull request Oct 21, 2023

chore(quickstart): clarify private pools instructions (#268)

c575a48

parthea added a commit that referenced this pull request Dec 20, 2024

chore: add generation of pb2 files via owlbot/bazel (#268)

4dc815b

* chore: remove pb2 file generation during post processing * fix: remove obsolete proto files

parthea pushed a commit that referenced this pull request Aug 21, 2025

chore: remove unused samples (#268)

d52a0b9

Follow-up to googleapis/python-bigquery-storage#225 I noticed we didn't need several of these samples now that BQ Storage API is used by default.

parthea pushed a commit that referenced this pull request Sep 16, 2025

chore: remove unused samples (#268)

cf7d2cb

Follow-up to googleapis/python-bigquery-storage#225 I noticed we didn't need several of these samples now that BQ Storage API is used by default.

parthea pushed a commit that referenced this pull request Nov 24, 2025

chore: release 1.19.6 (#268)

ee60553

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

parthea pushed a commit that referenced this pull request Nov 26, 2025

Fix some typos in test_urllib3.py (#268)

0c61531

#260: break datastore cycles via factories #268

#260: break datastore cycles via factories #268

Uh oh!

Conversation

tseaver commented Oct 20, 2014

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

coveralls commented Oct 20, 2014

Uh oh!

coveralls commented Oct 20, 2014

Uh oh!

This comment was marked as spam.

Uh oh!

dhermes commented Oct 20, 2014

Uh oh!

tseaver commented Oct 20, 2014

Uh oh!

coveralls commented Oct 20, 2014

Uh oh!

dhermes commented Oct 21, 2014

Uh oh!

dhermes commented Oct 21, 2014

Uh oh!

tseaver commented Oct 21, 2014

Uh oh!

coveralls commented Oct 21, 2014

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

coveralls commented Oct 21, 2014

Uh oh!

coveralls commented Oct 21, 2014

Uh oh!

coveralls commented Oct 22, 2014

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants