Skip to content

PROPOSAL: Storage API consistency fix #632

@dhermes

Description

@dhermes

Guiding principles:

  • Getters and setters should never make HTTP requests. Lazy loading is OK,
    but only when it involves instance creation / other local (non-network bound) behavior. For example, in Bucket.acl this already happens:
@property
def acl(self):
    """Create our ACL on demand."""
    if self._acl is None:
        self._acl = BucketACL(self)
    return self._acl
  • More generally HTTP requests should be limited to explicit method calls. This also rules out constructors loading data.
  • Blob, Bucket, and *ACL (the only nouns) instances should have load(), exists(), create(), delete(), and update() methods. This design gives rise to code like
blob = Blob('/remote/path.txt', bucket=bucket, properties=properties)
try:
    blob.load()  # Just metadata
except NotFound:
    blob.upload_from_file(filename)  # Sends metadata from properties

(this maybe screams for get_or_create(), we'll feel it out as we develop). It's unclear if it's worth making a distinction between storage.NOUN.update <--> PUT and storage.NOUN.patch <--> PATCH. (As of right now, we don't implement PUT / update anywhere.)

  • exists() should use fields in the requests to minimize the payload.
  • A Connection should not be required to be bound to any object (one of the nouns Bucket, ACL, or Blob) but should be an optional argument to methods which actually talk to the API.
  • We should strongly consider adding a last_updated field to classes to indicate the last time the values were updated (from the server).
  • For list methods: List all buckets top-level, e.g.
storage.get_all_buckets(connection=optional_connection)

and then bucket.get_all_objects(). It's unclear how the other 3 nouns (objectAccessControls, bucketAccessControls and defaultObjectAccessControls) will handle this. Right now they are handled via ObjectACL.reload() and BucketACL.reload() (a superclass of DefaultObjectACL).

  • Implicit behavior (default project, default bucket and/or default connection) should be used wherever possible (and documented)

@tseaver Please weigh in. This was inspired by our discussion at the end of #604.

/cc @thobrla I'd like to close #545 and focus on this (possibly broken down into sub-bugs). Does that sound OK?

Metadata

Metadata

Labels

api: storageIssues related to the Cloud Storage API.

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions