Skip to content

Storage: blob.update_storage_class() does not work for large files #6524

@erikwebb

Description

@erikwebb

Blob.update_storage_class() does not work the same as Blob.rewrite() despite sharing a common underlying API. When multiple API calls are required to complete the rewrite operation, Blob.update_storage_class() will fail after the first call. Blob.update_storage_class() assumes that api_response['resource'] exists. However, according to the docs, this is only true when the operation completes on the first operation. Instead, I think that Blob.update_storage_class() should just leverage the logic in Blob.rewrite(), which handles this case already. I will also look to see how gsutil implements this.

Stack trace

Traceback (most recent call last):
  File "rehost.py", line 31, in <module>
    blob.update_storage_class('COLDLINE')
  File "/var/lib/mysql/.local/lib/python2.7/site-packages/google/cloud/storage/blob.py", line 1497, in update_storage_class
    self._set_properties(api_response['resource'])
KeyError: 'resource'
Command exited with non-zero status 1

Implementation

  • Re-implement Blob.update_storage_class() as a wrapper around Blob.rewrite()
  • Update any Blob.update_storage_class() tests
  • Manually test new implementation with a large file

References

How gsutil handles looping through a rewrite operation - https://github.com/GoogleCloudPlatform/gsutil/blob/704781b250314791dda08191a7472d1a72da2c6c/gslib/gcs_json_api.py#L1477-L1526

Metadata

Metadata

Assignees

Labels

api: storageIssues related to the Cloud Storage API.type: questionRequest for information or clarification. Not an issue.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions