Skip to content

storage: Reader uses the xml api but sends json preconditions #4470

Closed

Description

@fejta
  • NewRangerReader uses the XML api to download files.
  • conditionsQuery() uses the JSON API to configure preconditions
  • NewRangerReader calls conditionsQuery() rather than setting the expected precondition headers.

ObjectHandle.NewReader() calls NewRangeReader:

func (o *ObjectHandle) NewReader(ctx context.Context) (*Reader, error) {
return o.NewRangeReader(ctx, 0, -1)
}

This eventually sets req.URL.RawQuery = conditionsQuery(gen, o.conds):

// We wait to assign conditions here because the generation number can change in between reopen() runs.
req.URL.RawQuery = conditionsQuery(gen, o.conds)

conditionsQuery() appends the applicable JSON API URL parameters:

switch {
case conds.GenerationMatch != 0:
appendParam("ifGenerationMatch=", conds.GenerationMatch)
case conds.GenerationNotMatch != 0:
appendParam("ifGenerationNotMatch=", conds.GenerationNotMatch)
case conds.DoesNotExist:
appendParam("ifGenerationMatch=", 0)
}
switch {
case conds.MetagenerationMatch != 0:
appendParam("ifMetagenerationMatch=", conds.MetagenerationMatch)
case conds.MetagenerationNotMatch != 0:
appendParam("ifMetagenerationNotMatch=", conds.MetagenerationNotMatch)
}

However, the readHost is not the json api, it is the xml api:

readHost = "storage.googleapis.com"

The xml API expects an x-goog-if-generation-match header. It does not recognize the JSON API's url parameters.

Thus the following will always work and never return the expected preconditon error:

client := storage.NewClient()
r, err := client.Bucket("whatever").Object("some/object").If(&storage.Conditions{GenerationMatch: 666}).NewReader(context.Background())
if err == nil {
  panic("should have failed")
}
gerr, ok := err.(*googleapi.Error)
if !ok {
  panic("should have returned a googleapi error")
}
if gerr.Code != http.PreconditionFailed {
  panic("should have return a precondition failed error")
}
// congrats! This bug no longer repros.

Activity

  1. added
    type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.
    priority: p2Moderately-important priority. Fix may not be included in next release.
    and removed
    triage meI really want to be triaged.
    on Jul 21, 2021
  2. added a commit that references this issue on Jul 23, 2021
    e36b29a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

api: storageIssues related to the Cloud Storage API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions