Skip to content

Commit 2defcfe

Browse files
CAOShurongFernando Jesús Fernández Gallardo
authored andcommitted
s3: treat UploadPart success without ETag as retryable error
A successful UploadPart whose response carries no ETag header made WriteChunk panic dereferencing uout.ETag in a debug log line. The part ETag is required by CompleteMultipartUpload, so an ETag-less 200 is unusable: return a retryable error from inside the pacer callback so the chunk is retried instead of crashing the transfer or completing the upload with a broken part list. Fixes rclone#9822 Co-authored-by: Shurong Cao <[email protected]>
1 parent 75731b8 commit 2defcfe

1 file changed

Lines changed: 10 additions & 0 deletions

File tree

‎backend/s3/s3.go‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4653,6 +4653,16 @@ func (w *s3ChunkWriter) WriteChunk(ctx context.Context, chunkNumber int, reader
46534653
// retry all chunks once have done the first few
46544654
return true, err
46554655
}
4656+
if uout == nil || uout.ETag == nil {
4657+
// A successful UploadPart without an ETag header is unusable: the
4658+
// part ETag is required by CompleteMultipartUpload. Proxies and
4659+
// load balancers have been observed emitting empty 200 responses
4660+
// under load - see #9822. Treat it as a retryable error so the
4661+
// pacer retries this chunk, instead of dereferencing a nil ETag
4662+
// in the debug log below or completing the upload with a broken
4663+
// part list.
4664+
return true, fmt.Errorf("UploadPart response for chunk %d has no ETag", chunkNumber+1)
4665+
}
46564666
return false, nil
46574667
})
46584668
if err != nil {

0 commit comments

Comments
 (0)