Skip to content
This repository was archived by the owner on Feb 13, 2026. It is now read-only.
This repository was archived by the owner on Feb 13, 2026. It is now read-only.

Multi-part upload completion very slow and eventually timed out for >=9GB files #3223

@henryzzhang

Description

@henryzzhang

This issue might be related to #2334, which seems has been closed. But I got similar issue when I am using minio server built from source code (downloaded 11/09/2016). Basically, I was uploading a 9GB file divided into 100 chunks. After all chunks have been uploaded successfully with multiple worker threads, I sent the multi-part upload completion request, which blocks for a long time and then timed out.

Expected Behavior

If issue #2334 was fixed, the completion request should return much faster because the concatenation was performed during chunks uploading phase. With aws s3, no such delay or timeout was observed for huge file upload completion. Theirs appears to be an async call.

Current Behavior

Long-time blocking call for the multi-part upload completion and eventually timed out for huge files.

Possible Solution

Make sure the concatenation was performed while parts where being uploaded or make the completion request become async. Concatenation during uploading might not be achievable if the client sends lots of chunks in parallel. Making the completion call async may be a better option.

Steps to Reproduce (for bugs)

1.Create a huge file, say 10GB
2.Get multi-part pre-signed urls (100 parts)
3.Upload all parts using 4 worker threads (sending 4 parts at a time in parallel)
4.Send completion request after all chunks have been uploaded
5. You will see the delay and timeout eventually

Context

I am trying to speed up the uploading of huge files using multiple worker threads for the chunks.

Your Environment

  • Version used:
    minio server built from source code downloaded on 11/09/2016

  • Environment name and version (e.g. nginx 1.9.1):
    No nginx used. Both client and minio server are in the same local network for testing.

  • Server type and version:
    latest version 11/09/2016

  • Operating System and version:
    Ubuntu 16.04

  • Link to your project:

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions