This issue might be related to #2334, which seems has been closed. But I got similar issue when I am using minio server built from source code (downloaded 11/09/2016). Basically, I was uploading a 9GB file divided into 100 chunks. After all chunks have been uploaded successfully with multiple worker threads, I sent the multi-part upload completion request, which blocks for a long time and then timed out.
Expected Behavior
If issue #2334 was fixed, the completion request should return much faster because the concatenation was performed during chunks uploading phase. With aws s3, no such delay or timeout was observed for huge file upload completion. Theirs appears to be an async call.
Current Behavior
Long-time blocking call for the multi-part upload completion and eventually timed out for huge files.
Possible Solution
Make sure the concatenation was performed while parts where being uploaded or make the completion request become async. Concatenation during uploading might not be achievable if the client sends lots of chunks in parallel. Making the completion call async may be a better option.
Steps to Reproduce (for bugs)
1.Create a huge file, say 10GB
2.Get multi-part pre-signed urls (100 parts)
3.Upload all parts using 4 worker threads (sending 4 parts at a time in parallel)
4.Send completion request after all chunks have been uploaded
5. You will see the delay and timeout eventually
Context
I am trying to speed up the uploading of huge files using multiple worker threads for the chunks.
Your Environment
-
Version used:
minio server built from source code downloaded on 11/09/2016
-
Environment name and version (e.g. nginx 1.9.1):
No nginx used. Both client and minio server are in the same local network for testing.
-
Server type and version:
latest version 11/09/2016
-
Operating System and version:
Ubuntu 16.04
-
Link to your project: