You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Feb 13, 2026. It is now read-only.
The FS version concatenates all the parts of a multi-part upload when the multi-upload is completed. Multi-part uploads are often used for huge files, so this takes a lot of time and also spikes the IOPS for a while. In some situations (like ours), it would be much better to use the XL method to keep the parts separate and combine them during download.
Of course, the parts cannot be accessed the regular way (via the filesystem) anymore. But most systems that use S3 only use S3 to access them (you can't do otherwise in AWS), so this wouldn't be a big problem.
Concatenating files might be the sensible default option, but when performance and I/O is critical, then keeping the files separate might be the better option. Now, we reverted using the XL method, solely for this feature, but this costs us a lot of additional disk space.
The FS version concatenates all the parts of a multi-part upload when the multi-upload is completed. Multi-part uploads are often used for huge files, so this takes a lot of time and also spikes the IOPS for a while. In some situations (like ours), it would be much better to use the XL method to keep the parts separate and combine them during download.
Of course, the parts cannot be accessed the regular way (via the filesystem) anymore. But most systems that use S3 only use S3 to access them (you can't do otherwise in AWS), so this wouldn't be a big problem.
Concatenating files might be the sensible default option, but when performance and I/O is critical, then keeping the files separate might be the better option. Now, we reverted using the XL method, solely for this feature, but this costs us a lot of additional disk space.