Is your feature request related to a problem? Please describe.
Related issue: #1703
Currently qdrant can only store snapshots in local filesystem (even if it is a mounted s3 bucket), but it might be convenient to upload and download snapshots from S3 directly.
Describe the solution you'd like
Ability to configure snapshot directory as S3 bucket, specifying access credentials.
It should be possible to list snapshots saved on attached S3 bucket as well.
Describe alternatives you've considered
Our current recommendations are to mount s3 as local fs, but it might be non-trivial to do in some environments.
Additional context
Important qualities of the solution:
- Solution should not change default behavior and should be backward compatible
- Additional dependency to work with S3 should be have reasonable size, do not contribute significantly into qdrant compile time & binary size
- Should work with all types of S3-compatible storage, not only aws, but also google storage, minio, e.t.c.
Note for contributors: Please consider this as tracking issue. If you think that it would be beneficial to split the task into multiple smaller PRs, please you are welcome to do so. Bounty will be rewarded for each PR independently
Is your feature request related to a problem? Please describe.
Related issue: #1703
Currently qdrant can only store snapshots in local filesystem (even if it is a mounted s3 bucket), but it might be convenient to upload and download snapshots from S3 directly.
Describe the solution you'd like
Ability to configure snapshot directory as S3 bucket, specifying access credentials.
It should be possible to list snapshots saved on attached S3 bucket as well.
Describe alternatives you've considered
Our current recommendations are to mount s3 as local fs, but it might be non-trivial to do in some environments.
Additional context
Important qualities of the solution:
Note for contributors: Please consider this as tracking issue. If you think that it would be beneficial to split the task into multiple smaller PRs, please you are welcome to do so. Bounty will be rewarded for each PR independently