Can you describe more clearly what the issue was and what the fix addresses? Thanks.

@Xiaobenbenben any thoughts on what @rxin asked? What is the issue that you are seeing that this fix addresses?

@rxin, @tdas, the obvious problem with original code is that it doesn't take into account huge S3 prefixes (more than 1000 objects). In this case original code will delete only first 1000 objects.

AWS S3 client has isTruncated method which indicates if there are more objects to list. The proposed fix repeats deletion in a loop until there are no unlisted objects.

AWS S3 client also exposes method to delete multiple objects public DeleteObjectsResult deleteObjects(DeleteObjectsRequest deleteObjectsRequest). It will improve performance and reduce S3 throttling probability.

@Xiaobenbenben thank you for the explanation. you are right. have you tested this with actual S3? Our automated tests don't have any actual S3 tests... so we have to manually test this.

@ravivj-db @vikrantpuppala can one of you test this out?

Looking into this.

@tdas Have tested out this code, and it's working fine. You can go ahead and merge it. @Xiaobenbenben thanks for handling pagination and ensuring that all objects inside the s3 folder are deleted.

Thank you @Xiaobenbenben for contributing this.

Thank you @Xiaobenbenben for contributing this.

I am very sorry to miss your discussion here, very glad to contribute.