Fixes for objects removal in S3ObjectStorage#37882
Fixes for objects removal in S3ObjectStorage#37882rschu1ze merged 5 commits intoClickHouse:masterfrom
S3ObjectStorage#37882Conversation
robot-ch-test-poll
added
the
pr-improvement
|
@alz said it is worth making this option to determine automatically. |
excitoon
force-pushed
the
nodeleteobjects
branch
from
8d5583c to
2df1107
Compare
|
Related to #37659 |
excitoon
force-pushed
the
nodeleteobjects
branch
2 times, most recently
from
8b010ea to
55fe159
Compare
excitoon
changed the title
has_delete_objects setting for S3 disk
excitoon
changed the title
S3ObjectStorage
excitoon
force-pushed
the
nodeleteobjects
branch
from
55fe159 to
a08f5c6
Compare
rschu1ze
added
the
can be tested
|
clang-tidy failure (BuilderBinTidy) is related, besides that this PR lgtm. |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
excitoon
force-pushed
the
nodeleteobjects
branch
from
8f16c4b to
0ec18c9
Compare
|
@excitoon AFAIR lists are not parsed correctly in changelog entries, and I doubt that "better removal" important for user, so maybe something like this will be better and enough "Automatic detection for |
This comment was marked as outdated.
This comment was marked as outdated.
excitoon
force-pushed
the
nodeleteobjects
branch
from
04c8bc0 to
92880d7
Compare
|
@rschu1ze I'm done with the tests |
|
Thanks. Tests look good, it just complains about a style check for the Python test ... would you like to fix that cosmetic issue (I have seen PRs being rolled back due to style issues 😕) ? |
|
@rschu1ze Sure I can. BTW, what is the point of this style check if it could be automatically applied on my changes? |
|
@rschu1ze Fixed! |
|
Thanks! Will merge as soon as the final round of tests is done. (My guess is that "automatically" fixing commits opens a can of worms ... starting from authorization problems (the style checker needs to have GitHub credentials) up to legal problems, e.g. does the originals authors CLA also cover an automatically "fixed commit". Really don't know what kind of edge cases are out there. But I agree that there should at least be a way of semi-automatically fixing style, e.g. via a Git hook integration ...) |
|
Remaining test failures are unrelated and addressed in a different PR --> merged |
…bjects Fixes for objects removal in `S3ObjectStorage`
Window functions and GCP fixes ClickHouse#36944, ClickHouse#34632, ClickHouse#37659 and ClickHouse#37882
…bjects Fixes for objects removal in `S3ObjectStorage`
Window functions and GCP fixes ClickHouse#36944, ClickHouse#34632, ClickHouse#37659 and ClickHouse#37882
…bjects Fixes for objects removal in `S3ObjectStorage`
Window functions and GCP fixes ClickHouse#36944, ClickHouse#34632, ClickHouse#37659 and ClickHouse#37882
…bjects Fixes for objects removal in `S3ObjectStorage`
…4632, ClickHouse#37659 and ClickHouse#37882 Merge pull request ClickHouse#36944 from excitoon-favorites/better_exp_smooth Merge pull request ClickHouse#34632 from excitoon-favorites/optimizedprocessing Merge pull request ClickHouse#37659 from frew/master Support `batch_delete` capability for GCS Merge pull request ClickHouse#37882 from excitoon-favorites/nodeleteobjects Fixes for objects removal in `S3ObjectStorage`
5 participants
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
S3 single objects are now removed with
RemoveObjectRequest(sic). Fixed a bug withS3ObjectStorageon GCP which did not allow to useremoveFileIfExistseffectively breaking approximately half ofremovefunctionality. Automatic detection forDeleteObjectsS3 API, that is not supported by GCS. This will allow to use GCS without explicitsupport_batch_delete=0in configuration.