Skip to content

Catch exception in ~WriteBufferFromS3#34963

Merged
vdimir merged 3 commits intoClickHouse:masterfrom
vdimir:writebufferfroms3-try-finalize
Mar 4, 2022
Merged

Catch exception in ~WriteBufferFromS3#34963
vdimir merged 3 commits intoClickHouse:masterfrom
vdimir:writebufferfroms3-try-finalize

Conversation

@vdimir
Copy link
Copy Markdown
Member

@vdimir vdimir commented Mar 1, 2022

@robot-clickhouse robot-clickhouse added the pr-not-for-changelog This PR should not be mentioned in the changelog label Mar 1, 2022
@alesapin alesapin self-assigned this Mar 1, 2022
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it is better to put finalize call into caller to avoid suppressing errors?

@azat
Copy link
Copy Markdown
Member

azat commented Mar 1, 2022

@azat
Copy link
Copy Markdown
Member

azat commented Mar 1, 2022

Oops, I've attached wrong report

At least I've finally figured out why the assertion in jemalloc was hitting (hopefully).

https://s3.amazonaws.com/clickhouse-test-reports/34951/96390a9263cb5af3d6e42a84988239c9ae87ce32/stress_test__undefined__actions_/clickhouse-server.err.log

So maybe it worth to do proper destroy in StorageS3Sink, instead of adding try/catch in dtor.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't fully understand why this is needed ...

@vdimir vdimir force-pushed the writebufferfroms3-try-finalize branch from 7783bb2 to 6c29a57 Compare March 3, 2022 12:57
@vdimir
Copy link
Copy Markdown
Member Author

vdimir commented Mar 3, 2022

@azat

Still not sure what's the proper way.

We can call cancel or finish for all processors for QueryPipeline::reset . Also maybe in ~QueryPipeline

void QueryPipeline::reset()
{
QueryPipeline to_remove = std::move(*this);
*this = QueryPipeline();
}

But it would throw anyway. Also, should any retries be performed after errors from ~WriteBufferFromS3?

@vdimir
Copy link
Copy Markdown
Member Author

vdimir commented Mar 4, 2022

Let's merge current solution with try ... catch ... in ~WriteBufferFromS3, proper fix can be added later

@vdimir vdimir merged commit 0276140 into ClickHouse:master Mar 4, 2022
@vdimir vdimir deleted the writebufferfroms3-try-finalize branch March 4, 2022 10:52
@azat
Copy link
Copy Markdown
Member

azat commented Mar 4, 2022

But it would throw anyway. Also, should any retries be performed after errors from ~WriteBufferFromS3?

Yes, but it should call onFinish that calls WriteBufferFromS3::finalize and it will not throw from dtor since finalize will set to true, so finalize from dtor will be no-op.

@MaxWk
Copy link
Copy Markdown
Contributor

MaxWk commented Jun 8, 2022

what happened if finalize throw excaption? Looks like there is a possibility of data loss.

@azat
Copy link
Copy Markdown
Member

azat commented Jun 8, 2022

what happened if finalize throw excaption? Looks like there is a possibility of data loss.

Indeed, and to me it is better to abort over data loss.
That's why initially (#19451) there was no try/catch in dtors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-not-for-changelog This PR should not be mentioned in the changelog

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants