Skip to content

Try to fix test_storage_s3: crash in WriteBufferFromS3#21624

Merged
alexey-milovidov merged 1 commit intoClickHouse:masterfrom
vdimir:storage-s3-fail
Mar 16, 2021
Merged

Try to fix test_storage_s3: crash in WriteBufferFromS3#21624
alexey-milovidov merged 1 commit intoClickHouse:masterfrom
vdimir:storage-s3-fail

Conversation

@vdimir
Copy link
Copy Markdown
Member

@vdimir vdimir commented Mar 11, 2021

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • Bug Fix

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
std::terminate was called if there is an error writing data into s3.

@robot-clickhouse robot-clickhouse added the pr-bugfix Pull request with bugfix, not backported by default label Mar 11, 2021
}
catch (...)
{
tryLogCurrentException(__PRETTY_FUNCTION__);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this will hide write error, does not looks right...

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge this PR first and another PR later.

@azat
Copy link
Copy Markdown
Member

azat commented Mar 12, 2021

BTW There is another PR (#21325) that is trying to address this

@alexey-milovidov alexey-milovidov self-assigned this Mar 16, 2021
@alexey-milovidov alexey-milovidov merged commit f52e29c into ClickHouse:master Mar 16, 2021
@alexey-milovidov
Copy link
Copy Markdown
Member

@azat FYI another idea is to make std::terminate in debug and logging in release.

@azat
Copy link
Copy Markdown
Member

azat commented Mar 25, 2021

@azat FYI another idea is to make std::terminate in debug and logging in release.

@alexey-milovidov I have similar thoughts, but decided not to, since write error may hide data loss.

@azat
Copy link
Copy Markdown
Member

azat commented Mar 25, 2021

@vdimir can you provide a stack trace for uncaught exception?

@vdimir
Copy link
Copy Markdown
Member Author

vdimir commented Mar 26, 2021

@azat
Copy link
Copy Markdown
Member

azat commented Mar 27, 2021

(from here https://clickhouse-test-reports.s3.yandex.net/21457/e69f7c7a94d7c0a5776cc408b99a028d3eeae06b/integration_tests_flaky_check_(asan).html#fail1 see file test_storage_s3/_instances/dummy/logs/clickhouse-server.err.log from test_dir.tar)

@vdimir thanks!

I will try to remove that try/catch to see if test 100% fails, and later push some fixes - #22208

azat added a commit to azat/ClickHouse that referenced this pull request Apr 25, 2021
That may lead to uncaught exception [1].

  [1]: ClickHouse#21624 (comment)

v0: New method BlockOutputStream::finalize() -- bad (forgot some places
    and superfluous)
v2: Flush BlockOutputStream in QueryStatus::releaseQueryStreams()
    Safer query termination (report some uncaugh errors to user)
v3: Flush BlockOutputStream from TCPHandler only
    Other places uses different executeQuery() implementation
v4: Do not call writeSuffix() twice
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-bugfix Pull request with bugfix, not backported by default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants