Skip to content

Store exception generated when we tried to update the queue last time#26843

Merged
alesapin merged 5 commits intoClickHouse:masterfrom
nvartolomei:nv/last-queue-update-exception
Aug 20, 2021
Merged

Store exception generated when we tried to update the queue last time#26843
alesapin merged 5 commits intoClickHouse:masterfrom
nvartolomei:nv/last-queue-update-exception

Conversation

@nvartolomei
Copy link
Copy Markdown
Contributor

The use case is to alert when queue contains broken entries. Especially
important when ClickHouse breaks backwards compatibility between
versions and log entries written by newer versions aren't parseable by
old versions.

Code: 27, e.displayText() = DB::Exception: Cannot parse input: expected 'quorum: ' before: 'merge_type: 2\n'

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Add new column last_queue_update_exception to system.replicas table.

The use case is to alert when queue contains broken entries. Especially
important when ClickHouse breaks backwards compatibility between
versions and log entries written by newer versions aren't parseable by
old versions.

```
Code: 27, e.displayText() = DB::Exception: Cannot parse input: expected 'quorum: ' before: 'merge_type: 2\n'
```
@robot-clickhouse robot-clickhouse added doc-alert pr-feature Pull request with new product feature labels Jul 27, 2021
catch (const Coordination::Exception & e)
catch (...)
{
last_queue_update_exception.set(std::make_unique<String>(getCurrentExceptionMessage(false)));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just leave old logic and copy this line to the catch (...).

{
restarting_thread.wakeup();
return;
throw;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

??? Remove.

ReplicatedMergeTreeQueue queue;
std::atomic<time_t> last_queue_update_start_time{0};
std::atomic<time_t> last_queue_update_finish_time{0};
MultiVersion<String> last_queue_update_exception;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MultiVersion is not intended to be something like std::atomic<AnyClass>. Why not just string with mutex (or even reuse some mutex...) + two methods?

@alesapin alesapin self-assigned this Jul 28, 2021
Copy link
Copy Markdown
Member

@alesapin alesapin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let's rerun CI

@alesapin
Copy link
Copy Markdown
Member

@Mergifyio update

@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Aug 18, 2021

Command update: success

Branch has been successfully updated

@alesapin
Copy link
Copy Markdown
Member

@Mergifyio update

@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Aug 18, 2021

Command update: success

Branch has been successfully updated

@alesapin
Copy link
Copy Markdown
Member

alesapin commented Aug 20, 2021

Performance failures are not related to changes.

@alesapin alesapin merged commit 3bf412c into ClickHouse:master Aug 20, 2021
@nvartolomei nvartolomei deleted the nv/last-queue-update-exception branch August 20, 2021 12:38
@sevirov
Copy link
Copy Markdown
Contributor

sevirov commented Aug 20, 2021

Internal documentation ticket: DOCSUP-13565

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-feature Pull request with new product feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants