Skip to content

RabbitMQ - fix writing many small blocks by flushing data only exactly by flush_interval_ms or by max_block_size#44404

Merged
kssenii merged 18 commits intoClickHouse:masterfrom
kssenii:storage-rabbitmq-fix-flush
Feb 3, 2023
Merged

RabbitMQ - fix writing many small blocks by flushing data only exactly by flush_interval_ms or by max_block_size#44404
kssenii merged 18 commits intoClickHouse:masterfrom
kssenii:storage-rabbitmq-fix-flush

Conversation

@kssenii
Copy link
Copy Markdown
Member

@kssenii kssenii commented Dec 19, 2022

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in official stable or prestable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Flush data exactly by rabbitmq_flush_interval_ms or by rabbitmq_max_block_size in StorageRabbitMQ. Closes #42389. Closes #45160.

@robot-ch-test-poll3 robot-ch-test-poll3 added the pr-bugfix Pull request with bugfix, not backported by default label Dec 19, 2022
@Avogar Avogar self-assigned this Dec 19, 2022
@kssenii kssenii force-pushed the storage-rabbitmq-fix-flush branch from 5d8bc6f to 499565a Compare December 20, 2022 19:02
@Avogar
Copy link
Copy Markdown
Member

Avogar commented Jan 2, 2023

I merged my big PR with changes aroung RabbitMQ and it produced complex conflicts, if you will need help to resolve them, you can ask me.

@Recodify
Copy link
Copy Markdown

Any news on this getting merged? Big issue for us :(

@kssenii
Copy link
Copy Markdown
Member Author

kssenii commented Jan 12, 2023

@Recodify, I need to fix some remaining failing tests... Will try to finish this week or at the beginning of the next.

@kssenii kssenii mentioned this pull request Jan 31, 2023
@kssenii
Copy link
Copy Markdown
Member Author

kssenii commented Feb 3, 2023

🥲
Screenshot 2023-02-03 at 11 48 35

@kssenii kssenii merged commit ef7acb9 into ClickHouse:master Feb 3, 2023
kssenii added a commit that referenced this pull request Apr 7, 2023
Fix cpu usage in rabbitmq (was worsened in 23.2 after #44404)
kssenii added a commit that referenced this pull request Apr 11, 2023
Backport #48311 to 23.3: Fix cpu usage in rabbitmq (was worsened in 23.2 after #44404)
kssenii added a commit that referenced this pull request Apr 11, 2023
Backport #48311 to 23.2: Fix cpu usage in rabbitmq (was worsened in 23.2 after #44404)
Enmk pushed a commit to Altinity/ClickHouse that referenced this pull request Jun 24, 2023
…-flush

RabbitMQ - fix writing many small blocks by flushing data only exactly by flush_interval_ms or by max_block_size
Enmk pushed a commit to Altinity/ClickHouse that referenced this pull request Jun 26, 2023
Fix cpu usage in rabbitmq (was worsened in 23.2 after ClickHouse#44404)

cherry-picked d5889a4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-bugfix Pull request with bugfix, not backported by default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test_storage_rabbitmq is broken Many small blocks wrote for RabbitMQ Engine table

4 participants