Skip to content

Fix race between async insert completion and query_log entry#96958

Merged
alexey-milovidov merged 3 commits intomasterfrom
fix-async-insert-query-log-race
Feb 16, 2026
Merged

Fix race between async insert completion and query_log entry#96958
alexey-milovidov merged 3 commits intomasterfrom
fix-async-insert-query-log-race

Conversation

@alexey-milovidov
Copy link
Copy Markdown
Member

Summary

  • Fix race condition in AsynchronousInsertQueue::processData where entry->finish() (which wakes up the client) was called before logQueryFinish (which writes the QueryFinish entry to query_log)
  • The client waiting with wait_for_async_insert=1 could wake up and execute SYSTEM FLUSH LOGS + SELECT FROM system.query_log before the QueryFinish entry was added to the log buffer
  • Fix by reordering: write log entries first, then notify clients — consistent with the error path where logQueryException already runs before finishWithException

Relation to #96957: that PR addresses a similar race in the HTTP/curl path at the test level (retry loops). It does not cover this case because test 02790 uses the native client (not curl), and the race window is in AsynchronousInsertQueue, not in the HTTP response path. The two fixes are complementary.

Test plan

  • Test 02790_async_queries_in_query_log passes consistently (5/5 local runs)
  • CI green

Closes #80813

Changelog category (leave one):

  • CI Fix or Improvement (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

...

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

🤖 Generated with Claude Code

In `AsynchronousInsertQueue::processData`, the `finish_entries` lambda
was calling `entry->finish()` (which sets the promise and wakes up the
client waiting with `wait_for_async_insert=1`) before `logQueryFinish`
(which writes the `QueryFinish` entry to `query_log`).

This created a race window: the client could wake up, execute
`SYSTEM FLUSH LOGS`, and query `system.query_log` before the
`QueryFinish` entry was even added to the log buffer. Under TSan or
CPU contention, this window was wide enough to trigger reliably.

Fix by reordering: write log entries first, then notify the waiting
clients. This is consistent with the error path, where
`logQueryException` already runs before `finishWithException`.

Note: this is complementary to #96957, which addresses a similar race
in the HTTP/curl path at the test level. That PR does not cover this
case because test `02790` uses the native client, not curl, and the
race window is in `AsynchronousInsertQueue`, not in the HTTP response
path.

Closes #80813

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Feb 15, 2026

Workflow [PR], commit [5897be1]

Summary:

@clickhouse-gh clickhouse-gh bot added the pr-ci label Feb 15, 2026
@alexey-milovidov alexey-milovidov self-assigned this Feb 16, 2026
@alexey-milovidov alexey-milovidov merged commit ba4bcd2 into master Feb 16, 2026
140 of 141 checks passed
@alexey-milovidov alexey-milovidov deleted the fix-async-insert-query-log-race branch February 16, 2026 06:21
@robot-clickhouse-ci-1 robot-clickhouse-ci-1 added the pr-synced-to-cloud The PR is synced to the cloud repo label Feb 16, 2026
@hanfei1991 hanfei1991 added the post-approved Approved, but after the PR is merged. label Mar 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

post-approved Approved, but after the PR is merged. pr-ci pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test 02790_async_queries_in_query_log is flaky

3 participants