Allow cancellation of scalar subqueries and other analysis-time pipelines#100230
Open
Algunenano wants to merge 8 commits intoClickHouse:masterfrom
Open
Allow cancellation of scalar subqueries and other analysis-time pipelines#100230Algunenano wants to merge 8 commits intoClickHouse:masterfrom
Algunenano wants to merge 8 commits intoClickHouse:masterfrom
Conversation
Scalar subqueries are executed during the query analysis phase, where the interactive cancel callback was not being checked. This made it impossible to cancel queries like `SELECT (SELECT max(number) FROM system.numbers) + 1` using Ctrl+C or KILL QUERY. Poll the `getInteractiveCancelCallback` between timed pulls (100ms timeout) during scalar subquery execution, in both the new analyzer and the old interpreter paths. Co-Authored-By: Claude Opus 4.6 <[email protected]>
Use the single `interactive_cancel_callback` for all subquery pipelines (scalar subqueries, prepared sets, GLOBAL IN/JOIN, PARALLEL WITH) instead of a separate `subquery_cancel_callback`. Remove the wrapping pattern that converted the callback return value into an exception. Fix `sendProgress` in `TCPHandler` to compute deltas via a `sent_progress` tracker instead of resetting `state.progress` with `fetchValuesAndResetPiecewiseAtomically`. This allows the cancel callback to safely send progress during subquery execution without losing accumulated statistics. Fix the client to buffer Progress packets received before `output_format` is created (e.g. from scalar subqueries during analysis) and replay them once the output format is available. Also cancel through the `ProcessListElement` in `processCancel` so all registered pipeline executors see the cancellation. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Contributor
|
Workflow [PR], commit [9fb0d3a] Summary: ❌
AI ReviewSummaryThis PR adds cancellation polling for analysis-time pipelines (scalar subqueries, set building, Missing context
ClickHouse Rules
Final Verdict
|
clickhouse-gh
bot
added
the
pr-bugfix
1 task
…pipelines The `cancelQuery` call on the `ProcessListElement` was too aggressive: it cancels ALL registered pipeline executors, not just the one that received the Cancel packet. This breaks parallel replica queries where multiple executors share the same `ProcessListElement`. Cancellation of scalar subqueries still works correctly: `processCancel` throws `QUERY_WAS_CANCELLED_BY_CLIENT`, which propagates through the cancel callback up through the pull loop. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Contributor
LLVM Coverage Report
PR changed lines: PR changed-lines coverage: 98.10% (155/158, 0 noise lines excluded) |
Member
Author
|
@novikd Can you please review this? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1576
Closes #98163
Co-authored with @YjyJeff, based on #98163.
During query analysis, several pipelines run synchronously before the main query pipeline starts: scalar subqueries, IN subquery set building, GLOBAL IN/JOIN materialization, and PARALLEL WITH queries. Previously, these could not be cancelled via Ctrl+C because nobody was reading Cancel packets from the TCP socket during analysis.
This PR enables cancellation by using the existing
interactive_cancel_callbackto periodically poll for Cancel packets during these analysis-time pipelines.For
clickhouse-local, a cancel callback is set that checks the signal handler flag and cancels through theProcessListElementwhen SIGINT is received.The client is also fixed to buffer Progress packets that arrive before the output format is created (from scalar subqueries during analysis), and replay them once the format is available. This ensures JSON statistics correctly include rows read by scalar subqueries, and the progress bar updates during scalar subquery execution. The same fix is applied to
LocalConnectionwhich now always tracks progress and sends a final Progress packet, fixingrows_read: 0inclickhouse-localJSON output.Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):
Allow cancellation of scalar subqueries and other analysis-time pipelines via Ctrl+C. Previously, pressing Ctrl+C during a long-running scalar subquery had no effect until the subquery completed. Also fix the progress bar and JSON statistics to correctly report rows read during scalar subquery execution, both in
clickhouse-clientandclickhouse-local. Co-authored with @YjyJeff.Documentation entry for user-facing changes
🤖 Generated with Claude Code