Skip to content

Allow oneline comment in insert query#1693

Merged
kavirajk merged 3 commits intoClickHouse:mainfrom
luckyevildev:allow-oneline-comment-in-insert-query
Nov 11, 2025
Merged

Allow oneline comment in insert query#1693
kavirajk merged 3 commits intoClickHouse:mainfrom
luckyevildev:allow-oneline-comment-in-insert-query

Conversation

@luckyevildev
Copy link
Copy Markdown
Contributor

@luckyevildev luckyevildev commented Nov 1, 2025 •
edited
Loading

Summary

This PR adds support for SQL single-line comments before INSERT statements in batch operations. This enhancement improves compatibility with SQL tools and query builders that automatically add comments to generated queries.

Checklist

Delete items not relevant to your PR:

  • [*] Unit and integration tests covering the common scenarios were added
  • [*] A human-readable description of the changes was provided to include in CHANGELOG

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Nov 1, 2025 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@kavirajk
Copy link
Copy Markdown
Contributor

kavirajk commented Nov 6, 2025

@luckyevildev thanks for the PR.

I'm not sure if I understand the purpose of this change. The single line comment is already seems to be working with INSERTS.

In fact all your tests (excluding the last one) already passes in the main branch.

$ git rev-parse HEAD && git rev-parse --abbrev-ref HEAD
3ad16cc4646aea6a001f3ac12fffd2a24658c57f
main
$ go test . -run TestExtractNormalizedInsertQueryAndColumns/-- -v
=== RUN   TestExtractNormalizedInsertQueryAndColumns
=== RUN   TestExtractNormalizedInsertQueryAndColumns/--_comment_INSERT_INTO_table_name_(col1,_col2)_VALUES_(1,_2)
=== RUN   TestExtractNormalizedInsertQueryAndColumns/--_comment_1_--_comment_2_INSERT_INTO_table_name_(col1,_col2)_VALUES_(1,_2)
=== RUN   TestExtractNormalizedInsertQueryAndColumns/--_This_is_a_batch_insert_INSERT_INTO_table_name_(__col1,__col2_)_VALUES_(1,_2)
=== RUN   TestExtractNormalizedInsertQueryAndColumns/--_Comment_about_the_table_--_Another_comment_INSERT_INTO_"db"."table_name"_(__col1,__col2_)_VALUES_(1,_2)
=== RUN   TestExtractNormalizedInsertQueryAndColumns/--_комментарий_на_русском_языке_INSERT_INTO_table_name_(col1,_col2)
=== RUN   TestExtractNormalizedInsertQueryAndColumns/--_batch_insert_INSERT_INTO_table_name_FORMAT_JSONEachRow
=== RUN   TestExtractNormalizedInsertQueryAndColumns/--_comment______INSERT_INTO_table_name_(col1,_col2)
--- PASS: TestExtractNormalizedInsertQueryAndColumns (0.00s)
    --- PASS: TestExtractNormalizedInsertQueryAndColumns/--_comment_INSERT_INTO_table_name_(col1,_col2)_VALUES_(1,_2) (0.00s)
    --- PASS: TestExtractNormalizedInsertQueryAndColumns/--_comment_1_--_comment_2_INSERT_INTO_table_name_(col1,_col2)_VALUES_(1,_2) (0.00s)
    --- PASS: TestExtractNormalizedInsertQueryAndColumns/--_This_is_a_batch_insert_INSERT_INTO_table_name_(__col1,__col2_)_VALUES_(1,_2) (0.00s)
    --- PASS: TestExtractNormalizedInsertQueryAndColumns/--_Comment_about_the_table_--_Another_comment_INSERT_INTO_"db"."table_name"_(__col1,__col2_)_VALUES_(1,_2) (0.00s)
    --- PASS: TestExtractNormalizedInsertQueryAndColumns/--_комментарий_на_русском_языке_INSERT_INTO_table_name_(col1,_col2) (0.00s)
    --- PASS: TestExtractNormalizedInsertQueryAndColumns/--_batch_insert_INSERT_INTO_table_name_FORMAT_JSONEachRow (0.00s)
    --- PASS: TestExtractNormalizedInsertQueryAndColumns/--_comment______INSERT_INTO_table_name_(col1,_col2) (0.00s)
PASS
ok  	github.com/ClickHouse/clickhouse-go/v2	(cached)

chernser
chernser reviewed Nov 7, 2025
View reviewed changes
batch.go Outdated
)

var normalizeInsertQueryMatch = regexp.MustCompile(`(?i)(INSERT\s+INTO\s+([^(]+)(?:\s*\([^()]*(?:\([^()]*\)[^()]*)*\))?)(?:\s*VALUES)?`)
var normalizeInsertQueryMatch = regexp.MustCompile(`(?i)(?:--[^\n]*\n\s*)*(INSERT\s+INTO\s+([^(]+)(?:\s*\([^()]*(?:\([^()]*\)[^()]*)*\))?)(?:\s*VALUES)?`)
Copy link
Copy Markdown

@chernser chernser Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ClickHouse supports also # and #! according to documentation - https://clickhouse.com/docs/sql-reference/syntax#comments
I have seen only once but have fixed in Java parser.

Copy link
Copy Markdown
Contributor Author

@luckyevildev luckyevildev Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, added support for #, #!

mshustov
mshustov reviewed Nov 9, 2025
View reviewed changes
batch_test.go
{
name: "Insert with comment and backtick quoted names",
query: `-- Insert into special table
INSERT INTO ` + "`_test_1345# $.ДБ`.`2. Таблица №2`" + ` (col1, col2)`,
Copy link
Copy Markdown
Member

@mshustov mshustov Nov 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you add tests for the comment in the middle and tail? see https://github.com/ClickHouse/clickhouse-rs/pull/267/files

Copy link
Copy Markdown
Contributor Author

@luckyevildev luckyevildev Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, added

@luckyevildev
Copy link
Copy Markdown
Contributor Author

luckyevildev commented Nov 10, 2025

@kavirajk
This functionality is needed to add metadata and identifiers to batch queries. Comments before INSERT INTO enable:

  • Specifying batch IDs for tracking in logs
  • Adding metadata (data source, version, timestamp)
  • Improving debugging and monitoring
  • Integrating with tools that automatically add comments

Without comment support, such queries were not parsed correctly. Supporting all three comment styles (--, #!, #) ensures compatibility with different SQL generators and tools.

kavirajk
kavirajk approved these changes Nov 11, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@kavirajk kavirajk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @luckyevildev

@kavirajk kavirajk merged commit a6d5a5f into ClickHouse:main Nov 11, 2025
14 checks passed
@luckyevildev luckyevildev deleted the allow-oneline-comment-in-insert-query branch November 11, 2025 11:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chernser chernser chernser left review comments

@mshustov mshustov mshustov left review comments

@kavirajk kavirajk kavirajk approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@luckyevildev @CLAassistant @kavirajk @chernser @mshustov