Skip to content

Commit 8359bb5

Browse files
morningmanclaude
andcommitted
[fix](stream) Decode the STREAM comment string literal per sql mode
### What problem does this PR solve? Issue Number: close #65388, related #65418 Related PR: #65810 Problem Summary: Review follow-up for this PR. Two problems were found in the first revision. 1. `visitAlterStreamComment` decoded the comment with `stripQuotes`, which only removes the outer delimiters. `ALTER STREAM s SET COMMENT 'a''b'` therefore stored `a''b` instead of `a'b`, and with the default sql mode `COMMENT '\n'` stored a backslash plus `n` instead of a newline. The value is exposed by `information_schema.table_streams.STREAM_COMMENT` and printed by `SHOW CREATE STREAM`, so the stored metadata was simply wrong. `visitCreateStream` had the other half of the problem: it decoded backslash escapes (`LogicalPlanBuilderAssistant.escapeBackSlash`) but did not collapse doubled quotes and ignored `NO_BACKSLASH_ESCAPES`, so the same literal produced different comments depending on whether it went through CREATE or ALTER. Both now use `SqlLiteralUtils.parseStringLiteral`, which collapses doubled quotes and honours the sql mode exactly like the lexer does (`NereidsParser` feeds `SqlModeHelper.hasNoBackSlashEscapes()` to the lexer). Not addressed here: `Env.addTableComment` wraps the comment in single quotes but only escapes double quotes (`SqlUtils.escapeQuota`), so a comment holding a single quote makes `SHOW CREATE` emit non-parsable DDL. That is pre-existing, shared by all 19 call sites of every table type, and is not stream specific. 2. The two new regression suites did not follow the repository test standards: names were indirected through `def`, deterministic results were asserted by hand instead of a generated `.out`, and both dropped their fixtures at the end. Fixed all three. While converting, the skip guard of `test_binlog_property_alter_exception` turned out to be dead: it compared the config name against `enable_feature_binlog`, but `SHOW FRONTEND CONFIG` reports the EXPERIMENTAL config as `experimental_enable_feature_binlog`, so the suite skipped itself unconditionally, in CI as well. It now uses the framework helper `getSyncer().checkEnableFeatureBinlog()`, and the suite actually runs. ### Release note None (the behaviour change of this PR is described in the first commit). ### Check List (For Author) - Test: - Unit Test: `AlterTableStreamCommentTest#testAlterStreamCommentStringLiteral` covers doubled single quotes, doubled double quotes, `\n`/`\t` and `NO_BACKSLASH_ESCAPES`. Both tests of the class pass locally (`Tests run: 2, Failures: 0, Errors: 0`). - Regression test: `test_table_stream_alter_comment` (the `.out` was generated with `-genOut` against a local cluster, then verified by a second run without it) and `test_binlog_property_alter_exception`, both pass locally and the six expected error messages were observed in the run log. - Behavior changed: Yes. A stream comment containing a doubled quote or a backslash escape is now stored decoded, and the sql mode is honoured. - Does this need documentation: No (the statement itself is documented by the first commit's doc PR). Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01Xx7TgjXJCiChnzLYa6hgtL
1 parent eabbe52 commit 8359bb5

5 files changed

Lines changed: 120 additions & 59 deletions

File tree

‎fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4012,8 +4012,8 @@ public LogicalPlan visitCreateStream(CreateStreamContext ctx) {
40124012
if (ctx.REPLACE() != null && ctx.EXISTS() != null) {
40134013
throw new AnalysisException("[OR REPLACE] and [IF NOT EXISTS] cannot used at the same time");
40144014
}
4015-
String comment = ctx.STRING_LITERAL() == null ? "" : LogicalPlanBuilderAssistant.escapeBackSlash(
4016-
ctx.STRING_LITERAL().getText().substring(1, ctx.STRING_LITERAL().getText().length() - 1));
4015+
String comment = ctx.STRING_LITERAL() == null ? ""
4016+
: SqlLiteralUtils.parseStringLiteral(ctx.STRING_LITERAL().getText());
40174017
Map<String, String> properties = ctx.properties != null
40184018
// NOTICE: we should not generate immutable map here, because it will be modified when analyzing.
40194019
? Maps.newHashMap(visitPropertyClause(ctx.properties))
@@ -7272,7 +7272,7 @@ public LogicalPlan visitAlterCatalogComment(AlterCatalogCommentContext ctx) {
72727272
@Override
72737273
public LogicalPlan visitAlterStreamComment(DorisParser.AlterStreamCommentContext ctx) {
72747274
TableNameInfo streamName = new TableNameInfo(visitMultipartIdentifier(ctx.name));
7275-
String comment = stripQuotes(ctx.comment.getText());
7275+
String comment = SqlLiteralUtils.parseStringLiteral(ctx.comment.getText());
72767276
return AlterStreamCommand.forSetComment(streamName, comment);
72777277
}
72787278

‎fe/fe-core/src/test/java/org/apache/doris/catalog/AlterTableStreamCommentTest.java‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
import org.apache.doris.common.Config;
2222
import org.apache.doris.common.ExceptionChecker;
2323
import org.apache.doris.common.FeConstants;
24+
import org.apache.doris.qe.SqlModeHelper;
2425
import org.apache.doris.utframe.TestWithFeService;
2526

2627
import org.junit.jupiter.api.Assertions;
@@ -76,4 +77,44 @@ public void testAlterStreamComment() throws Exception {
7677

7778
dropDatabase("test_alter_stream_comment");
7879
}
80+
81+
@Test
82+
public void testAlterStreamCommentStringLiteral() throws Exception {
83+
createDatabase("test_alter_stream_comment_literal");
84+
createTable("create table test_alter_stream_comment_literal.base_tbl (k1 int, v1 int)\n"
85+
+ "duplicate key(k1)\n"
86+
+ "distributed by hash(k1) buckets 1\n"
87+
+ "properties('replication_num' = '1', 'binlog.enable' = 'true', 'binlog.format' = 'ROW');");
88+
// CREATE STREAM and ALTER STREAM must decode the string literal in the same way:
89+
// a doubled quote is one quote and, unless NO_BACKSLASH_ESCAPES is set, backslash escapes are decoded
90+
createTable("create stream test_alter_stream_comment_literal.s1"
91+
+ " on table test_alter_stream_comment_literal.base_tbl\n"
92+
+ "comment 'a''b\\nc'\n"
93+
+ "properties('type' = 'append_only');");
94+
95+
Database db = Env.getCurrentInternalCatalog().getDbOrDdlException("test_alter_stream_comment_literal");
96+
BaseTableStream stream = (BaseTableStream) db.getTableOrDdlException("s1");
97+
Assertions.assertEquals("a'b\nc", stream.getComment());
98+
99+
executeSql("alter stream test_alter_stream_comment_literal.s1 set comment 'x''y\\tz'");
100+
Assertions.assertEquals("x'y\tz", stream.getComment());
101+
102+
// a double quoted literal doubles the double quote instead
103+
executeSql("alter stream test_alter_stream_comment_literal.s1 set comment \"p\"\"q\"");
104+
Assertions.assertEquals("p\"q", stream.getComment());
105+
106+
// under NO_BACKSLASH_ESCAPES a backslash is an ordinary character. Both the lexer and
107+
// SqlLiteralUtils read the sql mode from the thread local ConnectContext, so bind it first.
108+
connectContext.setThreadLocalInfo();
109+
long originalSqlMode = connectContext.getSessionVariable().getSqlMode();
110+
try {
111+
connectContext.getSessionVariable().setSqlMode(SqlModeHelper.MODE_NO_BACKSLASH_ESCAPES);
112+
executeSql("alter stream test_alter_stream_comment_literal.s1 set comment 'a\\nb'");
113+
Assertions.assertEquals("a\\nb", stream.getComment());
114+
} finally {
115+
connectContext.getSessionVariable().setSqlMode(originalSqlMode);
116+
}
117+
118+
dropDatabase("test_alter_stream_comment_literal");
119+
}
79120
}
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
-- This file is automatically generated. You should know what you did if you want to edit this
2+
-- !comment_after_create --
3+
initial comment
4+
5+
-- !comment_after_set --
6+
updated comment
7+
8+
-- !comment_after_modify --
9+
modified comment
10+
11+
-- !comment_doubled_quote --
12+
a'b 3
13+
14+
-- !comment_after_clear --
15+
16+
17+
-- !comment_final --
18+
final comment
19+

‎regression-test/suites/table_stream_p0/test_binlog_property_alter_exception.groovy‎

Lines changed: 15 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -24,25 +24,18 @@ suite("test_binlog_property_alter_exception") {
2424
return
2525
}
2626

27-
def featureBinlogEnabled = false
28-
for (row in sql("SHOW FRONTEND CONFIG LIKE '%enable_feature_binlog%'")) {
29-
if (row[0] == 'enable_feature_binlog' && row[1] == 'true') {
30-
featureBinlogEnabled = true
31-
}
32-
}
33-
if (!featureBinlogEnabled) {
34-
logger.info("enable_feature_binlog=false in frontend config, skip test_binlog_property_alter_exception")
27+
// enable_feature_binlog is an EXPERIMENTAL config, so SHOW FRONTEND CONFIG reports it as
28+
// experimental_enable_feature_binlog. checkEnableFeatureBinlog() accounts for that prefix.
29+
if (!getSyncer().checkEnableFeatureBinlog()) {
30+
logger.info("fe enable_feature_binlog is false, skip case test_binlog_property_alter_exception")
3531
return
3632
}
3733

38-
def rowTable = "test_binlog_property_alter_row_tbl"
39-
def plainTable = "test_binlog_property_alter_plain_tbl"
40-
41-
sql "DROP TABLE IF EXISTS ${rowTable} FORCE"
42-
sql "DROP TABLE IF EXISTS ${plainTable} FORCE"
34+
sql "DROP TABLE IF EXISTS test_binlog_property_alter_row_tbl FORCE"
35+
sql "DROP TABLE IF EXISTS test_binlog_property_alter_plain_tbl FORCE"
4336

4437
sql """
45-
CREATE TABLE ${rowTable} (
38+
CREATE TABLE test_binlog_property_alter_row_tbl (
4639
k1 INT NOT NULL,
4740
v1 INT
4841
)
@@ -59,34 +52,34 @@ suite("test_binlog_property_alter_exception") {
5952

6053
// binlog.format can not be changed once the table is created
6154
test {
62-
sql """ALTER TABLE ${rowTable} SET ("binlog.format" = "STATEMENT_AND_SNAPSHOT")"""
55+
sql """ALTER TABLE test_binlog_property_alter_row_tbl SET ("binlog.format" = "STATEMENT_AND_SNAPSHOT")"""
6356
exception "not support change binlog format from ROW to STATEMENT_AND_SNAPSHOT"
6457
}
6558

6659
// binlog.need_historical_value can not be changed either
6760
test {
68-
sql """ALTER TABLE ${rowTable} SET ("binlog.need_historical_value" = "false")"""
61+
sql """ALTER TABLE test_binlog_property_alter_row_tbl SET ("binlog.need_historical_value" = "false")"""
6962
exception "not support change binlog.need_historical_value from true to false"
7063
}
7164

7265
// binlog can not be disabled while the format is ROW
7366
test {
74-
sql """ALTER TABLE ${rowTable} SET ("binlog.enable" = "false")"""
67+
sql """ALTER TABLE test_binlog_property_alter_row_tbl SET ("binlog.enable" = "false")"""
7568
exception "can't disable binlog when format is [Row]"
7669
}
7770

7871
// setting the same value is a no-op and must not be rejected
79-
sql """ALTER TABLE ${rowTable} SET ("binlog.format" = "ROW")"""
72+
sql """ALTER TABLE test_binlog_property_alter_row_tbl SET ("binlog.format" = "ROW")"""
8073

8174
// mutable binlog properties are still accepted on a ROW binlog table
82-
sql """ALTER TABLE ${rowTable} SET ("binlog.ttl_seconds" = "7200")"""
83-
def rowTableDdl = sql("SHOW CREATE TABLE ${rowTable}")[0][1].toString()
75+
sql """ALTER TABLE test_binlog_property_alter_row_tbl SET ("binlog.ttl_seconds" = "7200")"""
76+
def rowTableDdl = sql("SHOW CREATE TABLE test_binlog_property_alter_row_tbl")[0][1].toString()
8477
assertTrue(rowTableDdl.contains('"binlog.format" = "ROW"'), rowTableDdl)
8578
assertTrue(rowTableDdl.contains('"binlog.ttl_seconds" = "7200"'), rowTableDdl)
8679

8780
// the same check applies to a table without binlog: turning on ROW format afterwards is rejected
8881
sql """
89-
CREATE TABLE ${plainTable} (
82+
CREATE TABLE test_binlog_property_alter_plain_tbl (
9083
k1 INT NOT NULL,
9184
v1 INT
9285
)
@@ -97,10 +90,7 @@ suite("test_binlog_property_alter_exception") {
9790
)
9891
"""
9992
test {
100-
sql """ALTER TABLE ${plainTable} SET ("binlog.format" = "ROW")"""
93+
sql """ALTER TABLE test_binlog_property_alter_plain_tbl SET ("binlog.format" = "ROW")"""
10194
exception "not support change binlog format from STATEMENT_AND_SNAPSHOT to ROW"
10295
}
103-
104-
sql "DROP TABLE IF EXISTS ${rowTable} FORCE"
105-
sql "DROP TABLE IF EXISTS ${plainTable} FORCE"
10696
}

‎regression-test/suites/table_stream_p0/test_table_stream_alter_comment.groovy‎

Lines changed: 42 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -22,23 +22,11 @@ suite("test_table_stream_alter_comment") {
2222
return
2323
}
2424

25-
def baseTable = "test_stream_alter_comment_base"
26-
def streamName = "test_stream_alter_comment_stream"
27-
28-
def streamComment = { name ->
29-
def rows = sql """
30-
SELECT STREAM_COMMENT FROM information_schema.table_streams
31-
WHERE DB_NAME = DATABASE() AND STREAM_NAME = '${name}'
32-
"""
33-
assertEquals(1, rows.size())
34-
return rows[0][0].toString()
35-
}
36-
37-
sql "DROP STREAM IF EXISTS ${streamName}"
38-
sql "DROP TABLE IF EXISTS ${baseTable} FORCE"
25+
sql "DROP STREAM IF EXISTS test_stream_alter_comment_stream"
26+
sql "DROP TABLE IF EXISTS test_stream_alter_comment_base FORCE"
3927

4028
sql """
41-
CREATE TABLE ${baseTable} (
29+
CREATE TABLE test_stream_alter_comment_base (
4230
k1 INT NOT NULL,
4331
v1 INT
4432
)
@@ -52,34 +40,60 @@ suite("test_table_stream_alter_comment") {
5240
"""
5341

5442
sql """
55-
CREATE STREAM ${streamName} ON TABLE ${baseTable}
43+
CREATE STREAM test_stream_alter_comment_stream ON TABLE test_stream_alter_comment_base
5644
COMMENT 'initial comment'
5745
PROPERTIES ("type" = "append_only")
5846
"""
59-
assertEquals("initial comment", streamComment(streamName))
47+
48+
order_qt_comment_after_create """
49+
SELECT STREAM_COMMENT FROM information_schema.table_streams
50+
WHERE DB_NAME = DATABASE() AND STREAM_NAME = 'test_stream_alter_comment_stream'
51+
"""
6052

6153
// SET COMMENT
62-
sql """ALTER STREAM ${streamName} SET COMMENT 'updated comment'"""
63-
assertEquals("updated comment", streamComment(streamName))
64-
def createStmt = sql("SHOW CREATE STREAM ${streamName}")[0][1].toString()
54+
sql """ALTER STREAM test_stream_alter_comment_stream SET COMMENT 'updated comment'"""
55+
order_qt_comment_after_set """
56+
SELECT STREAM_COMMENT FROM information_schema.table_streams
57+
WHERE DB_NAME = DATABASE() AND STREAM_NAME = 'test_stream_alter_comment_stream'
58+
"""
59+
60+
// the new comment is part of SHOW CREATE STREAM as well. The whole DDL is not used as the
61+
// expected output on purpose: its PROPERTIES section changes as the stream feature evolves.
62+
def createStmt = sql("SHOW CREATE STREAM test_stream_alter_comment_stream")[0][1].toString()
6563
assertTrue(createStmt.contains("COMMENT 'updated comment'"), createStmt)
6664

6765
// MODIFY COMMENT is accepted as well, to stay consistent with ALTER TABLE
68-
sql """ALTER STREAM ${streamName} MODIFY COMMENT 'modified comment'"""
69-
assertEquals("modified comment", streamComment(streamName))
66+
sql """ALTER STREAM test_stream_alter_comment_stream MODIFY COMMENT 'modified comment'"""
67+
order_qt_comment_after_modify """
68+
SELECT STREAM_COMMENT FROM information_schema.table_streams
69+
WHERE DB_NAME = DATABASE() AND STREAM_NAME = 'test_stream_alter_comment_stream'
70+
"""
71+
72+
// the literal is decoded the same way CREATE STREAM decodes it: a doubled quote is one quote
73+
sql """ALTER STREAM test_stream_alter_comment_stream SET COMMENT 'a''b'"""
74+
order_qt_comment_doubled_quote """
75+
SELECT STREAM_COMMENT, LENGTH(STREAM_COMMENT) FROM information_schema.table_streams
76+
WHERE DB_NAME = DATABASE() AND STREAM_NAME = 'test_stream_alter_comment_stream'
77+
"""
7078

7179
// an empty comment clears the comment
72-
sql """ALTER STREAM ${streamName} SET COMMENT ''"""
73-
assertEquals("", streamComment(streamName))
74-
createStmt = sql("SHOW CREATE STREAM ${streamName}")[0][1].toString()
80+
sql """ALTER STREAM test_stream_alter_comment_stream SET COMMENT ''"""
81+
order_qt_comment_after_clear """
82+
SELECT STREAM_COMMENT FROM information_schema.table_streams
83+
WHERE DB_NAME = DATABASE() AND STREAM_NAME = 'test_stream_alter_comment_stream'
84+
"""
85+
createStmt = sql("SHOW CREATE STREAM test_stream_alter_comment_stream")[0][1].toString()
7586
assertFalse(createStmt.contains("COMMENT '"), createStmt)
7687

77-
sql """ALTER STREAM ${streamName} SET COMMENT 'final comment'"""
78-
assertEquals("final comment", streamComment(streamName))
88+
sql """ALTER STREAM test_stream_alter_comment_stream SET COMMENT 'final comment'"""
89+
order_qt_comment_final """
90+
SELECT STREAM_COMMENT FROM information_schema.table_streams
91+
WHERE DB_NAME = DATABASE() AND STREAM_NAME = 'test_stream_alter_comment_stream'
92+
"""
7993

8094
// altering a normal table through ALTER STREAM is rejected
8195
test {
82-
sql """ALTER STREAM ${baseTable} SET COMMENT 'not a stream'"""
96+
sql """ALTER STREAM test_stream_alter_comment_base SET COMMENT 'not a stream'"""
8397
exception "is not STREAM"
8498
}
8599

@@ -88,7 +102,4 @@ suite("test_table_stream_alter_comment") {
88102
sql """ALTER STREAM test_stream_alter_comment_not_exist SET COMMENT 'no such stream'"""
89103
exception "Unknown table"
90104
}
91-
92-
sql "DROP STREAM IF EXISTS ${streamName}"
93-
sql "DROP TABLE IF EXISTS ${baseTable} FORCE"
94105
}

0 commit comments

Comments
 (0)