Repository navigation
Commit 41581e5
authored
[fix](streaming-job) drop neighbour-table rows leaked by JDBC LIKE wildcards in JdbcPostgreSQLClient (apache#63402)
### What problem does this PR solve?
`JdbcPostgreSQLClient.getJdbcColumnsInfo` calls
`DatabaseMetaData.getColumns(catalog, schemaPattern, tableNamePattern,
columnNamePattern)`. Per the JDBC spec the 3rd argument is a **SQL LIKE
pattern**, so literal `_` / `%` characters in the requested table name
are interpreted as wildcards by the Postgres driver. When a streaming
job is created with `include_tables = "user_info_pg_normal1"` and a
neighbour table like `userXinfo_pg_normal1` happens to coexist in the
same schema, the metadata query returns columns from **both** tables.
The combined result then trips `CREATE TABLE` on the Doris side with
errors such as `errCode = 2, detailMessage = Duplicate column name
'name'`, or pollutes the auto-created table schema with stray columns.
The repro is trivial: in the same Postgres schema create
- `user_info_pg_normal1(name varchar, age int2)` — the table we want to
capture
- `userXinfo_pg_normal1(name varchar, weight float8)` — a decoy whose
name only differs from the target by a single character that `_` matches
then run `CREATE JOB ... include_tables = "user_info_pg_normal1"`.
Without the fix the schema fetched for the target leaks `weight` (or
`Duplicate column name 'name'`, depending on column order).
Fix: after fetching the `ResultSet`, drop rows whose `TABLE_NAME` does
not exactly equal the requested `remoteTableName`. We deliberately do
**not** escape `_` / `%` at the source — relying on
`DatabaseMetaData.getSearchStringEscape()` is driver-version dependent
(older Oracle drivers don't honour escape sequences in `getTables`),
while filtering on the consumer side is deterministic and
driver-agnostic.
Scope:
- Only `JdbcPostgreSQLClient` is patched. This is the path used by
Postgres streaming jobs (the failing case). MySQL streaming jobs were
checked against the same decoy pattern and do not reproduce the bug
because MySQL Connector/J doesn't pull neighbour rows here in practice —
so `JdbcMySQLClient` is left untouched in this PR.
- The JDBC catalog path lives in a separate module
(`fe-connector-jdbc/.../JdbcConnectorClient`) and is **not** part of
this PR. It already does partial escape but intentionally skips `_` /
`%` for driver-compatibility reasons; a follow-up can apply the same
after-the-fact filter there.1 parent db27399 commit 41581e5
2 files changed
Lines changed: 32 additions & 0 deletions
File tree
- fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/client
- regression-test/suites/job_p0/streaming_job/cdc
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
63 | 63 | | |
64 | 64 | | |
65 | 65 | | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
66 | 71 | | |
67 | 72 | | |
68 | 73 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
78 | 101 | | |
79 | 102 | | |
80 | 103 | | |
| |||
108 | 131 | | |
109 | 132 | | |
110 | 133 | | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
111 | 138 | | |
112 | 139 | | |
113 | 140 | | |
| |||
0 commit comments