Repository navigation
Commit f109e13
[fix](arrow-flight) Support table type metadata and filters (#67899)
### What problem does this PR solve?
Related to #67578 (partial A3).
Flight SQL clients can obtain FlightInfo for GetTableTypes, but reading
its ticket fails with UNIMPLEMENTED. GetTables also forwards only the
first table_types entry to a service that treats only VIEW as a filter:
[VIEW, BASE TABLE] loses base tables, while [BASE TABLE] or an unknown
type can return excluded rows.
Implement the type stream using the same Doris names as GetTables (BASE
TABLE, SYSTEM VIEW and VIEW), sorted according to Flight SQL. Match the
complete filter against the existing authorized listing before
collecting optional schemas. Request only name/type metadata to avoid
unused table statistics, and retain the session guard and Arrow buffer
cleanup on failed sends.
A type is the TABLE_TYPE that information_schema.tables reports for the
object: table streams are VIEW, and external catalogs report their
tables and views as BASE TABLE. The service lists every authorized
object and the filter runs on that listing. Requested names match
case-insensitively, and TABLE and SYSTEM TABLE, which Flight SQL calls
commonly supported, select BASE TABLE and SYSTEM VIEW: the ADBC Flight
SQL driver asks for `table` (Doris's own ADBC catalog lists a Doris
source with `getObjects(..., {"table"})`), and JDBC tools ask for
`TABLE`. Any other name matches only a type of that name, so UNKNOWN
selects nothing.
Because the filter now runs on the full listing,
`FrontendServiceImpl.listTableStatus` refreshes a cloud follower's
partition versions from the meta service only when UPDATE_TIME is
requested, the one column that reads them. GetTables, which asks for
names and types, never waits on those RPCs or is cut short when they
fail, and information_schema.tables scans that do not read UPDATE_TIME
skip the refresh too.
Co-authored-by: morningman <[email protected]>
Co-authored-by: Gabriel <[email protected]>1 parent 173a0e2 commit f109e13
7 files changed
Lines changed: 601 additions & 11 deletions
File tree
- fe/fe-core/src
- main/java/org/apache/doris
- arrowflight
- service
- test/java/org/apache/doris
- arrowflight
- service
- regression-test/suites/arrow_flight_sql_p0
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
619 | 619 | | |
620 | 620 | | |
621 | 621 | | |
622 | | - | |
| 622 | + | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
623 | 632 | | |
624 | 633 | | |
625 | 634 | | |
| |||
761 | 770 | | |
762 | 771 | | |
763 | 772 | | |
764 | | - | |
| 773 | + | |
765 | 774 | | |
766 | 775 | | |
767 | 776 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| 25 | + | |
25 | 26 | | |
26 | 27 | | |
27 | 28 | | |
| |||
64 | 65 | | |
65 | 66 | | |
66 | 67 | | |
| 68 | + | |
67 | 69 | | |
| 70 | + | |
68 | 71 | | |
| 72 | + | |
69 | 73 | | |
70 | 74 | | |
71 | 75 | | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
72 | 82 | | |
73 | 83 | | |
74 | 84 | | |
75 | 85 | | |
76 | 86 | | |
77 | 87 | | |
78 | | - | |
| 88 | + | |
79 | 89 | | |
80 | 90 | | |
81 | 91 | | |
| |||
161 | 171 | | |
162 | 172 | | |
163 | 173 | | |
164 | | - | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
165 | 196 | | |
166 | 197 | | |
167 | 198 | | |
| |||
199 | 230 | | |
200 | 231 | | |
201 | 232 | | |
202 | | - | |
203 | | - | |
204 | | - | |
| 233 | + | |
| 234 | + | |
205 | 235 | | |
206 | 236 | | |
207 | 237 | | |
| |||
318 | 348 | | |
319 | 349 | | |
320 | 350 | | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
321 | 362 | | |
322 | 363 | | |
323 | 364 | | |
| |||
333 | 374 | | |
334 | 375 | | |
335 | 376 | | |
336 | | - | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
337 | 387 | | |
338 | 388 | | |
339 | 389 | | |
340 | 390 | | |
341 | | - | |
| 391 | + | |
342 | 392 | | |
343 | 393 | | |
344 | 394 | | |
| |||
347 | 397 | | |
348 | 398 | | |
349 | 399 | | |
350 | | - | |
| 400 | + | |
351 | 401 | | |
352 | 402 | | |
353 | 403 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
756 | 756 | | |
757 | 757 | | |
758 | 758 | | |
| 759 | + | |
| 760 | + | |
| 761 | + | |
| 762 | + | |
759 | 763 | | |
760 | | - | |
| 764 | + | |
| 765 | + | |
761 | 766 | | |
762 | 767 | | |
763 | 768 | | |
| |||
0 commit comments