Repository navigation
Commit 7ca6e54
authored
Support UDTFs in information_schema.routines / SHOW FUNCTIONS (apache#23438)
# Which issue does this PR close?
- Closes apache#23437.
# Rationale for this change
`information_schema.routines` and `SHOW FUNCTIONS` currently enumerate
only scalar / aggregate / window UDFs. Table functions (UDTFs)
registered via `SessionContext::register_udtf` are omitted, making them
undiscoverable through SQL. Discovery matters because downstream tooling
(DataFusion CLI, Massive's atlas SQL surface, dbt-datafusion, etc.) uses
these SQL surfaces to list available functions.
# What changes are included in this PR?
**1. Snapshot table functions into the information_schema provider.**
`Session::table_functions()` cannot exist on the `Session` trait today:
`TableFunction` lives in `datafusion-catalog`, which already depends on
`datafusion-session` (both `TableProvider::scan` and `TableFunction`
take `&dyn Session`). Adding a trait method returning `&HashMap<String,
Arc<TableFunction>>` would reverse that edge and create a
crate-dependency cycle. `ScalarUDF` / `AggregateUDF` / `WindowUDF` don't
hit this because they live in `datafusion-expr`, which sits below
`datafusion-session`.
This is the same underlying constraint as apache#23348. A follow-up PR will
move `TableFunction` (or hoist the relevant traits) so the builder can
be deleted.
For this PR, `SessionState.table_functions` is snapshotted into
`InformationSchemaProvider` via `with_table_functions()` at
construction. The provider is built per-query, so the snapshot stays
fresh.
**2. Emit UDTF rows in `information_schema.routines` only.**
`make_routines`: one row per UDTF with `routine_type = "FUNCTION"`,
`function_type = "TABLE"`, `data_type = "TABLE"`.
UDTFs deliberately do NOT appear in `information_schema.parameters`. A
same-named scalar UDF (e.g. `generate_series` exists as both a scalar
UDF in `functions-nested` and a UDTF in `functions-table`) would
cross-join with a UDTF row keyed only by (name, rid) and produce
spurious `TABLE`-typed variants of every scalar signature in `SHOW
FUNCTIONS`.
**3. Rewrite the `SHOW FUNCTIONS` SQL to UNION UDTFs directly from
`routines`.**
The old query joined `parameters p (INNER)` requiring both IN and OUT
rows before joining `routines`. UDTFs have no IN parameters, so they
fell out. The new plan sources scalar / aggregate / window signatures
from `parameters` as before, and UDTFs from a separate `UNION` branch
that reads `routines` directly (guarded by `function_type = TABLE`).
This avoids the cross-join blowup mentioned above.
# Are these changes tested?
Verified locally by registering a UDTF and running `SHOW FUNCTIONS`:
```
> SHOW FUNCTIONS LIKE 'stale%'
+---------------+-------------+---------------+
| function_name | return_type | function_type |
| stale_files | TABLE | TABLE |
+---------------+-------------+---------------+
```
Happy to add sqllogictests in
`datafusion/sqllogictest/test_files/information_schema.slt` if
maintainers prefer.
# Are there any user-facing changes?
Yes: `SHOW FUNCTIONS` output now includes table functions.
`information_schema.routines` gains one row per registered UDTF.
`information_schema.parameters` is intentionally unchanged (see
rationale in point 2 above). Existing rows are unchanged.1 parent f151c10 commit 7ca6e54
4 files changed
Lines changed: 109 additions & 19 deletions
File tree
- datafusion
- catalog/src
- core/src/execution
- sqllogictest/test_files
- sql/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
| |||
81 | 82 | | |
82 | 83 | | |
83 | 84 | | |
84 | | - | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
85 | 89 | | |
86 | 90 | | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
87 | 101 | | |
88 | 102 | | |
89 | 103 | | |
90 | 104 | | |
91 | 105 | | |
| 106 | + | |
92 | 107 | | |
93 | 108 | | |
94 | 109 | | |
| |||
301 | 316 | | |
302 | 317 | | |
303 | 318 | | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
304 | 339 | | |
305 | 340 | | |
306 | 341 | | |
| |||
400 | 435 | | |
401 | 436 | | |
402 | 437 | | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
403 | 446 | | |
404 | 447 | | |
405 | 448 | | |
| |||
1522 | 1565 | | |
1523 | 1566 | | |
1524 | 1567 | | |
| 1568 | + | |
1525 | 1569 | | |
1526 | 1570 | | |
1527 | 1571 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
349 | 349 | | |
350 | 350 | | |
351 | 351 | | |
352 | | - | |
353 | | - | |
354 | | - | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
355 | 356 | | |
356 | 357 | | |
357 | 358 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2631 | 2631 | | |
2632 | 2632 | | |
2633 | 2633 | | |
| 2634 | + | |
| 2635 | + | |
| 2636 | + | |
| 2637 | + | |
| 2638 | + | |
| 2639 | + | |
| 2640 | + | |
2634 | 2641 | | |
2635 | 2642 | | |
2636 | 2643 | | |
2637 | | - | |
2638 | | - | |
2639 | | - | |
2640 | | - | |
2641 | | - | |
2642 | | - | |
| 2644 | + | |
| 2645 | + | |
| 2646 | + | |
| 2647 | + | |
| 2648 | + | |
| 2649 | + | |
| 2650 | + | |
| 2651 | + | |
| 2652 | + | |
| 2653 | + | |
| 2654 | + | |
| 2655 | + | |
| 2656 | + | |
| 2657 | + | |
| 2658 | + | |
| 2659 | + | |
| 2660 | + | |
2643 | 2661 | | |
2644 | | - | |
| 2662 | + | |
2645 | 2663 | | |
2646 | 2664 | | |
2647 | 2665 | | |
| |||
2657 | 2675 | | |
2658 | 2676 | | |
2659 | 2677 | | |
2660 | | - | |
2661 | | - | |
2662 | | - | |
| 2678 | + | |
| 2679 | + | |
| 2680 | + | |
2663 | 2681 | | |
2664 | 2682 | | |
2665 | 2683 | | |
| |||
2672 | 2690 | | |
2673 | 2691 | | |
2674 | 2692 | | |
2675 | | - | |
2676 | | - | |
| 2693 | + | |
| 2694 | + | |
2677 | 2695 | | |
2678 | 2696 | | |
2679 | 2697 | | |
2680 | 2698 | | |
2681 | | - | |
| 2699 | + | |
2682 | 2700 | | |
2683 | | - | |
2684 | | - | |
| 2701 | + | |
| 2702 | + | |
| 2703 | + | |
| 2704 | + | |
| 2705 | + | |
| 2706 | + | |
| 2707 | + | |
| 2708 | + | |
| 2709 | + | |
| 2710 | + | |
| 2711 | + | |
| 2712 | + | |
| 2713 | + | |
| 2714 | + | |
| 2715 | + | |
| 2716 | + | |
| 2717 | + | |
| 2718 | + | |
2685 | 2719 | | |
2686 | 2720 | | |
2687 | 2721 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
897 | 897 | | |
898 | 898 | | |
899 | 899 | | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
900 | 911 | | |
901 | 912 | | |
902 | 913 | | |
| |||
0 commit comments