Repository navigation
Commit 910e8ee
Support partitioning of information schema content per catalog (apache#25887)
## Which issue does this PR close?
- Closes apache#15960.
- Supersedes / closes apache#24200.
- Prerequisite for PR apache#26057.
## Rationale for this change
Catalog-qualified information schema queries currently return metadata
for object from every registered catalog.
If we follow the letter of the SQL specification,
`my_catalog.information_schema.tables` should describe only
`my_catalog`. This is consistent with PostgreSQL's information schema
representing the current database.
When working with multiple catalogs, for instance multiple Iceberg REST
catalogs, it is more practical to be able to query just the objects
defined in one specific catalog without further filtering.
A special 'system' catalog is optionally provided to be able to query
across catalogs. This can be seen as an approximation of SQL's
definition schema.
## What changes are included in this PR?
- The catalog list provided to `InformationSchemaProvider` is now
constructed dynamically based on the catalog being queried. An optional
'system' catalog provides the existing global catalog behaviour. All
other catalogs are restricted to just their own content.
- Unqualified information_schema queries now use the configured default
catalog instead of returning metadata from all catalogs.
## What is the testing strategy for this PR?
Added multi-catalog SQL logic coverage for qualified and unqualified
information schema queries.
## Are there any user-facing changes?
Yes. Information schema results are now scoped to the resolved catalog.
---------
Co-authored-by: Vaibhave Sekar <[email protected]>
Co-authored-by: Andrew Lamb <[email protected]>1 parent da11e68 commit 910e8ee
10 files changed
Lines changed: 613 additions & 137 deletions
File tree
- datafusion
- catalog/src
- common/src
- core/src/execution
- context
- execution/src
- sqllogictest/test_files
- sql/src
- docs/source/user-guide
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| 47 | + | |
47 | 48 | | |
48 | 49 | | |
49 | 50 | | |
| |||
81 | 82 | | |
82 | 83 | | |
83 | 84 | | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
84 | 132 | | |
85 | 133 | | |
86 | 134 | | |
87 | 135 | | |
| 136 | + | |
88 | 137 | | |
89 | 138 | | |
90 | 139 | | |
| |||
100 | 149 | | |
101 | 150 | | |
102 | 151 | | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
103 | 157 | | |
104 | 158 | | |
105 | 159 | | |
106 | 160 | | |
| 161 | + | |
107 | 162 | | |
108 | 163 | | |
109 | 164 | | |
| |||
115 | 170 | | |
116 | 171 | | |
117 | 172 | | |
| 173 | + | |
118 | 174 | | |
119 | | - | |
120 | | - | |
| 175 | + | |
| 176 | + | |
121 | 177 | | |
122 | 178 | | |
123 | 179 | | |
| |||
128 | 184 | | |
129 | 185 | | |
130 | 186 | | |
131 | | - | |
| 187 | + | |
132 | 188 | | |
133 | 189 | | |
134 | 190 | | |
| |||
142 | 198 | | |
143 | 199 | | |
144 | 200 | | |
145 | | - | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
146 | 215 | | |
147 | 216 | | |
148 | 217 | | |
| |||
179 | 248 | | |
180 | 249 | | |
181 | 250 | | |
182 | | - | |
183 | | - | |
184 | | - | |
185 | | - | |
186 | | - | |
187 | | - | |
188 | | - | |
189 | | - | |
190 | | - | |
191 | | - | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
192 | 258 | | |
193 | 259 | | |
194 | 260 | | |
| |||
197 | 263 | | |
198 | 264 | | |
199 | 265 | | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
200 | 285 | | |
201 | 286 | | |
202 | 287 | | |
| |||
209 | 294 | | |
210 | 295 | | |
211 | 296 | | |
212 | | - | |
213 | | - | |
214 | | - | |
215 | | - | |
216 | | - | |
217 | | - | |
218 | | - | |
219 | | - | |
220 | | - | |
221 | | - | |
222 | | - | |
223 | | - | |
224 | | - | |
225 | | - | |
226 | | - | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
227 | 304 | | |
228 | 305 | | |
229 | 306 | | |
| |||
232 | 309 | | |
233 | 310 | | |
234 | 311 | | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
235 | 335 | | |
236 | 336 | | |
237 | 337 | | |
| |||
561 | 661 | | |
562 | 662 | | |
563 | 663 | | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
564 | 682 | | |
565 | 683 | | |
566 | 684 | | |
| |||
572 | 690 | | |
573 | 691 | | |
574 | 692 | | |
575 | | - | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
576 | 707 | | |
577 | 708 | | |
578 | 709 | | |
| |||
588 | 719 | | |
589 | 720 | | |
590 | 721 | | |
591 | | - | |
592 | | - | |
593 | | - | |
594 | | - | |
595 | 722 | | |
596 | 723 | | |
597 | 724 | | |
| |||
1553 | 1680 | | |
1554 | 1681 | | |
1555 | 1682 | | |
| 1683 | + | |
1556 | 1684 | | |
1557 | 1685 | | |
1558 | 1686 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
242 | 242 | | |
243 | 243 | | |
244 | 244 | | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
245 | 251 | | |
246 | 252 | | |
247 | 253 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2859 | 2859 | | |
2860 | 2860 | | |
2861 | 2861 | | |
2862 | | - | |
| 2862 | + | |
| 2863 | + | |
| 2864 | + | |
2863 | 2865 | | |
2864 | 2866 | | |
2865 | 2867 | | |
| |||
2875 | 2877 | | |
2876 | 2878 | | |
2877 | 2879 | | |
2878 | | - | |
| 2880 | + | |
2879 | 2881 | | |
2880 | 2882 | | |
2881 | 2883 | | |
| |||
0 commit comments