Repository navigation
Commit c57da5d
committed
[fix](lance) Read managed tables by URI and version, as the BE does
The FE opened a table with managed_versioning through the SDK's namespace
client while the BE opened it by URI and version through lance-c, and the
previous rounds kept adding code to align the two (store identity,
credentials, the SDK's second describe, the manifest it opened). The
premise of namespace mode, that a reader must finalize a staged manifest,
does not hold for Lance's namespaces: they finalize inside
CreateTableVersion and list the canonical manifests.
- The FE reads a managed version by the location and options of the
read's own DescribeTable and the version number, and hands the same
access to the BE. The namespace still decides which versions exist:
Doris lists or describes them itself, and a managed table's access is
not cached. LanceSdkNamespace and its tests are removed.
- The manifest the namespace records for the selected version must be its
canonical path on the chain, or a staged manifest beside it
(LanceManifestPaths); otherwise the read fails and asks to retry. The
canonical name no longer depends on the FE's locale.
- A version recorded only at a staged manifest whose canonical manifest
does not exist cannot be read; nothing is written on read.
- On a managed chain, a version number the namespace no longer records
between recorded versions cuts the FOR TIME AS OF history like a removed
version. A branch's FOR TIME AS OF only checks the version it selects,
as main does, and @Branch no longer needs a recorded main version.
- Version requests run outside the catalog's namespace lock. A missing
table from a branch's version request, after the table was described,
is reported as the missing branch (Lance's Directory namespace reports
it that way). A time before every candidate on main is a user-facing
error.
- The REST fixture can record a version at a staged or arbitrary manifest
path; three managed tables cover an unfinalized head, a finalized staged
record and a misrecorded manifest.1 parent 8e0eda2 commit c57da5d
21 files changed
Lines changed: 1025 additions & 2088 deletions
File tree
- docker/thirdparties/docker-compose/iceberg
- scripts
- fe/fe-core/src
- main/java/org/apache/doris/datasource/lance
- metadata
- storage
- test/java/org/apache/doris/datasource/lance
- metadata
- storage
- regression-test
- data/external_table_p0/lance
- suites/external_table_p0/lance
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
130 | 130 | | |
131 | 131 | | |
132 | 132 | | |
133 | | - | |
| 133 | + | |
134 | 134 | | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
135 | 141 | | |
136 | 142 | | |
137 | 143 | | |
138 | | - | |
| 144 | + | |
139 | 145 | | |
140 | 146 | | |
141 | 147 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
49 | 52 | | |
50 | 53 | | |
51 | 54 | | |
| |||
106 | 109 | | |
107 | 110 | | |
108 | 111 | | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
109 | 145 | | |
110 | 146 | | |
111 | 147 | | |
112 | 148 | | |
113 | | - | |
114 | | - | |
115 | | - | |
| 149 | + | |
| 150 | + | |
116 | 151 | | |
117 | 152 | | |
118 | 153 | | |
| |||
124 | 159 | | |
125 | 160 | | |
126 | 161 | | |
127 | | - | |
128 | | - | |
| 162 | + | |
129 | 163 | | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
139 | | - | |
140 | | - | |
141 | 164 | | |
142 | 165 | | |
143 | | - | |
144 | | - | |
145 | | - | |
146 | | - | |
| 166 | + | |
| 167 | + | |
147 | 168 | | |
148 | 169 | | |
149 | | - | |
150 | | - | |
| 170 | + | |
| 171 | + | |
151 | 172 | | |
152 | 173 | | |
153 | 174 | | |
| |||
166 | 187 | | |
167 | 188 | | |
168 | 189 | | |
169 | | - | |
| 190 | + | |
| 191 | + | |
170 | 192 | | |
171 | | - | |
172 | | - | |
173 | | - | |
174 | | - | |
175 | | - | |
176 | | - | |
177 | | - | |
178 | | - | |
179 | | - | |
180 | | - | |
181 | | - | |
182 | | - | |
183 | | - | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
184 | 203 | | |
185 | 204 | | |
186 | 205 | | |
187 | 206 | | |
188 | 207 | | |
189 | 208 | | |
190 | | - | |
| 209 | + | |
191 | 210 | | |
192 | 211 | | |
193 | 212 | | |
| |||
0 commit comments