Repository navigation
Commit e850bff
committed
[fix](lance) Carry the endpoint under both keys object_store consults
Review of the previous commit turned up a hole in it. `aws_endpoint_url_s3` is
not an alias of the generic endpoint: object_store parses it into a config key
of its own and always prefers it,
// object_store-0.13.2 src/aws/builder.rs
let endpoint = self.s3_endpoint.or(self.endpoint);
so folding it onto the generic entry, as that commit did, replaced a precedence
rule with map order. A namespace vending the standard AWS pair - a global
`AWS_ENDPOINT_URL` plus an `AWS_ENDPOINT_URL_S3` override - would land on either
value depending on iteration, and `allow_http` would follow whichever won.
Worse, the map never contained that spelling at all, so `with_env_s3` was free
to fill it from the environment, and object_store would then prefer it over the
catalog's endpoint. A stray `AWS_ENDPOINT_URL_S3` in the FE or BE process
silently redirected every request - the exact failure the previous commit set
out to close, on the one key its approach could not reach.
Both go away by writing the resolved endpoint under both spellings: object_store
reads the same value whichever key it consults, `with_env_s3` finds both
occupied, and OpenDAL ignores the S3-specific one as an unknown field. A
namespace that vends both spellings now resolves to the S3-specific value, which
is the one object_store itself would have used.
Also from the review:
- The NUL and protected-key log lines now name the offending key, with the
NULs rendered, since an unnamed dropped credential surfaces later as an
opaque 403.
- allow_http gets a note that its canonical spelling buys no protection from
the environment: `StorageOptions::new` overwrites that key outright from
AWS_ALLOW_HTTP before `with_env_s3` runs.
- testAllowHttpIsRetractedWhenTheEndpointBecomesHttps now covers the canonical
spelling too. It only exercised `endpoint`, so the alias table's identity
rows were uncovered: dropping `aws_endpoint -> aws_endpoint` left a stale
allow_http=true behind a vended HTTPS endpoint and broke no test.
Claude-Session: https://claude.ai/code/session_01PfVSEUJsfWwKy7WEuBaFCG1 parent ed17099 commit e850bff
2 files changed
Lines changed: 115 additions & 36 deletions
File tree
- fe/fe-core/src
- main/java/org/apache/doris/datasource/lance
- test/java/org/apache/doris/datasource/lance
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
50 | 56 | | |
51 | | - | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
52 | 65 | | |
53 | 66 | | |
54 | 67 | | |
| |||
106 | 119 | | |
107 | 120 | | |
108 | 121 | | |
109 | | - | |
110 | | - | |
111 | | - | |
112 | | - | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
113 | 126 | | |
114 | 127 | | |
115 | 128 | | |
| |||
150 | 163 | | |
151 | 164 | | |
152 | 165 | | |
153 | | - | |
| 166 | + | |
154 | 167 | | |
155 | 168 | | |
156 | 169 | | |
| |||
168 | 181 | | |
169 | 182 | | |
170 | 183 | | |
171 | | - | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
172 | 188 | | |
173 | | - | |
| 189 | + | |
174 | 190 | | |
175 | 191 | | |
176 | 192 | | |
177 | 193 | | |
178 | | - | |
179 | | - | |
180 | | - | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
181 | 197 | | |
182 | 198 | | |
183 | 199 | | |
184 | 200 | | |
185 | 201 | | |
186 | | - | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
187 | 206 | | |
188 | 207 | | |
189 | 208 | | |
190 | 209 | | |
191 | 210 | | |
192 | 211 | | |
193 | 212 | | |
194 | | - | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
195 | 220 | | |
196 | 221 | | |
197 | 222 | | |
| |||
202 | 227 | | |
203 | 228 | | |
204 | 229 | | |
205 | | - | |
| 230 | + | |
206 | 231 | | |
207 | 232 | | |
208 | 233 | | |
209 | | - | |
| 234 | + | |
| 235 | + | |
210 | 236 | | |
211 | 237 | | |
212 | 238 | | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
213 | 244 | | |
214 | | - | |
| 245 | + | |
215 | 246 | | |
216 | | - | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
217 | 253 | | |
218 | 254 | | |
219 | 255 | | |
220 | 256 | | |
221 | 257 | | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
222 | 263 | | |
223 | 264 | | |
224 | 265 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| 26 | + | |
26 | 27 | | |
27 | 28 | | |
28 | 29 | | |
| 30 | + | |
29 | 31 | | |
30 | 32 | | |
31 | 33 | | |
| |||
49 | 51 | | |
50 | 52 | | |
51 | 53 | | |
| 54 | + | |
| 55 | + | |
52 | 56 | | |
53 | 57 | | |
54 | 58 | | |
| |||
71 | 75 | | |
72 | 76 | | |
73 | 77 | | |
74 | | - | |
| 78 | + | |
75 | 79 | | |
76 | 80 | | |
77 | 81 | | |
| |||
141 | 145 | | |
142 | 146 | | |
143 | 147 | | |
144 | | - | |
| 148 | + | |
145 | 149 | | |
146 | 150 | | |
147 | | - | |
148 | | - | |
149 | | - | |
150 | | - | |
| 151 | + | |
151 | 152 | | |
152 | 153 | | |
153 | 154 | | |
| |||
156 | 157 | | |
157 | 158 | | |
158 | 159 | | |
159 | | - | |
160 | | - | |
161 | | - | |
162 | | - | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
163 | 166 | | |
164 | 167 | | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
165 | 195 | | |
166 | 196 | | |
167 | 197 | | |
| |||
238 | 268 | | |
239 | 269 | | |
240 | 270 | | |
241 | | - | |
242 | | - | |
243 | | - | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
244 | 277 | | |
245 | | - | |
246 | | - | |
247 | | - | |
248 | | - | |
249 | | - | |
250 | | - | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
251 | 289 | | |
252 | 290 | | |
253 | 291 | | |
| |||
0 commit comments