Repository navigation
Commit 4728d69
[fix](fe) Keep the Iceberg SDK manifest cache off under a metadata cache weight limit
### What problem does this PR solve?
Issue Number: None
Related PR: #66717, #67726
Problem Summary: Under a Doris metadata cache weight limit the Iceberg catalog left
`io.manifest.cache-enabled` out unless the user set it, so the SDK manifest content cache, which
keeps up to `io.manifest.cache.max-total-bytes` of manifests per FileIO outside the Doris limit,
stayed at its default of off. An Iceberg REST catalog, however, merges the server's `/config`
response into the client properties, and its defaults fill in any key the client leaves out: a
server default of `true` turned the cache on. The catalog now writes `false` explicitly, which the
merge keeps over the server's defaults. The server's overrides and a table's own config still take
precedence over client properties, as the REST protocol intends, and an explicit user setting
still wins as before.
### Release note
Under an external metadata cache weight limit, an Iceberg catalog keeps the SDK manifest cache off
even when an Iceberg REST server's /config defaults enable it, unless the catalog sets
io.manifest.cache-enabled itself.
### Check List (For Author)
- Test: Unit Test
- IcebergCatalogFactoryTest: under a weight limit the catalog properties carry
io.manifest.cache-enabled=false, and merging them with a REST /config response whose defaults
enable the cache keeps it off. Leaving the key out makes it fail.
- Behavior changed: Yes (see Release note)
- Does this need documentation: No
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>1 parent b6db733 commit 4728d69
2 files changed
Lines changed: 33 additions & 15 deletions
File tree
- fe/fe-connector/fe-connector-iceberg/src
- main/java/org/apache/doris/connector/iceberg
- test/java/org/apache/doris/connector/iceberg
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
163 | 163 | | |
164 | 164 | | |
165 | 165 | | |
166 | | - | |
167 | | - | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
168 | 170 | | |
169 | 171 | | |
170 | 172 | | |
| |||
173 | 175 | | |
174 | 176 | | |
175 | 177 | | |
176 | | - | |
177 | | - | |
178 | | - | |
179 | | - | |
180 | | - | |
181 | | - | |
182 | | - | |
183 | | - | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
184 | 191 | | |
185 | 192 | | |
186 | 193 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| 30 | + | |
30 | 31 | | |
31 | 32 | | |
32 | 33 | | |
| |||
226 | 227 | | |
227 | 228 | | |
228 | 229 | | |
229 | | - | |
| 230 | + | |
230 | 231 | | |
231 | 232 | | |
232 | | - | |
233 | | - | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
234 | 237 | | |
235 | 238 | | |
236 | | - | |
| 239 | + | |
237 | 240 | | |
238 | | - | |
| 241 | + | |
239 | 242 | | |
240 | 243 | | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
241 | 252 | | |
242 | 253 | | |
243 | 254 | | |
| |||
0 commit comments