You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit eef214d
Browse filesBrowse the repository at this point in the historyBrowse files
committed
[improvement](fe) Add the branch-4.x weight budget columns to catalog_meta_cache_statistics
### What problem does this PR solve?
Issue Number: None
Related PR: #66717, #67726
Problem Summary: branch-4.x (#66717) shows ten memory-weight columns in
information_schema.catalog_meta_cache_statistics. Master (#67726) has only
MAX_WEIGHT, ESTIMATED_WEIGHT, WEIGHT_REJECT_COUNT and
LAST_WEIGHT_REJECT_REASON, so monitoring queries written for 4.x, and the
weight scenarios of 4.x's own regression suites, fail on master with unknown
columns: WEIGHT_BOUNDED, EVICTION_WEIGHT, CATALOG_MAX_WEIGHT,
CATALOG_ESTIMATED_WEIGHT, GLOBAL_MAX_WEIGHT and GLOBAL_ESTIMATED_WEIGHT.
For an entry without a weight bound master also reports ESTIMATED_WEIGHT and
WEIGHT_REJECT_COUNT as 0, where 4.x reports -1.
The six columns are appended after LAST_WEIGHT_REJECT_REASON with the 4.x
names, types and meanings: whether the entry is weight bounded; the bytes of
the values removed by eviction, by Caffeine or to make room in the budget;
the catalog limit the entry is admitted under (the smaller of the catalog and
FE-global limits) and the bytes charged to its catalog; the FE-global limit
and the bytes charged to it. A limit that does not exist is -1, and an entry
without a weight bound reports -1 in every weight column, as on 4.x. The BE
scanner asks an older FE that rejects the new columns for the leading 28,
then 24 columns. SELECT * positions still differ from 4.x, whose base columns
lack EVICTION_RATE; queries by column name now work on both.
### Release note
information_schema.catalog_meta_cache_statistics adds the columns
WEIGHT_BOUNDED, EVICTION_WEIGHT, CATALOG_MAX_WEIGHT, CATALOG_ESTIMATED_WEIGHT,
GLOBAL_MAX_WEIGHT and GLOBAL_ESTIMATED_WEIGHT. A cache entry without a weight
bound now reports -1 instead of 0 in ESTIMATED_WEIGHT and WEIGHT_REJECT_COUNT.
### Check List (For Author)
- Test: Regression test / Unit Test
- MetaCacheWeightGovernanceTest, FeMetaCacheEntryTest, HudiConnectorHmsCacheTest
- test_paimon_table_meta_cache and test_remote_doris_meta_cache_weight, run against a local cluster;
test_iceberg_table_meta_cache needs the hive3 environment and was not run locally
- Manual: a new BE against an FE without the six columns falls back to the leading 28
- Behavior changed: Yes. A cache entry without a weight bound reports -1 instead of 0 in ESTIMATED_WEIGHT and
WEIGHT_REJECT_COUNT. Upgrade BE before FE: an older BE cannot fill the six new columns.
- Does this need documentation: Yes. The catalog_meta_cache_statistics page should list the six columns (doc PR to follow).
Co-authored-by: Calvin Kirs <[email protected]>
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-cache/src/main/java/org/apache/doris/connector/cache/MetaCacheBudgetManager.java
+20Lines changed: 20 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -530,6 +530,26 @@ public long getUsedWeight() {
530
530
}
531
531
}
532
532
533
+
/** The catalog limit this entry is admitted under: the smaller of the catalog and FE-global limits, or -1. */
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-cache/src/test/java/org/apache/doris/connector/cache/MetaCacheWeightGovernanceTest.java
Copy file name to clipboardExpand all lines: fe/fe-connector/fe-connector-hudi/src/test/java/org/apache/doris/connector/hudi/HudiConnectorHmsCacheTest.java
0 commit comments