Repository navigation
Commit 74e1260
committed
[chore](conf) Remove deprecated/unused FE/BE configs and stale session variables
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Doris has accumulated a number of FE configs, BE configs and session
variables that are either explicitly marked `@Deprecated`, or simply have
no remaining references anywhere in the codebase (production sources,
regression tests, docs, or tools). They add noise to the user-facing
configuration surface and to the source tree without changing any
behavior. This commit removes them in three coordinated parts and adds
the minimum amount of backward-compatibility plumbing required to keep a
BE-then-FE rolling upgrade safe.
1. FE configs (fe/fe-common/.../Config.java)
Remove 13 `@Deprecated` fields with no remaining production
references: `meta_publish_timeout_ms`, `backup_plugin_path`,
`max_unfinished_load_job`, `use_new_tablet_scheduler`,
`enable_concurrent_update`, `cbo_max_statistics_job_num`,
`max_cbo_statistics_task_timeout_sec`,
`cbo_concurrency_statistics_task_num`, `cbo_default_sample_percentage`,
`finish_job_max_saved_second`, `enable_array_type`,
`period_analyze_simultaneously_running_task_num`,
`maximum_parallelism_of_export_job`.
`ConfigBase.java` gains a small `warnDeprecatedKeys()` helper. After
`setFields()` runs, if any of the legacy keys above is still listed in
`fe.conf`, FE logs a single WARN line so operators notice the stale
entry on upgrade. FE startup is not affected: the FE config loader
iterates over Java fields, so unknown keys in `fe.conf` were already
silently ignored before this change.
2. BE configs (be/src/common/config.{cpp,h}, be/src/cloud/config.{cpp,h})
Remove 24 configs that have zero references in the BE source tree:
`doris_scanner_max_run_time_ms`, `multi_get_max_threads`, `num_disks`,
`read_size`, `load_max_wg_active_memtable_percent`,
`jdbc_connection_pool_cache_clear_time_sec`, `big_column_size_buffer`,
`small_column_size_buffer`, `max_fragment_start_wait_time_seconds`,
`enable_workload_group_for_scan`,
`workload_group_scan_task_wait_timeout_ms`,
`variant_use_cloud_schema_dict_cache`,
`variant_threshold_rows_to_estimate_sparse_column`,
`variant_nested_group_max_depth`, `enable_ttl_cache_evict_using_lru`,
`file_cache_background_ttl_gc_batch`,
`kerberos_refresh_interval_second`, `enable_debug_log_timeout_secs`,
`enable_index_compaction`, `schema_dict_cache_capacity`,
`test_s3_resource`, `meta_service_use_load_balancer`,
`meta_service_rpc_timeout_ms`, `delete_bitmap_rpc_retry_times`.
The BE config loader (`be/src/common/config.cpp` `init()`) also
iterates over the registered field map, so unknown keys in `be.conf`
were already silently ignored — hard removal is safe on a rolling
upgrade. None of the removed options affect on-disk data format or
the BE↔BE / BE↔FE wire format, so the standard BE-first rolling
upgrade ordering is preserved.
3. Session variables
(fe/fe-core/.../qe/SessionVariable.java, .../qe/VariableMgr.java)
Remove 15 stale Doris-specific session variables along with their
public name constants and any getter/setter methods:
`use_v2_rollup`,
`rewrite_count_distinct_to_bitmap_hll`,
`enable_variant_access_in_original_planner`, `extract_wide_range_expr`,
`auto_broadcast_join_threshold`, `runtime_filters_max_num`,
`disable_inverted_index_v1_for_variant`, `enable_infer_predicate`,
`limit_rows_for_single_instance`, `nereids_star_schema_support`,
`enable_cbo_statistics`, `enable_eliminate_sort_node`,
`drop_table_if_ctas_failed`, `trace_nereids`,
`enable_sync_mv_cost_based_rewrite`.
Selection criteria for each removed variable: zero references in
`fe/fe-core/src/main/java` and `fe/fe-common/src/main/java` outside
`SessionVariable.java` itself (after expanding the search to cover
`get`/`set`/`is`-prefixed accessors of the field name, the public
name constant, the underlying lowercase string literal, the body of
`toThrift()`, and the body of `setForwardedSessionVariables()`), and
zero hits in `regression-test/`, `docs/`, `tools/`, and `be/test/`.
Upgrade compatibility is the critical concern here. Unlike fe.conf and
be.conf, unknown session-variable names cause
`VariableMgr` to throw `ERR_UNKNOWN_SYSTEM_VARIABLE` at four sites
(`setVar`, `setVarForNonMasterFE`, `fillValue`, `getValue`). A hard
delete would therefore break existing user scripts and JDBC client
connection init paths that still issue `SET` on the old names during
a BE-then-FE rolling upgrade. To stay safe, `VariableMgr` now keeps
an explicit `REMOVED_SESSION_VAR_NAMES` set listing the 15 removed
lowercase names; each of the four lookup sites short-circuits on this
set so that `SET <removed_var> = ...` silently no-ops and
`SELECT @@<removed_var>` returns an empty string. The same pattern is
already used for `mysql_compat_var_whitelist`.
The legacy FE configs and BE configs listed above have been removed. The
following session variables have been removed; `SET` and `SELECT @@`
against them now silently succeed (with empty/no-op behavior) instead of
returning the variable's previous default:
`use_v2_rollup`,
`rewrite_count_distinct_to_bitmap_hll`,
`enable_variant_access_in_original_planner`, `extract_wide_range_expr`,
`auto_broadcast_join_threshold`, `runtime_filters_max_num`,
`disable_inverted_index_v1_for_variant`, `enable_infer_predicate`,
`limit_rows_for_single_instance`, `nereids_star_schema_support`,
`enable_cbo_statistics`, `enable_eliminate_sort_node`,
`drop_table_if_ctas_failed`, `trace_nereids`,
`enable_sync_mv_cost_based_rewrite`. If any of the removed FE keys is
still present in `fe.conf` on upgrade, FE logs a WARN line on startup.1 parent 7fc8e27 commit 74e1260
8 files changed
Lines changed: 131 additions & 346 deletions
File tree
- be/src
- cloud
- common
- fe
- fe-common/src/main/java/org/apache/doris/common
- fe-core/src/main/java/org/apache/doris/qe
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
28 | | - | |
29 | | - | |
30 | 28 | | |
31 | 29 | | |
32 | 30 | | |
| |||
107 | 105 | | |
108 | 106 | | |
109 | 107 | | |
110 | | - | |
111 | | - | |
112 | 108 | | |
113 | 109 | | |
114 | 110 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
58 | 58 | | |
59 | 59 | | |
60 | 60 | | |
61 | | - | |
62 | 61 | | |
63 | 62 | | |
64 | 63 | | |
| |||
147 | 146 | | |
148 | 147 | | |
149 | 148 | | |
150 | | - | |
151 | | - | |
152 | 149 | | |
153 | 150 | | |
154 | 151 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| 37 | + | |
37 | 38 | | |
38 | 39 | | |
39 | 40 | | |
| |||
343 | 344 | | |
344 | 345 | | |
345 | 346 | | |
346 | | - | |
347 | | - | |
348 | 347 | | |
349 | 348 | | |
350 | 349 | | |
| |||
509 | 508 | | |
510 | 509 | | |
511 | 510 | | |
512 | | - | |
513 | | - | |
514 | | - | |
515 | 511 | | |
516 | 512 | | |
517 | 513 | | |
| |||
686 | 682 | | |
687 | 683 | | |
688 | 684 | | |
689 | | - | |
690 | | - | |
691 | 685 | | |
692 | 686 | | |
693 | 687 | | |
694 | 688 | | |
695 | | - | |
696 | 689 | | |
697 | 690 | | |
698 | 691 | | |
| |||
748 | 741 | | |
749 | 742 | | |
750 | 743 | | |
751 | | - | |
752 | | - | |
753 | | - | |
754 | 744 | | |
755 | 745 | | |
756 | 746 | | |
| |||
978 | 968 | | |
979 | 969 | | |
980 | 970 | | |
981 | | - | |
982 | | - | |
983 | | - | |
984 | 971 | | |
985 | 972 | | |
986 | 973 | | |
| |||
1054 | 1041 | | |
1055 | 1042 | | |
1056 | 1043 | | |
1057 | | - | |
1058 | | - | |
1059 | | - | |
1060 | 1044 | | |
1061 | 1045 | | |
1062 | 1046 | | |
| |||
1090 | 1074 | | |
1091 | 1075 | | |
1092 | 1076 | | |
1093 | | - | |
1094 | | - | |
1095 | | - | |
1096 | | - | |
1097 | | - | |
1098 | | - | |
1099 | | - | |
1100 | 1077 | | |
1101 | 1078 | | |
1102 | 1079 | | |
| |||
1156 | 1133 | | |
1157 | 1134 | | |
1158 | 1135 | | |
1159 | | - | |
1160 | | - | |
1161 | | - | |
1162 | | - | |
1163 | | - | |
1164 | | - | |
1165 | 1136 | | |
1166 | 1137 | | |
1167 | 1138 | | |
1168 | 1139 | | |
1169 | 1140 | | |
1170 | | - | |
1171 | | - | |
1172 | | - | |
1173 | | - | |
1174 | 1141 | | |
1175 | 1142 | | |
1176 | 1143 | | |
| |||
1207 | 1174 | | |
1208 | 1175 | | |
1209 | 1176 | | |
1210 | | - | |
1211 | | - | |
1212 | | - | |
1213 | 1177 | | |
1214 | 1178 | | |
1215 | 1179 | | |
| |||
1240 | 1204 | | |
1241 | 1205 | | |
1242 | 1206 | | |
1243 | | - | |
1244 | 1207 | | |
1245 | 1208 | | |
1246 | 1209 | | |
| |||
1342 | 1305 | | |
1343 | 1306 | | |
1344 | 1307 | | |
1345 | | - | |
1346 | | - | |
1347 | | - | |
1348 | 1308 | | |
1349 | 1309 | | |
1350 | 1310 | | |
| |||
1460 | 1420 | | |
1461 | 1421 | | |
1462 | 1422 | | |
1463 | | - | |
1464 | | - | |
1465 | | - | |
1466 | 1423 | | |
1467 | 1424 | | |
1468 | 1425 | | |
| |||
1578 | 1535 | | |
1579 | 1536 | | |
1580 | 1537 | | |
1581 | | - | |
1582 | | - | |
1583 | 1538 | | |
1584 | 1539 | | |
1585 | 1540 | | |
| |||
1693 | 1648 | | |
1694 | 1649 | | |
1695 | 1650 | | |
1696 | | - | |
1697 | | - | |
1698 | 1651 | | |
1699 | 1652 | | |
1700 | 1653 | | |
| |||
1787 | 1740 | | |
1788 | 1741 | | |
1789 | 1742 | | |
1790 | | - | |
1791 | | - | |
1792 | 1743 | | |
1793 | 1744 | | |
1794 | 1745 | | |
| |||
2095 | 2046 | | |
2096 | 2047 | | |
2097 | 2048 | | |
| 2049 | + | |
| 2050 | + | |
| 2051 | + | |
| 2052 | + | |
| 2053 | + | |
| 2054 | + | |
| 2055 | + | |
| 2056 | + | |
| 2057 | + | |
| 2058 | + | |
| 2059 | + | |
| 2060 | + | |
| 2061 | + | |
| 2062 | + | |
| 2063 | + | |
| 2064 | + | |
| 2065 | + | |
| 2066 | + | |
| 2067 | + | |
| 2068 | + | |
| 2069 | + | |
| 2070 | + | |
| 2071 | + | |
| 2072 | + | |
| 2073 | + | |
| 2074 | + | |
| 2075 | + | |
| 2076 | + | |
| 2077 | + | |
| 2078 | + | |
| 2079 | + | |
2098 | 2080 | | |
2099 | 2081 | | |
2100 | 2082 | | |
| |||
2123 | 2105 | | |
2124 | 2106 | | |
2125 | 2107 | | |
| 2108 | + | |
| 2109 | + | |
| 2110 | + | |
| 2111 | + | |
| 2112 | + | |
| 2113 | + | |
| 2114 | + | |
| 2115 | + | |
2126 | 2116 | | |
2127 | 2117 | | |
2128 | 2118 | | |
| |||
0 commit comments