Repository navigation
Commit c94cb29
committed
[fix](nereids) HBO: address persistence code review findings
### What problem does this PR solve?
Problem Summary: v4pro 审阅 commit 3962b55(pinned 落盘 internal db)后
0 Blocker / 0 Major / 7 Minor / 6 Nit,结论可合入;本轮修订全部 Minor 与
可落地 Nit:
- P-1:加载与 SET/DELETE 并发互斥 —— 内存阶段(load 批量写入、SET put、
DELETE invalidate)统一走 pinnedLoadLock;加载改用
`asMap().putIfAbsent` 合并,杜绝加载覆盖并发 SET 新值/复活已删条目。
- P-2:加载失败不再每个 group 重复 DDL+SELECT —— loadAll 失败返回 null
(原返回空表语义无法区分),manager 侧 30s backoff(lastLoadFailedMs)后
才重试。
- P-3:多 FE 语义文档化 —— 每个 FE 进程只加载一次快照、不刷新其它 FE
的 SET(类 javadoc + Config javadoc)。
- P-4:转义复用 StatisticsUtil.escapeSQL(同时双写 ' 与 \\,internal ctx
默认 sql_mode 无 NO_BACKSLASH_ESCAPES 时反斜杠不再被解析吞掉)。
- P-5:struct_info 截断改按 UTF-8 字节并在字符边界截断(varchar 上限按
字节计,原 UTF-16 char 截断对多字节文本可能超限被拒)。
- P-6:replication_num 从硬编码 1 改为 Math.max(1,
Config.min_replication_num_per_tablet),与 InternalSchemaInitializer
约定一致,min_replication>1 集群不再静默建表失败。
- P-7:配置热切换语义文档化(默认 false 期 SET 不落盘、DELETE 不清行;
重新开启后加载以 putIfAbsent 合并、内存新值优先)+ putIfAbsent 落实。
- N-1/N-2/N-3:删除本地 escape(null)死代码、改用
InternalCatalog.INTERNAL_CATALOG_NAME、catch Throwable 收窄为
Exception。
- N-4:回归 finally 先还原 config 再 DELETE,清理失败不再泄漏热配置。
- 记录在案:N-5(.out 尾随空白为生成器固有,惯例接受)、N-6(空串/NULL
以内存 '' 为规范,读侧 NULL->'' 已统一;DB 列保持 NULL 兼容)。
验证:build.sh --fe EXIT=0(checkstyle 0 违规);70k ASCII 与 30k 多字节
STRUCT 截断实测(65533 字节、21844 个完整 3 字节字符、无截断字符);含
引号/反斜杠 STRUCT 落盘往返无损;重启后惰性加载恢复 explain 翻转
(loaded 1 entries);hbo_structinfo_inject_test 与
hbo_persist_internal_db_test 均 Test 1 suites, failed 0。
### Release note
None(实验特性内部修订,默认关闭)
### Check List (For Author)
- Test: FE 构建通过;回归两套通过;手工验证转义/截断/重启恢复
- Behavior changed: No
- Does this need documentation: No1 parent 3962b55 commit c94cb29
4 files changed
Lines changed: 122 additions & 51 deletions
File tree
- fe
- fe-common/src/main/java/org/apache/doris/common
- fe-core/src/main/java/org/apache/doris/nereids/stats
- regression-test/suites/query_p0/hbo
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2333 | 2333 | | |
2334 | 2334 | | |
2335 | 2335 | | |
2336 | | - | |
| 2336 | + | |
| 2337 | + | |
| 2338 | + | |
| 2339 | + | |
| 2340 | + | |
| 2341 | + | |
2337 | 2342 | | |
2338 | 2343 | | |
2339 | 2344 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| 29 | + | |
29 | 30 | | |
30 | 31 | | |
31 | 32 | | |
| |||
36 | 37 | | |
37 | 38 | | |
38 | 39 | | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
39 | 48 | | |
40 | 49 | | |
41 | 50 | | |
42 | 51 | | |
| 52 | + | |
| 53 | + | |
43 | 54 | | |
44 | 55 | | |
45 | 56 | | |
46 | 57 | | |
47 | 58 | | |
48 | 59 | | |
49 | | - | |
| 60 | + | |
50 | 61 | | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
51 | 68 | | |
52 | 69 | | |
53 | 70 | | |
| |||
80 | 97 | | |
81 | 98 | | |
82 | 99 | | |
83 | | - | |
84 | | - | |
85 | | - | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
86 | 105 | | |
87 | 106 | | |
88 | 107 | | |
| |||
97 | 116 | | |
98 | 117 | | |
99 | 118 | | |
100 | | - | |
101 | | - | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
102 | 123 | | |
103 | 124 | | |
104 | 125 | | |
| |||
109 | 130 | | |
110 | 131 | | |
111 | 132 | | |
112 | | - | |
113 | | - | |
| 133 | + | |
114 | 134 | | |
115 | 135 | | |
116 | | - | |
117 | | - | |
118 | | - | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
119 | 153 | | |
120 | 154 | | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | | - | |
| 155 | + | |
127 | 156 | | |
128 | 157 | | |
129 | 158 | | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
130 | 163 | | |
131 | 164 | | |
132 | 165 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
20 | 21 | | |
| 22 | + | |
21 | 23 | | |
22 | 24 | | |
23 | 25 | | |
24 | 26 | | |
25 | 27 | | |
26 | 28 | | |
| 29 | + | |
27 | 30 | | |
28 | 31 | | |
29 | 32 | | |
| |||
42 | 45 | | |
43 | 46 | | |
44 | 47 | | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
61 | 52 | | |
62 | 53 | | |
63 | 54 | | |
64 | 55 | | |
65 | 56 | | |
66 | 57 | | |
67 | | - | |
| 58 | + | |
68 | 59 | | |
69 | 60 | | |
70 | | - | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
71 | 64 | | |
72 | 65 | | |
73 | 66 | | |
74 | 67 | | |
75 | | - | |
76 | | - | |
77 | | - | |
78 | | - | |
| 68 | + | |
| 69 | + | |
79 | 70 | | |
80 | 71 | | |
81 | | - | |
82 | | - | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
83 | 75 | | |
84 | | - | |
| 76 | + | |
85 | 77 | | |
86 | 78 | | |
87 | 79 | | |
| |||
90 | 82 | | |
91 | 83 | | |
92 | 84 | | |
93 | | - | |
| 85 | + | |
| 86 | + | |
94 | 87 | | |
95 | | - | |
| 88 | + | |
96 | 89 | | |
97 | 90 | | |
98 | 91 | | |
99 | 92 | | |
100 | | - | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
101 | 99 | | |
102 | 100 | | |
103 | 101 | | |
| |||
117 | 115 | | |
118 | 116 | | |
119 | 117 | | |
120 | | - | |
| 118 | + | |
| 119 | + | |
121 | 120 | | |
| 121 | + | |
122 | 122 | | |
123 | | - | |
124 | 123 | | |
125 | 124 | | |
126 | | - | |
127 | | - | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
128 | 160 | | |
129 | 161 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
39 | | - | |
| 39 | + | |
40 | 40 | | |
| 41 | + | |
41 | 42 | | |
42 | 43 | | |
0 commit comments