Repository navigation
Commit 66c9704
committed
[fix](nereids) HBO: address doris-skills code review round 1
### What problem does this PR solve?
Problem Summary: 按 doris-skills(doris-repo-review 方法论:风险扫描 → 分模块
并发 v4pro full-review → 共享 ledger 收敛)对 master..hbo-structinfo 全量
diff(25 文件)复审。合并后 A1 Major / A2 Minor / C1 Minor 及 Nit C2-C4、
N-B1 全部落实:
- A1(Major,pinned filter-on-scan 谓词盲):注入 key 为 scan-group
fingerprint(表 token 不含 filter 常量),pinned 绕过 learned 侧
ScanPlanStatistics 谓词匹配 → 同表同版本同裁剪下不同常量共享同一 pin。
该语义为 scan-group key 模型的固有权衡(SET 无 plan 上下文,谓词敏感
key 不可表达),修复=精确文档化(HboStatsCalculator 注释 + 设计文档
§4.1)+ 回归锁定:同扫描不同常量(R.b=1 vs b=2)共享 pin 且同样翻转,
防止未来误改 key 模型。
- A2/N-B2(Minor):HboUtils.genPlanStatisticsMap struct 模式下快照整体为
空(全部节点无 fingerprint)时单独 warn(原逻辑仅部分缺失时告警,最坏
诊断场景反而静默)。
- C1(Minor):putPinnedPlanStatistics 的 DB persist 移入 pinnedLoadLock,
与 DELETE 的 DB 删除同锁 —— 同键并发 SET/DELETE 的 DB 写序与内存序
一致,库行恒等于最后内存写入者(修复 round2 "last memory writer ==
last DB writer" 不成立的残余)。
- C2(Nit):pinned cache 上限语义对齐兄弟 hbo cache(num<=0 为无界而非
静默禁用注入),Config javadoc 注明。
- C3(Nit):HboStatisticsCommand fingerprint 统一小写(ROOT),SET 与
DELETE 均做 64-hex 校验 —— 大写/畸形 DELETE 不再静默无效。
- C4(Nit):builder 行数 Long.parseLong 溢出 → 清晰 ParseException。
- N-B1(Nit):EXPLAIN 注解块仅在物理层级输出,排除 PARSED/ANALYZED/
REWRITTEN 的误导性 "(no hbo fingerprint attached ...)" 回退。
- B-open(filter 发布 key 一致性)经代码证据排除:扫描父 filter 折叠进干净
scan 时不注册 nereids→planId 映射(仅建 SelectNode 路径注册),collect
不会覆盖 scan 条目;SelectNode filter 条目以其自身组 fp 发布,读侧无
任何 lookup 使用该 key,惰性无害。
验证:build.sh --fe EXIT=0(checkstyle 0 违规);hbo_structinfo_inject_test
(含新增 b=2 共享 pin 断言)与 hbo_persist_internal_db_test 均 Test 1
suites, failed 0;手工负例:大写 SET→小写 DELETE 命中、畸形 DELETE 报错、
rows 溢出报 "out of range"、PARSED/REWRITTEN explain 无注解块而默认
explain 有块。
### Release note
None(实验特性内部修订,默认关闭)
### Check List (For Author)
- Test: FE 构建通过;回归两套通过;手工负例验证
- Behavior changed: No
- Does this need documentation: No1 parent d95d7fb commit 66c9704
8 files changed
Lines changed: 61 additions & 13 deletions
File tree
- fe
- fe-common/src/main/java/org/apache/doris/common
- fe-core/src/main/java/org/apache/doris/nereids
- parser
- stats
- trees/plans/commands
- regression-test/suites/query_p0/hbo
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2322 | 2322 | | |
2323 | 2323 | | |
2324 | 2324 | | |
2325 | | - | |
| 2325 | + | |
| 2326 | + | |
2326 | 2327 | | |
2327 | 2328 | | |
2328 | 2329 | | |
2329 | | - | |
| 2330 | + | |
| 2331 | + | |
2330 | 2332 | | |
2331 | 2333 | | |
2332 | 2334 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1290 | 1290 | | |
1291 | 1291 | | |
1292 | 1292 | | |
1293 | | - | |
| 1293 | + | |
| 1294 | + | |
| 1295 | + | |
| 1296 | + | |
| 1297 | + | |
| 1298 | + | |
| 1299 | + | |
| 1300 | + | |
1294 | 1301 | | |
1295 | 1302 | | |
1296 | 1303 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7114 | 7114 | | |
7115 | 7115 | | |
7116 | 7116 | | |
7117 | | - | |
| 7117 | + | |
| 7118 | + | |
| 7119 | + | |
| 7120 | + | |
| 7121 | + | |
| 7122 | + | |
7118 | 7123 | | |
7119 | 7124 | | |
7120 | 7125 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
67 | | - | |
68 | | - | |
69 | | - | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
70 | 74 | | |
71 | 75 | | |
72 | 76 | | |
| |||
124 | 128 | | |
125 | 129 | | |
126 | 130 | | |
127 | | - | |
128 | | - | |
129 | | - | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
130 | 137 | | |
131 | 138 | | |
132 | 139 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
101 | 101 | | |
102 | 102 | | |
103 | 103 | | |
104 | | - | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
105 | 111 | | |
106 | 112 | | |
107 | 113 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
535 | 535 | | |
536 | 536 | | |
537 | 537 | | |
| 538 | + | |
538 | 539 | | |
539 | 540 | | |
540 | 541 | | |
| |||
554 | 555 | | |
555 | 556 | | |
556 | 557 | | |
| 558 | + | |
557 | 559 | | |
558 | 560 | | |
559 | 561 | | |
| |||
570 | 572 | | |
571 | 573 | | |
572 | 574 | | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
573 | 581 | | |
574 | 582 | | |
575 | 583 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
78 | | - | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
79 | 81 | | |
80 | 82 | | |
81 | 83 | | |
| |||
87 | 89 | | |
88 | 90 | | |
89 | 91 | | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
90 | 95 | | |
91 | 96 | | |
92 | | - | |
93 | 97 | | |
94 | 98 | | |
95 | 99 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
83 | 83 | | |
84 | 84 | | |
85 | 85 | | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
86 | 95 | | |
87 | 96 | | |
88 | 97 | | |
| |||
0 commit comments