Skip to content

[Opt](encoding) Let storage_format control default encoding for interger and binary - #59075

Merged
dataroaring merged 8 commits into
apache:masterfrom
bobhan1:segment-v3-controls-default-encoding
Dec 18, 2025
Merged

[Opt](encoding) Let storage_format control default encoding for interger and binary#59075
dataroaring merged 8 commits into
apache:masterfrom
bobhan1:segment-v3-controls-default-encoding

Conversation

@bobhan1

@bobhan1 bobhan1 commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@bobhan1
bobhan1 force-pushed the segment-v3-controls-default-encoding branch 4 times, most recently from 2c78869 to c6b151b Compare December 16, 2025 07:39
@bobhan1
bobhan1 force-pushed the segment-v3-controls-default-encoding branch from e21385e to 7c4c779 Compare December 16, 2025 08:36
@bobhan1
bobhan1 force-pushed the segment-v3-controls-default-encoding branch 2 times, most recently from 887bfd8 to 64cc954 Compare December 16, 2025 15:14
@bobhan1
bobhan1 force-pushed the segment-v3-controls-default-encoding branch from 64cc954 to 3cfa9cc Compare December 16, 2025 15:38
@bobhan1
bobhan1 force-pushed the segment-v3-controls-default-encoding branch from ae43559 to d5d7572 Compare December 16, 2025 15:52
@bobhan1

bobhan1 commented Dec 16, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Dec 16, 2025

Copy link
Copy Markdown

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.85% (1752/2194)
Line Coverage 65.75% (30749/46764)
Region Coverage 66.54% (15367/23096)
Branch Coverage 56.86% (8168/14364)

@hello-stephen

hello-stephen commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

@doris-robot

doris-robot commented Dec 16, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 35103 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d5d7572adccfd9f77adc7c999bf1a9790a2be778, data reload: false

------ Round 1 ----------------------------------
q1	17598	4155	4094	4094
q2	2054	365	242	242
q3	10467	1322	734	734
q4	10311	816	322	322
q5	9971	2173	1955	1955
q6	241	172	137	137
q7	1019	855	712	712
q8	9373	1425	1246	1246
q9	7203	5398	5302	5302
q10	6879	2396	1951	1951
q11	524	331	297	297
q12	714	730	589	589
q13	17795	3718	3062	3062
q14	286	302	274	274
q15	590	523	506	506
q16	692	675	633	633
q17	697	757	591	591
q18	8018	7258	6992	6992
q19	1098	967	608	608
q20	415	370	242	242
q21	4245	4086	3624	3624
q22	1054	994	990	990
Total cold run time: 111244 ms
Total hot run time: 35103 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4413	4068	4080	4068
q2	325	395	324	324
q3	2137	2704	2315	2315
q4	1302	1748	1314	1314
q5	4730	4709	4609	4609
q6	216	173	133	133
q7	2020	2034	1787	1787
q8	2722	2496	2556	2496
q9	7497	7689	7390	7390
q10	3119	3283	2932	2932
q11	595	517	489	489
q12	712	752	629	629
q13	3628	4001	3355	3355
q14	300	300	282	282
q15	543	523	505	505
q16	666	761	676	676
q17	1215	1531	1583	1531
q18	8030	7900	7766	7766
q19	856	872	899	872
q20	2010	2016	1836	1836
q21	4612	4403	4187	4187
q22	1099	1001	980	980
Total cold run time: 52747 ms
Total hot run time: 50476 ms

@doris-robot

doris-robot commented Dec 16, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 178599 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d5d7572adccfd9f77adc7c999bf1a9790a2be778, data reload: false

query5	4443	579	425	425
query6	335	232	229	229
query7	4219	471	274	274
query8	301	239	230	230
query9	8797	2570	2565	2565
query10	518	370	341	341
query11	15204	14794	14655	14655
query12	171	128	129	128
query13	1277	505	401	401
query14	5788	3023	2785	2785
query14_1	2688	2677	2667	2667
query15	239	201	178	178
query16	910	489	459	459
query17	1153	714	592	592
query18	2443	454	360	360
query19	231	236	209	209
query20	122	117	116	116
query21	221	154	118	118
query22	3877	3994	3991	3991
query23	16550	16302	16121	16121
query23_1	16113	16059	15919	15919
query24	7368	1682	1284	1284
query24_1	1263	1254	1254	1254
query25	590	503	456	456
query26	1245	277	169	169
query27	2753	463	318	318
query28	4466	2149	2139	2139
query29	848	574	484	484
query30	318	248	218	218
query31	839	717	596	596
query32	81	72	67	67
query33	561	356	301	301
query34	918	923	555	555
query35	773	878	730	730
query36	866	909	805	805
query37	131	99	79	79
query38	2866	2905	2756	2756
query39	774	733	710	710
query39_1	698	698	688	688
query40	233	139	117	117
query41	65	64	64	64
query42	105	104	101	101
query43	417	449	419	419
query44	1320	743	742	742
query45	194	186	183	183
query46	879	974	602	602
query47	1648	1694	1607	1607
query48	320	337	245	245
query49	631	428	345	345
query50	669	294	219	219
query51	3793	3755	3849	3755
query52	113	109	99	99
query53	316	349	290	290
query54	309	259	245	245
query55	84	76	72	72
query56	290	299	292	292
query57	1120	1128	1092	1092
query58	269	294	253	253
query59	2447	2603	2396	2396
query60	306	317	289	289
query61	162	159	159	159
query62	709	658	648	648
query63	326	293	304	293
query64	5056	1295	1010	1010
query65	4027	3992	3942	3942
query66	1451	434	321	321
query67	15308	14911	14788	14788
query68	8314	984	756	756
query69	500	340	311	311
query70	1064	1011	957	957
query71	374	301	280	280
query72	6145	4985	5071	4985
query73	703	610	315	315
query74	8908	8874	8597	8597
query75	3201	3183	2813	2813
query76	3916	1131	720	720
query77	602	396	293	293
query78	9327	9739	8867	8867
query79	1263	927	633	633
query80	705	696	553	553
query81	512	267	236	236
query82	228	137	105	105
query83	252	254	240	240
query84	262	124	107	107
query85	894	500	467	467
query86	380	297	284	284
query87	3030	3075	2935	2935
query88	3174	2280	2292	2280
query89	458	429	397	397
query90	2182	155	156	155
query91	171	176	152	152
query92	83	63	59	59
query93	1122	906	566	566
query94	472	287	264	264
query95	586	330	310	310
query96	611	466	216	216
query97	2290	2339	2215	2215
query98	215	188	192	188
query99	1315	1329	1216	1216
Total cold run time: 258998 ms
Total hot run time: 178599 ms

@doris-robot

doris-robot commented Dec 16, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 28.34 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d5d7572adccfd9f77adc7c999bf1a9790a2be778, data reload: false

query1	0.05	0.04	0.04
query2	0.14	0.07	0.07
query3	0.31	0.08	0.08
query4	1.61	0.10	0.10
query5	0.28	0.25	0.25
query6	1.20	0.64	0.65
query7	0.03	0.03	0.02
query8	0.07	0.06	0.06
query9	0.58	0.51	0.50
query10	0.56	0.57	0.56
query11	0.26	0.13	0.13
query12	0.27	0.14	0.14
query13	0.62	0.63	0.60
query14	1.02	1.02	1.01
query15	0.87	0.81	0.82
query16	0.39	0.39	0.38
query17	1.02	1.07	1.06
query18	0.25	0.22	0.21
query19	1.93	1.86	1.78
query20	0.02	0.02	0.02
query21	15.40	0.28	0.24
query22	4.99	0.10	0.10
query23	15.39	0.38	0.22
query24	2.46	0.44	0.31
query25	0.10	0.09	0.09
query26	0.17	0.17	0.17
query27	0.09	0.09	0.09
query28	3.87	1.37	1.18
query29	12.64	4.17	3.32
query30	0.35	0.12	0.10
query31	2.83	0.63	0.43
query32	3.24	0.60	0.49
query33	3.05	2.97	3.11
query34	16.87	5.18	4.58
query35	4.69	4.60	4.65
query36	0.62	0.50	0.48
query37	0.24	0.08	0.09
query38	0.21	0.06	0.06
query39	0.07	0.05	0.05
query40	0.21	0.19	0.16
query41	0.13	0.05	0.07
query42	0.07	0.05	0.05
query43	0.06	0.06	0.06
Total cold run time: 99.23 s
Total hot run time: 28.34 s

@hello-stephen

hello-stephen commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 86.61% (97/112) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.36% (18819/35271)
Line Coverage 39.17% (174336/445031)
Region Coverage 33.73% (134864/399810)
Branch Coverage 34.65% (58100/167693)

@hello-stephen

hello-stephen commented Dec 16, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 87.39% (97/111) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.27% (24979/34564)
Line Coverage 59.04% (262403/444415)
Region Coverage 53.91% (218029/404437)
Branch Coverage 55.43% (93406/168504)

@bobhan1

bobhan1 commented Dec 17, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Dec 17, 2025

Copy link
Copy Markdown

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.85% (1752/2194)
Line Coverage 65.75% (30747/46764)
Region Coverage 66.53% (15366/23096)
Branch Coverage 56.85% (8166/14364)

@hello-stephen

hello-stephen commented Dec 17, 2025

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

@doris-robot

doris-robot commented Dec 17, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 34842 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6ac9f4ed051fad082c59ed0c538402e2af115021, data reload: false

------ Round 1 ----------------------------------
q1	17650	4338	4027	4027
q2	2017	351	243	243
q3	10231	1387	746	746
q4	10279	807	309	309
q5	8654	2214	1901	1901
q6	213	169	133	133
q7	995	864	710	710
q8	9348	1451	1174	1174
q9	7243	5347	5368	5347
q10	6885	2409	1975	1975
q11	541	325	302	302
q12	715	716	578	578
q13	17790	3713	3099	3099
q14	302	296	277	277
q15	606	531	514	514
q16	705	698	650	650
q17	957	859	482	482
q18	7654	7124	6925	6925
q19	1447	959	615	615
q20	416	358	248	248
q21	4216	3946	3643	3643
q22	1030	987	944	944
Total cold run time: 109894 ms
Total hot run time: 34842 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4169	4096	4074	4074
q2	333	424	308	308
q3	2126	2707	2293	2293
q4	1354	1757	1269	1269
q5	4271	5056	4549	4549
q6	228	175	127	127
q7	2097	1967	1840	1840
q8	2722	2606	2572	2572
q9	7724	7609	7623	7609
q10	3061	3253	2782	2782
q11	604	522	504	504
q12	664	748	612	612
q13	3515	3946	3330	3330
q14	282	305	271	271
q15	561	503	497	497
q16	653	681	709	681
q17	1290	1473	1387	1387
q18	7727	7664	7439	7439
q19	874	838	869	838
q20	2089	2031	1987	1987
q21	4862	4312	4298	4298
q22	1063	1009	979	979
Total cold run time: 52269 ms
Total hot run time: 50246 ms

@doris-robot

doris-robot commented Dec 17, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 178337 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6ac9f4ed051fad082c59ed0c538402e2af115021, data reload: false

query5	4956	614	462	462
query6	378	269	227	227
query7	4231	484	274	274
query8	327	257	256	256
query9	8768	2576	2538	2538
query10	542	368	329	329
query11	15432	14770	14564	14564
query12	177	116	114	114
query13	1253	512	380	380
query14	6221	3031	2786	2786
query14_1	2673	2643	2693	2643
query15	217	196	177	177
query16	952	485	465	465
query17	1136	742	597	597
query18	2494	437	341	341
query19	237	227	198	198
query20	120	119	114	114
query21	219	135	120	120
query22	3947	3885	3854	3854
query23	16517	16174	15986	15986
query23_1	16020	15936	16101	15936
query24	7387	1629	1245	1245
query24_1	1259	1263	1274	1263
query25	546	478	437	437
query26	1246	276	156	156
query27	2753	465	301	301
query28	4520	2142	2135	2135
query29	811	556	445	445
query30	315	247	227	227
query31	841	714	586	586
query32	85	71	71	71
query33	529	342	296	296
query34	915	933	548	548
query35	793	839	728	728
query36	881	908	825	825
query37	128	98	83	83
query38	2855	2858	2769	2769
query39	755	759	720	720
query39_1	709	712	707	707
query40	227	142	124	124
query41	68	67	64	64
query42	115	112	107	107
query43	430	443	403	403
query44	1366	745	749	745
query45	195	193	183	183
query46	915	999	636	636
query47	1616	1693	1600	1600
query48	337	336	274	274
query49	639	455	359	359
query50	672	299	222	222
query51	3859	3835	3822	3822
query52	112	110	106	106
query53	333	371	314	314
query54	312	290	282	282
query55	85	78	72	72
query56	305	325	326	325
query57	1143	1162	1081	1081
query58	290	275	268	268
query59	2433	2454	2387	2387
query60	331	333	317	317
query61	193	192	189	189
query62	712	668	622	622
query63	332	307	313	307
query64	5066	1423	1133	1133
query65	4044	3957	3950	3950
query66	1461	458	330	330
query67	15357	14987	14747	14747
query68	7211	1022	729	729
query69	517	360	328	328
query70	1053	992	978	978
query71	369	357	290	290
query72	5886	4976	5131	4976
query73	676	606	307	307
query74	8824	8781	8522	8522
query75	3234	3164	2790	2790
query76	3819	1163	768	768
query77	522	411	298	298
query78	9454	9510	8836	8836
query79	1613	915	618	618
query80	708	676	598	598
query81	522	272	242	242
query82	518	135	115	115
query83	265	260	251	251
query84	264	122	105	105
query85	1029	518	466	466
query86	398	299	310	299
query87	2987	3082	2961	2961
query88	4404	2285	2294	2285
query89	474	430	419	419
query90	2138	169	159	159
query91	179	175	143	143
query92	84	68	66	66
query93	1691	916	563	563
query94	476	308	256	256
query95	586	334	301	301
query96	606	485	215	215
query97	2270	2273	2224	2224
query98	221	192	192	192
query99	1324	1330	1214	1214
Total cold run time: 261298 ms
Total hot run time: 178337 ms

@doris-robot

doris-robot commented Dec 17, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 27.22 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6ac9f4ed051fad082c59ed0c538402e2af115021, data reload: false

query1	0.05	0.04	0.04
query2	0.10	0.04	0.04
query3	0.25	0.08	0.09
query4	1.60	0.11	0.11
query5	0.27	0.24	0.28
query6	1.16	0.64	0.63
query7	0.04	0.02	0.03
query8	0.05	0.05	0.05
query9	0.57	0.52	0.50
query10	0.55	0.55	0.55
query11	0.16	0.11	0.12
query12	0.16	0.11	0.12
query13	0.62	0.60	0.62
query14	1.01	0.97	0.98
query15	0.81	0.80	0.81
query16	0.38	0.38	0.40
query17	1.00	1.02	0.99
query18	0.23	0.21	0.22
query19	1.96	1.85	1.82
query20	0.02	0.02	0.01
query21	15.44	0.29	0.14
query22	4.70	0.05	0.05
query23	15.87	0.27	0.10
query24	0.95	0.48	0.18
query25	0.09	0.10	0.07
query26	0.14	0.12	0.12
query27	0.09	0.06	0.06
query28	3.57	1.21	1.02
query29	12.59	4.07	3.20
query30	0.28	0.15	0.11
query31	2.81	0.62	0.39
query32	3.24	0.55	0.45
query33	3.01	3.00	3.01
query34	16.87	5.13	4.58
query35	4.62	4.58	5.30
query36	0.70	0.57	0.55
query37	0.12	0.07	0.07
query38	0.08	0.04	0.03
query39	0.05	0.03	0.03
query40	0.17	0.15	0.15
query41	0.09	0.03	0.02
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 96.55 s
Total hot run time: 27.22 s

@bobhan1
bobhan1 marked this pull request as ready for review December 17, 2025 06:45
@doris-robot

doris-robot commented Dec 17, 2025

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 86.61% (97/112) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.36% (18819/35271)
Line Coverage 39.18% (174341/445010)
Region Coverage 33.76% (134969/399776)
Branch Coverage 34.66% (58113/167689)

@hello-stephen

hello-stephen commented Dec 17, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 87.39% (97/111) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.95% (24868/34564)
Line Coverage 58.67% (260736/444394)
Region Coverage 53.53% (216483/404403)
Branch Coverage 55.11% (92866/168500)

@bobhan1
bobhan1 requested a review from w41ter as a code owner December 17, 2025 08:37
@bobhan1

bobhan1 commented Dec 17, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

hello-stephen commented Dec 17, 2025

Copy link
Copy Markdown
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.85% (1752/2194)
Line Coverage 65.72% (30759/46806)
Region Coverage 66.46% (15359/23111)
Branch Coverage 56.82% (8168/14374)

@doris-robot

doris-robot commented Dec 17, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 36756 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4a247b87ec944b421f4a28b032dcea1f160a9644, data reload: false

------ Round 1 ----------------------------------
q1	17639	4312	4132	4132
q2	2001	357	247	247
q3	10186	1360	737	737
q4	10226	900	328	328
q5	7550	2152	2014	2014
q6	221	174	139	139
q7	1003	854	762	762
q8	9391	1549	1200	1200
q9	7161	5427	5374	5374
q10	6917	2416	1983	1983
q11	543	337	304	304
q12	713	743	573	573
q13	17781	3714	3019	3019
q14	301	302	287	287
q15	595	521	505	505
q16	721	667	623	623
q17	710	829	556	556
q18	7579	7901	7731	7731
q19	1214	1031	703	703
q20	431	384	252	252
q21	4681	4338	4253	4253
q22	1133	1066	1034	1034
Total cold run time: 108697 ms
Total hot run time: 36756 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4385	4327	4312	4312
q2	347	434	319	319
q3	2334	2865	2472	2472
q4	1478	1863	1373	1373
q5	4610	4619	4440	4440
q6	220	174	125	125
q7	2094	1939	1781	1781
q8	2744	2529	2554	2529
q9	7614	7535	7568	7535
q10	3072	3222	2706	2706
q11	607	512	480	480
q12	632	686	553	553
q13	3362	3859	3104	3104
q14	270	285	262	262
q15	542	495	495	495
q16	665	642	612	612
q17	1137	1348	1301	1301
q18	7412	6993	6938	6938
q19	859	839	859	839
q20	1913	1953	1807	1807
q21	4652	4271	4187	4187
q22	1080	1021	957	957
Total cold run time: 52029 ms
Total hot run time: 49127 ms

@doris-robot

doris-robot commented Dec 17, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 178063 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4a247b87ec944b421f4a28b032dcea1f160a9644, data reload: false

query5	4432	626	477	477
query6	360	242	218	218
query7	4234	468	281	281
query8	330	260	244	244
query9	8763	2580	2593	2580
query10	542	416	331	331
query11	15249	15207	14692	14692
query12	173	121	118	118
query13	1277	512	384	384
query14	5980	3011	2816	2816
query14_1	2694	2687	2716	2687
query15	219	201	185	185
query16	887	474	446	446
query17	1154	730	610	610
query18	2512	449	354	354
query19	243	236	208	208
query20	118	116	115	115
query21	225	146	117	117
query22	3846	3976	3735	3735
query23	16550	16024	15879	15879
query23_1	16066	16131	15846	15846
query24	7337	1696	1215	1215
query24_1	1271	1221	1264	1221
query25	585	511	466	466
query26	1262	270	167	167
query27	2749	460	309	309
query28	4460	2135	2126	2126
query29	814	554	465	465
query30	320	242	225	225
query31	817	683	602	602
query32	82	69	73	69
query33	539	345	291	291
query34	931	923	548	548
query35	786	813	738	738
query36	847	922	820	820
query37	129	102	85	85
query38	2878	2929	2860	2860
query39	744	733	710	710
query39_1	710	701	692	692
query40	223	140	122	122
query41	66	63	64	63
query42	107	108	109	108
query43	438	429	398	398
query44	1370	759	750	750
query45	193	190	184	184
query46	910	995	617	617
query47	1651	1699	1621	1621
query48	313	325	263	263
query49	662	495	355	355
query50	696	300	221	221
query51	3771	3885	3819	3819
query52	106	113	100	100
query53	318	348	294	294
query54	285	265	259	259
query55	78	74	75	74
query56	292	298	296	296
query57	1135	1141	1090	1090
query58	274	265	254	254
query59	2385	2528	2404	2404
query60	308	303	294	294
query61	188	164	161	161
query62	706	677	654	654
query63	325	301	308	301
query64	5051	1331	1012	1012
query65	4015	3938	3964	3938
query66	1439	439	340	340
query67	15251	15019	14785	14785
query68	7151	1012	731	731
query69	495	346	308	308
query70	1083	1026	934	934
query71	366	307	276	276
query72	6118	4937	5023	4937
query73	701	607	314	314
query74	8809	8796	8633	8633
query75	3173	3180	2791	2791
query76	3858	1136	748	748
query77	522	399	287	287
query78	9392	9543	8801	8801
query79	1682	898	613	613
query80	922	649	557	557
query81	532	279	239	239
query82	418	135	105	105
query83	262	254	240	240
query84	263	126	108	108
query85	918	518	458	458
query86	426	301	301	301
query87	3037	3110	3034	3034
query88	3313	2293	2278	2278
query89	491	418	392	392
query90	2059	166	160	160
query91	176	168	144	144
query92	77	71	63	63
query93	1554	953	565	565
query94	540	300	271	271
query95	580	386	305	305
query96	619	494	218	218
query97	2301	2349	2209	2209
query98	215	194	190	190
query99	1287	1326	1217	1217
Total cold run time: 259000 ms
Total hot run time: 178063 ms

@doris-robot

doris-robot commented Dec 17, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 28.29 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4a247b87ec944b421f4a28b032dcea1f160a9644, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.25	0.09	0.09
query4	1.61	0.12	0.11
query5	0.28	0.27	0.26
query6	1.18	0.67	0.63
query7	0.03	0.03	0.02
query8	0.05	0.04	0.04
query9	0.58	0.52	0.51
query10	0.55	0.55	0.55
query11	0.16	0.10	0.11
query12	0.15	0.12	0.13
query13	0.61	0.60	0.61
query14	0.97	0.98	0.98
query15	0.82	0.79	0.84
query16	0.43	0.42	0.40
query17	1.06	0.99	1.00
query18	0.23	0.21	0.21
query19	1.95	1.86	1.88
query20	0.02	0.02	0.01
query21	15.43	0.29	0.15
query22	4.77	0.06	0.05
query23	15.97	0.30	0.10
query24	1.62	0.73	0.94
query25	0.10	0.06	0.06
query26	0.14	0.14	0.13
query27	0.07	0.06	0.04
query28	4.21	1.23	1.03
query29	12.59	4.07	3.20
query30	0.27	0.14	0.12
query31	2.83	0.64	0.40
query32	3.23	0.56	0.46
query33	3.07	3.02	3.05
query34	16.79	5.15	4.58
query35	4.54	4.95	5.17
query36	0.72	0.53	0.54
query37	0.11	0.07	0.07
query38	0.07	0.05	0.04
query39	0.04	0.03	0.04
query40	0.18	0.15	0.15
query41	0.08	0.03	0.03
query42	0.04	0.02	0.03
query43	0.04	0.03	0.04
Total cold run time: 98 s
Total hot run time: 28.29 s

@hello-stephen

hello-stephen commented Dec 17, 2025

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 86.61% (97/112) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.34% (18813/35267)
Line Coverage 39.17% (174307/445010)
Region Coverage 33.74% (134882/399750)
Branch Coverage 34.64% (58092/167685)

@hello-stephen

hello-stephen commented Dec 17, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 87.39% (97/111) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.33% (24996/34560)
Line Coverage 59.09% (262602/444394)
Region Coverage 54.00% (218346/404377)
Branch Coverage 55.50% (93508/168496)

@hello-stephen

hello-stephen commented Dec 17, 2025

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

@bobhan1 bobhan1 mentioned this pull request Dec 17, 2025
Merged
16 tasks
eldenmoon
eldenmoon approved these changes Dec 18, 2025
View reviewed changes

@eldenmoon eldenmoon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Dec 18, 2025
@github-actions

github-actions Bot commented Dec 18, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actions Bot commented Dec 18, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

dataroaring
dataroaring approved these changes Dec 18, 2025
View reviewed changes

@dataroaring dataroaring left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring
dataroaring merged commit 37114b8 into apache:master Dec 18, 2025
27 of 29 checks passed
yiguolei pushed a commit that referenced this pull request Dec 19, 2025
@bobhan1
branch-4.0: [Opt](storage) Optimize storage occupation #56815 #56846 #…
6560401 
…57220 #57327 #59075 (#57441)

### What problem does this PR solve?
pick:
- #56815
- #56846
- #57220
- #57327
- #59075

Clickbench result on alicloud c6.4xlarge machine, Intel(R) Xeon(R)
Platinum 8269CY CPU @ 2.50GHz

||(baseline) LZ4F, 64KB data page size, 48 bucket,
binary_plain_encoding_default_impl=v1,
integer_type_default_use_plain_encoding =false| zstd(3), 1MB data page
size, 48 bucket, binary_plain_encoding_default_impl=v1,
integer_type_default_use_plain_encoding =false | zstd(3), 1MB data page
size, 48 bucket, binary_plain_encoding_default_impl=v2,
integer_type_default_use_plain_encoding =true |
|---|---|---|---|
|data size|16.087 GB| 11.016 GB | 8.56 GB |
|load time|592s| 555s | 539s |
|cold run total| 214.22 s | 177.57 s | 131.4 s |
|hot run total| 36.03 s | 39.82 s | 39.05 s |
### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@eldenmoon eldenmoon eldenmoon approved these changes
@dataroaring dataroaring dataroaring approved these changes
@gavinchou gavinchou Awaiting requested review from gavinchou gavinchou is a code owner
@yiguolei yiguolei Awaiting requested review from yiguolei yiguolei is a code owner
@w41ter w41ter Awaiting requested review from w41ter

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@bobhan1 @doris-robot @hello-stephen @eldenmoon @dataroaring @yiguolei