Skip to content

[ut](datatype) Add BE UT for some basic data types and serializers - #48788

Merged
hello-stephen merged 5 commits into
apache:masterfrom
jacktengg:be-core-ut-datatypes
Mar 12, 2025
Merged

[ut](datatype) Add BE UT for some basic data types and serializers#48788
hello-stephen merged 5 commits into
apache:masterfrom
jacktengg:be-core-ut-datatypes

Conversation

@jacktengg

@jacktengg jacktengg commented Mar 6, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
DataTypeNumber, DataTypeDecimal, DataTypeString, DataTypeDateTimeV2

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

hello-stephen commented Mar 6, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@jacktengg
jacktengg force-pushed the be-core-ut-datatypes branch from 6cb0295 to f8fe567 Compare March 6, 2025 17:03
@jacktengg

jacktengg commented Mar 6, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@jacktengg
jacktengg force-pushed the be-core-ut-datatypes branch 2 times, most recently from f25334b to b9b981b Compare March 6, 2025 17:06
@jacktengg

jacktengg commented Mar 6, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Mar 6, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 32381 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b9b981b162592571a03b6d3ef3921f35879ea833, data reload: false

------ Round 1 ----------------------------------
q1	17582	5217	5065	5065
q2	2050	315	166	166
q3	10445	1352	708	708
q4	10237	1040	542	542
q5	7638	2315	2408	2315
q6	189	169	138	138
q7	929	750	618	618
q8	9331	1389	1080	1080
q9	4955	4658	4630	4630
q10	6823	2305	1893	1893
q11	479	285	264	264
q12	356	355	217	217
q13	17780	3708	3031	3031
q14	232	233	208	208
q15	545	502	492	492
q16	615	611	582	582
q17	584	858	354	354
q18	6919	6555	6353	6353
q19	1900	961	546	546
q20	316	321	186	186
q21	2773	2190	2020	2020
q22	1072	1020	973	973
Total cold run time: 103750 ms
Total hot run time: 32381 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5201	5132	5113	5113
q2	239	328	231	231
q3	2134	2682	2292	2292
q4	1448	1845	1415	1415
q5	4276	4154	4140	4140
q6	209	164	122	122
q7	1908	1996	1754	1754
q8	2589	2707	2629	2629
q9	7261	7086	7267	7086
q10	2976	3151	2734	2734
q11	579	492	491	491
q12	689	806	634	634
q13	3541	3848	3252	3252
q14	277	288	284	284
q15	529	479	471	471
q16	628	686	645	645
q17	1139	1617	1339	1339
q18	7861	7721	7528	7528
q19	828	795	822	795
q20	1981	2027	1916	1916
q21	5494	4911	4811	4811
q22	1095	1013	1001	1001
Total cold run time: 52882 ms
Total hot run time: 50683 ms

@doris-robot

doris-robot commented Mar 6, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 184848 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b9b981b162592571a03b6d3ef3921f35879ea833, data reload: false

query1	980	377	372	372
query2	6528	1977	1870	1870
query3	6797	234	220	220
query4	26236	23916	23148	23148
query5	4337	642	476	476
query6	306	191	189	189
query7	4598	508	291	291
query8	283	238	216	216
query9	8592	2571	2541	2541
query10	447	302	245	245
query11	15577	15056	14955	14955
query12	158	117	106	106
query13	1639	506	398	398
query14	9154	6082	6380	6082
query15	222	191	173	173
query16	7129	625	454	454
query17	923	713	565	565
query18	1942	409	310	310
query19	197	193	160	160
query20	122	114	115	114
query21	210	127	110	110
query22	4184	4157	4216	4157
query23	33763	32957	33178	32957
query24	7755	2410	2415	2410
query25	528	455	382	382
query26	1239	276	151	151
query27	2463	486	327	327
query28	4247	2413	2413	2413
query29	769	540	425	425
query30	286	216	188	188
query31	914	828	761	761
query32	72	65	64	64
query33	544	359	332	332
query34	785	846	502	502
query35	785	833	721	721
query36	939	971	869	869
query37	123	96	82	82
query38	4319	4094	4066	4066
query39	1467	1394	1407	1394
query40	206	115	105	105
query41	58	53	51	51
query42	122	103	102	102
query43	508	540	490	490
query44	1286	784	782	782
query45	175	171	163	163
query46	837	1016	638	638
query47	1755	1795	1705	1705
query48	368	399	295	295
query49	771	500	428	428
query50	682	736	398	398
query51	4179	4217	4172	4172
query52	106	99	98	98
query53	226	256	192	192
query54	480	485	443	443
query55	81	85	82	82
query56	257	268	244	244
query57	1115	1142	1071	1071
query58	246	243	237	237
query59	2669	2704	2566	2566
query60	295	264	249	249
query61	119	117	118	117
query62	799	761	699	699
query63	229	182	188	182
query64	4354	991	715	715
query65	4457	4359	4312	4312
query66	1141	409	312	312
query67	16012	15642	15243	15243
query68	8071	865	511	511
query69	457	296	267	267
query70	1195	1098	1060	1060
query71	394	290	262	262
query72	5617	3523	3607	3523
query73	711	683	345	345
query74	9198	9091	8942	8942
query75	3271	3203	2743	2743
query76	3273	1171	742	742
query77	546	368	277	277
query78	10063	10088	9346	9346
query79	1224	827	580	580
query80	606	518	458	458
query81	475	257	225	225
query82	195	126	95	95
query83	168	173	160	160
query84	287	94	71	71
query85	738	426	299	299
query86	371	321	296	296
query87	4486	4437	4289	4289
query88	2800	2182	2173	2173
query89	392	311	284	284
query90	1988	210	204	204
query91	139	148	110	110
query92	80	60	57	57
query93	1463	1058	584	584
query94	657	415	299	299
query95	356	259	250	250
query96	482	554	269	269
query97	3297	3394	3335	3335
query98	221	214	195	195
query99	1327	1406	1310	1310
Total cold run time: 269640 ms
Total hot run time: 184848 ms

@doris-robot

doris-robot commented Mar 6, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.06 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b9b981b162592571a03b6d3ef3921f35879ea833, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.03	0.04
query3	0.24	0.07	0.06
query4	1.63	0.10	0.11
query5	0.56	0.54	0.54
query6	1.20	0.73	0.71
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.61	0.53	0.52
query10	0.59	0.60	0.58
query11	0.15	0.10	0.10
query12	0.14	0.11	0.12
query13	0.62	0.61	0.59
query14	2.66	2.71	2.67
query15	0.93	0.85	0.86
query16	0.38	0.38	0.38
query17	1.01	1.01	1.00
query18	0.21	0.21	0.19
query19	1.88	1.74	1.97
query20	0.02	0.01	0.02
query21	15.35	0.91	0.55
query22	0.77	1.33	0.78
query23	14.69	1.40	0.62
query24	6.96	1.70	1.06
query25	0.50	0.25	0.13
query26	0.56	0.16	0.13
query27	0.05	0.05	0.06
query28	9.61	0.84	0.45
query29	12.54	4.02	3.29
query30	0.24	0.09	0.07
query31	2.81	0.60	0.39
query32	3.23	0.56	0.46
query33	2.99	3.05	3.06
query34	15.82	5.17	4.50
query35	4.55	4.57	4.48
query36	0.67	0.49	0.48
query37	0.09	0.07	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.02	0.03
Total cold run time: 104.84 s
Total hot run time: 31.06 s

@jacktengg
jacktengg force-pushed the be-core-ut-datatypes branch 2 times, most recently from 5ec7ffc to 5365220 Compare March 7, 2025 02:19
@jacktengg

jacktengg commented Mar 7, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Mar 7, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 32049 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5365220cc29e8d62c94e580ce33ce11e27fdd3a8, data reload: false

------ Round 1 ----------------------------------
q1	17636	5293	5053	5053
q2	2049	293	163	163
q3	10415	1255	739	739
q4	10200	1036	535	535
q5	7784	2403	2305	2305
q6	187	193	140	140
q7	902	789	637	637
q8	9329	1316	1034	1034
q9	4945	4508	4617	4508
q10	6803	2283	1877	1877
q11	475	275	253	253
q12	344	352	217	217
q13	17753	3704	3081	3081
q14	223	235	222	222
q15	523	488	478	478
q16	636	613	581	581
q17	583	832	363	363
q18	7128	6407	6225	6225
q19	1664	950	533	533
q20	309	327	193	193
q21	2859	2117	1926	1926
q22	1093	1017	986	986
Total cold run time: 103840 ms
Total hot run time: 32049 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5221	5135	5202	5135
q2	235	342	226	226
q3	2169	2671	2274	2274
q4	1434	1892	1382	1382
q5	4244	4094	4121	4094
q6	210	163	124	124
q7	1838	1850	1740	1740
q8	2645	2616	2502	2502
q9	7208	7257	7177	7177
q10	3005	3216	2763	2763
q11	567	511	472	472
q12	683	793	602	602
q13	3421	3897	3319	3319
q14	306	303	262	262
q15	528	476	477	476
q16	639	659	627	627
q17	1130	1621	1328	1328
q18	7689	7692	7426	7426
q19	838	772	877	772
q20	2061	2015	1876	1876
q21	5522	4927	4601	4601
q22	1087	1112	1049	1049
Total cold run time: 52680 ms
Total hot run time: 50227 ms

@doris-robot

doris-robot commented Mar 7, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 191794 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5365220cc29e8d62c94e580ce33ce11e27fdd3a8, data reload: false

query1	1391	989	980	980
query2	6196	1910	1924	1910
query3	11102	4557	4560	4557
query4	52740	25637	23129	23129
query5	5042	633	485	485
query6	337	210	191	191
query7	4874	504	288	288
query8	315	225	227	225
query9	5630	2552	2565	2552
query10	424	313	266	266
query11	15173	15049	14964	14964
query12	160	104	112	104
query13	1041	496	376	376
query14	10156	6296	6901	6296
query15	213	205	187	187
query16	7085	687	430	430
query17	1086	733	567	567
query18	1560	403	303	303
query19	200	187	152	152
query20	127	117	124	117
query21	202	129	122	122
query22	4311	4440	4443	4440
query23	34044	33258	33238	33238
query24	5643	2419	2437	2419
query25	504	478	419	419
query26	696	285	166	166
query27	1938	494	340	340
query28	2757	2471	2459	2459
query29	608	596	445	445
query30	277	224	192	192
query31	852	854	807	807
query32	78	67	64	64
query33	485	382	320	320
query34	767	856	512	512
query35	816	855	777	777
query36	954	995	903	903
query37	123	110	81	81
query38	4259	4309	4285	4285
query39	1485	1471	1447	1447
query40	207	114	98	98
query41	51	54	51	51
query42	123	110	101	101
query43	495	512	488	488
query44	1312	819	790	790
query45	180	173	173	173
query46	868	1037	642	642
query47	1840	1916	1763	1763
query48	376	414	303	303
query49	743	504	423	423
query50	716	822	422	422
query51	4377	4292	4310	4292
query52	107	111	95	95
query53	231	257	188	188
query54	485	512	416	416
query55	85	80	84	80
query56	255	259	259	259
query57	1145	1182	1134	1134
query58	242	253	242	242
query59	2662	2801	2805	2801
query60	295	275	262	262
query61	127	120	150	120
query62	759	733	683	683
query63	227	199	205	199
query64	1683	1031	775	775
query65	4565	4444	4496	4444
query66	760	403	284	284
query67	15741	15442	15290	15290
query68	6781	878	503	503
query69	530	294	262	262
query70	1229	1140	1111	1111
query71	503	298	268	268
query72	5578	3597	3769	3597
query73	1483	736	348	348
query74	9342	9108	8725	8725
query75	4046	3161	2710	2710
query76	4300	1182	748	748
query77	758	365	272	272
query78	10019	9995	9262	9262
query79	5837	811	576	576
query80	714	527	453	453
query81	495	254	221	221
query82	758	127	92	92
query83	218	176	153	153
query84	285	95	139	95
query85	762	346	301	301
query86	330	298	289	289
query87	4456	4388	4459	4388
query88	2836	2183	2232	2183
query89	460	321	278	278
query90	1986	203	204	203
query91	137	151	109	109
query92	78	63	56	56
query93	3063	1050	578	578
query94	676	411	381	381
query95	344	274	258	258
query96	497	552	278	278
query97	3347	3359	3273	3273
query98	218	209	204	204
query99	1436	1403	1286	1286
Total cold run time: 299810 ms
Total hot run time: 191794 ms

@doris-robot

doris-robot commented Mar 7, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.25 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5365220cc29e8d62c94e580ce33ce11e27fdd3a8, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.23	0.07	0.07
query4	1.62	0.10	0.10
query5	0.55	0.54	0.56
query6	1.20	0.71	0.71
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.54	0.52
query10	0.57	0.62	0.59
query11	0.16	0.11	0.10
query12	0.14	0.11	0.11
query13	0.62	0.61	0.60
query14	2.82	2.68	2.82
query15	0.92	0.86	0.86
query16	0.39	0.37	0.37
query17	1.05	1.04	1.04
query18	0.22	0.19	0.19
query19	1.97	1.80	1.92
query20	0.02	0.01	0.01
query21	15.35	0.91	0.57
query22	0.76	1.19	0.96
query23	14.72	1.37	0.60
query24	6.58	1.64	0.85
query25	0.51	0.24	0.12
query26	0.61	0.15	0.14
query27	0.05	0.06	0.05
query28	9.40	0.91	0.45
query29	12.54	4.07	3.33
query30	0.25	0.09	0.07
query31	2.83	0.57	0.38
query32	3.22	0.55	0.46
query33	3.06	3.01	2.98
query34	15.68	5.13	4.55
query35	4.56	4.60	4.54
query36	0.66	0.51	0.47
query37	0.08	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.03	0.03
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.02	0.03
Total cold run time: 104.47 s
Total hot run time: 31.25 s

@jacktengg
jacktengg force-pushed the be-core-ut-datatypes branch 2 times, most recently from 0a68e98 to 92e0daa Compare March 7, 2025 07:09
@jacktengg

jacktengg commented Mar 7, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Mar 7, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 32985 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 92e0daa7acab10d7c27f5a1d311b4ad1899d306d, data reload: false

------ Round 1 ----------------------------------
q1	17569	5290	5148	5148
q2	2054	311	171	171
q3	10476	1295	768	768
q4	10226	1025	535	535
q5	7599	2412	2374	2374
q6	198	177	134	134
q7	915	763	621	621
q8	9319	1305	1122	1122
q9	5019	4767	4864	4767
q10	6828	2332	1889	1889
q11	498	279	261	261
q12	350	352	213	213
q13	17764	3699	3138	3138
q14	244	219	216	216
q15	537	495	477	477
q16	624	607	605	605
q17	591	866	344	344
q18	6815	6437	6433	6433
q19	2051	972	575	575
q20	306	317	187	187
q21	2748	2156	2034	2034
q22	1061	992	973	973
Total cold run time: 103792 ms
Total hot run time: 32985 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5322	5267	5171	5171
q2	245	340	234	234
q3	2172	2653	2292	2292
q4	1387	1843	1352	1352
q5	4232	4142	4130	4130
q6	217	165	125	125
q7	1937	1917	1751	1751
q8	2672	2666	2558	2558
q9	7315	7207	7221	7207
q10	3062	3230	2777	2777
q11	578	508	502	502
q12	688	744	566	566
q13	3335	4027	3280	3280
q14	284	316	275	275
q15	535	473	483	473
q16	637	691	641	641
q17	1130	1554	1412	1412
q18	7773	7651	7498	7498
q19	825	807	872	807
q20	2000	2061	1882	1882
q21	5426	5099	4657	4657
q22	1084	1014	1011	1011
Total cold run time: 52856 ms
Total hot run time: 50601 ms

@doris-robot

doris-robot commented Mar 7, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 192504 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 92e0daa7acab10d7c27f5a1d311b4ad1899d306d, data reload: false

query1	1372	1027	987	987
query2	6157	1928	1951	1928
query3	11059	4496	4708	4496
query4	54112	25654	23478	23478
query5	4824	628	506	506
query6	324	215	189	189
query7	4872	520	289	289
query8	314	247	242	242
query9	5375	2505	2536	2505
query10	459	323	252	252
query11	15132	14935	14892	14892
query12	166	110	113	110
query13	1041	504	399	399
query14	10425	6309	6903	6309
query15	201	187	193	187
query16	7132	667	498	498
query17	1087	725	572	572
query18	1584	426	317	317
query19	195	188	169	169
query20	123	129	119	119
query21	209	122	106	106
query22	4492	4448	4412	4412
query23	33842	33414	33259	33259
query24	5762	2421	2468	2421
query25	455	478	396	396
query26	696	279	154	154
query27	1892	484	332	332
query28	2807	2428	2420	2420
query29	589	585	427	427
query30	278	219	188	188
query31	870	866	789	789
query32	69	66	61	61
query33	428	352	298	298
query34	760	850	490	490
query35	807	835	753	753
query36	915	1001	899	899
query37	122	99	80	80
query38	4211	4179	4164	4164
query39	1515	1487	1452	1452
query40	209	120	104	104
query41	56	52	54	52
query42	123	103	104	103
query43	522	513	480	480
query44	1284	802	790	790
query45	179	168	166	166
query46	857	1058	643	643
query47	1887	1910	1807	1807
query48	375	417	322	322
query49	691	506	433	433
query50	716	759	432	432
query51	4261	4353	4231	4231
query52	111	105	98	98
query53	231	263	191	191
query54	489	523	419	419
query55	86	82	81	81
query56	264	278	265	265
query57	1156	1172	1111	1111
query58	252	235	226	226
query59	2833	3000	2936	2936
query60	296	279	272	272
query61	119	120	120	120
query62	769	736	688	688
query63	231	192	188	188
query64	1798	1038	727	727
query65	4621	4415	4326	4326
query66	697	392	355	355
query67	15954	15525	15431	15431
query68	6885	873	501	501
query69	537	296	262	262
query70	1157	1096	1121	1096
query71	461	285	267	267
query72	5901	3705	3759	3705
query73	1280	745	350	350
query74	9054	9118	9234	9118
query75	3636	3137	2771	2771
query76	4130	1188	731	731
query77	630	379	284	284
query78	10070	10031	9274	9274
query79	2673	821	593	593
query80	599	516	491	491
query81	490	265	218	218
query82	584	132	93	93
query83	172	169	151	151
query84	295	92	86	86
query85	760	354	307	307
query86	353	302	275	275
query87	4526	4478	4408	4408
query88	3440	2175	2165	2165
query89	416	314	281	281
query90	1754	206	205	205
query91	137	139	105	105
query92	78	61	60	60
query93	1927	1067	579	579
query94	720	411	293	293
query95	347	258	257	257
query96	486	574	269	269
query97	3343	3437	3307	3307
query98	226	204	199	199
query99	1370	1393	1261	1261
Total cold run time: 296177 ms
Total hot run time: 192504 ms

@doris-robot

doris-robot commented Mar 7, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.05 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 92e0daa7acab10d7c27f5a1d311b4ad1899d306d, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.24	0.06	0.07
query4	1.61	0.10	0.11
query5	0.57	0.54	0.55
query6	1.20	0.73	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.54	0.52
query10	0.59	0.58	0.57
query11	0.16	0.11	0.10
query12	0.14	0.11	0.12
query13	0.62	0.60	0.60
query14	2.79	2.82	2.82
query15	0.93	0.85	0.85
query16	0.38	0.38	0.36
query17	1.06	1.03	1.01
query18	0.21	0.20	0.19
query19	1.93	1.81	1.95
query20	0.02	0.01	0.01
query21	15.35	0.92	0.55
query22	0.76	1.40	0.74
query23	14.71	1.41	0.66
query24	7.06	1.71	0.74
query25	0.49	0.14	0.16
query26	0.55	0.16	0.14
query27	0.05	0.06	0.04
query28	9.76	0.84	0.42
query29	12.57	3.97	3.31
query30	0.26	0.09	0.06
query31	2.81	0.58	0.38
query32	3.24	0.54	0.47
query33	3.00	3.04	3.04
query34	15.67	5.13	4.52
query35	4.49	4.58	4.54
query36	0.66	0.49	0.49
query37	0.09	0.07	0.07
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.14	0.13
query41	0.08	0.02	0.02
query42	0.04	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 105.11 s
Total hot run time: 31.05 s

@jacktengg
jacktengg force-pushed the be-core-ut-datatypes branch 2 times, most recently from 5bbe8aa to 5693f00 Compare March 8, 2025 01:29
@jacktengg

jacktengg commented Mar 8, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Mar 8, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 32820 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5693f0069c88748f4fff510517e6f62cc5c31600, data reload: false

------ Round 1 ----------------------------------
q1	17584	5247	5145	5145
q2	2050	300	169	169
q3	10627	1274	765	765
q4	10206	1051	531	531
q5	7540	2390	2465	2390
q6	190	174	138	138
q7	921	755	645	645
q8	9302	1345	1153	1153
q9	5055	4656	4698	4656
q10	6836	2335	1894	1894
q11	471	285	272	272
q12	344	362	220	220
q13	17770	3671	3115	3115
q14	228	234	214	214
q15	541	507	487	487
q16	635	637	580	580
q17	580	882	352	352
q18	7032	6446	6376	6376
q19	1226	983	551	551
q20	328	351	199	199
q21	2887	2247	1978	1978
q22	1092	1029	990	990
Total cold run time: 103445 ms
Total hot run time: 32820 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5182	5180	5185	5180
q2	242	338	232	232
q3	2187	2708	2300	2300
q4	1450	1875	1367	1367
q5	4294	4154	4175	4154
q6	221	177	128	128
q7	1937	1964	1781	1781
q8	2687	2714	2586	2586
q9	7304	7229	7170	7170
q10	3043	3290	2767	2767
q11	573	512	485	485
q12	712	768	558	558
q13	3377	3905	3305	3305
q14	289	289	270	270
q15	541	503	486	486
q16	663	712	637	637
q17	1167	1571	1376	1376
q18	7944	7719	7508	7508
q19	843	850	922	850
q20	1986	2077	1864	1864
q21	5519	5003	4731	4731
q22	1069	1046	1022	1022
Total cold run time: 53230 ms
Total hot run time: 50757 ms

@doris-robot

doris-robot commented Mar 8, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 191877 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5693f0069c88748f4fff510517e6f62cc5c31600, data reload: false

query1	1372	1003	990	990
query2	6243	1842	1851	1842
query3	10999	4502	4569	4502
query4	53750	25279	23554	23554
query5	5274	558	506	506
query6	400	218	192	192
query7	5192	512	295	295
query8	325	247	239	239
query9	6838	2527	2539	2527
query10	420	326	261	261
query11	15508	15117	14838	14838
query12	158	114	109	109
query13	1232	513	399	399
query14	10610	6634	6442	6442
query15	205	191	189	189
query16	7086	664	469	469
query17	1136	710	569	569
query18	1501	422	316	316
query19	188	197	174	174
query20	127	126	121	121
query21	214	122	101	101
query22	4424	4851	4520	4520
query23	34286	33310	33518	33310
query24	5990	2455	2434	2434
query25	457	474	399	399
query26	718	279	156	156
query27	1865	498	343	343
query28	2815	2459	2410	2410
query29	573	587	425	425
query30	274	215	197	197
query31	878	862	818	818
query32	70	63	59	59
query33	474	379	303	303
query34	741	864	503	503
query35	816	840	745	745
query36	938	995	909	909
query37	119	101	73	73
query38	4485	4241	4170	4170
query39	1475	1436	1424	1424
query40	208	118	105	105
query41	55	51	54	51
query42	127	106	108	106
query43	500	490	480	480
query44	1296	815	819	815
query45	184	173	170	170
query46	854	1042	667	667
query47	1841	1887	1866	1866
query48	389	424	313	313
query49	725	502	435	435
query50	696	764	424	424
query51	4341	4296	4285	4285
query52	111	104	96	96
query53	234	255	187	187
query54	494	511	427	427
query55	88	85	79	79
query56	282	270	254	254
query57	1167	1211	1116	1116
query58	257	242	245	242
query59	2656	2834	2627	2627
query60	316	281	290	281
query61	128	115	117	115
query62	748	733	699	699
query63	235	197	188	188
query64	1692	1053	681	681
query65	4510	4437	4418	4418
query66	723	386	317	317
query67	15814	15462	15130	15130
query68	5948	881	504	504
query69	534	294	269	269
query70	1178	1080	1120	1080
query71	441	292	275	275
query72	5922	3713	3771	3713
query73	1076	746	340	340
query74	9174	9165	8739	8739
query75	3216	3180	2731	2731
query76	3837	1191	746	746
query77	550	373	274	274
query78	10104	10159	9193	9193
query79	1945	832	578	578
query80	696	526	438	438
query81	499	257	220	220
query82	428	127	94	94
query83	173	170	151	151
query84	291	96	75	75
query85	845	343	300	300
query86	367	296	312	296
query87	4406	4587	4337	4337
query88	3248	2222	2240	2222
query89	393	309	283	283
query90	1780	214	208	208
query91	134	138	108	108
query92	67	64	58	58
query93	2762	1041	591	591
query94	680	390	303	303
query95	336	273	263	263
query96	476	570	280	280
query97	3324	3342	3339	3339
query98	236	217	195	195
query99	1348	1376	1267	1267
Total cold run time: 297261 ms
Total hot run time: 191877 ms

@doris-robot

doris-robot commented Mar 8, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.38 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5693f0069c88748f4fff510517e6f62cc5c31600, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.24	0.06	0.06
query4	1.63	0.11	0.10
query5	0.56	0.55	0.55
query6	1.19	0.71	0.73
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.60	0.50	0.52
query10	0.57	0.62	0.59
query11	0.16	0.12	0.11
query12	0.15	0.12	0.11
query13	0.62	0.61	0.61
query14	2.83	2.80	2.72
query15	0.94	0.85	0.83
query16	0.38	0.37	0.38
query17	1.10	1.05	1.05
query18	0.21	0.19	0.19
query19	1.92	1.82	1.98
query20	0.02	0.01	0.01
query21	15.35	0.91	0.56
query22	0.76	1.27	0.70
query23	14.77	1.36	0.63
query24	6.86	1.71	1.38
query25	0.50	0.34	0.06
query26	0.58	0.16	0.14
query27	0.04	0.04	0.05
query28	10.09	0.81	0.45
query29	12.55	3.94	3.27
query30	0.26	0.09	0.06
query31	2.81	0.58	0.39
query32	3.23	0.56	0.47
query33	3.00	3.00	3.10
query34	15.91	5.12	4.48
query35	4.52	4.55	4.51
query36	0.68	0.50	0.48
query37	0.08	0.06	0.07
query38	0.05	0.03	0.03
query39	0.03	0.02	0.03
query40	0.18	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.69 s
Total hot run time: 31.38 s

@doris-robot

doris-robot commented Mar 8, 2025

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 63.89% (46/72) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 47.64% (12732/26725)
Line Coverage 36.98% (108352/293030)
Region Coverage 36.05% (55415/153705)
Branch Coverage 31.29% (27846/88980)

@jacktengg
jacktengg force-pushed the be-core-ut-datatypes branch 2 times, most recently from b46a1ee to 31f8cb7 Compare March 9, 2025 17:12
@jacktengg

jacktengg commented Mar 9, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Mar 9, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 32865 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 31f8cb770bd6a78fd3a582dd194777011d33895b, data reload: false

------ Round 1 ----------------------------------
q1	17585	5208	5110	5110
q2	2052	291	171	171
q3	10427	1236	728	728
q4	10251	1037	531	531
q5	7820	2375	2356	2356
q6	183	167	132	132
q7	884	733	610	610
q8	9284	1262	1113	1113
q9	4900	4635	4679	4635
q10	6813	2299	1893	1893
q11	474	273	258	258
q12	342	350	220	220
q13	17776	3633	3133	3133
q14	236	227	214	214
q15	537	494	489	489
q16	636	618	585	585
q17	579	889	353	353
q18	7071	6748	6659	6659
q19	1401	963	559	559
q20	329	336	196	196
q21	2983	2138	1949	1949
q22	1070	1024	971	971
Total cold run time: 103633 ms
Total hot run time: 32865 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5156	5106	5108	5106
q2	240	331	234	234
q3	2097	2683	2300	2300
q4	1428	1901	1384	1384
q5	4246	4098	4145	4098
q6	207	166	124	124
q7	1927	1918	1759	1759
q8	2707	2644	2555	2555
q9	7265	7322	7154	7154
q10	3046	3241	2764	2764
q11	572	506	491	491
q12	704	741	573	573
q13	3479	3873	3259	3259
q14	277	298	281	281
q15	529	505	473	473
q16	641	681	640	640
q17	1123	1544	1346	1346
q18	7850	7666	7587	7587
q19	777	816	947	816
q20	1971	2007	1905	1905
q21	5361	5039	4716	4716
q22	1136	1088	1042	1042
Total cold run time: 52739 ms
Total hot run time: 50607 ms

@doris-robot

doris-robot commented Mar 9, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 192012 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 31f8cb770bd6a78fd3a582dd194777011d33895b, data reload: false

query1	1375	1034	1020	1020
query2	6332	1840	1889	1840
query3	11162	4600	4531	4531
query4	53912	26518	22933	22933
query5	5119	561	497	497
query6	341	202	213	202
query7	4881	497	289	289
query8	302	244	244	244
query9	5376	2564	2548	2548
query10	418	309	261	261
query11	15095	15112	14931	14931
query12	155	113	110	110
query13	1044	514	397	397
query14	11099	7017	6654	6654
query15	214	203	183	183
query16	7163	687	481	481
query17	1096	746	614	614
query18	1548	423	324	324
query19	198	198	173	173
query20	133	131	120	120
query21	210	122	108	108
query22	4382	4499	4295	4295
query23	34089	33269	33337	33269
query24	5765	2435	2427	2427
query25	474	477	411	411
query26	718	283	157	157
query27	1798	503	333	333
query28	2834	2493	2439	2439
query29	581	568	432	432
query30	269	220	201	201
query31	872	887	796	796
query32	71	62	66	62
query33	466	338	307	307
query34	755	870	520	520
query35	778	842	752	752
query36	954	988	906	906
query37	115	100	76	76
query38	4092	4239	4204	4204
query39	1521	1461	1416	1416
query40	226	118	107	107
query41	62	53	53	53
query42	133	107	105	105
query43	494	507	488	488
query44	1326	826	812	812
query45	183	178	175	175
query46	857	1038	666	666
query47	1876	1879	1766	1766
query48	386	424	319	319
query49	703	520	432	432
query50	707	751	429	429
query51	4318	4344	4231	4231
query52	103	109	88	88
query53	225	267	186	186
query54	473	499	429	429
query55	88	84	85	84
query56	257	270	264	264
query57	1129	1159	1139	1139
query58	256	233	237	233
query59	2640	2836	2720	2720
query60	286	274	263	263
query61	120	123	158	123
query62	727	773	685	685
query63	228	186	188	186
query64	1805	1057	733	733
query65	4526	4456	4456	4456
query66	713	416	302	302
query67	15814	15534	15373	15373
query68	7492	878	505	505
query69	536	301	267	267
query70	1220	1124	1098	1098
query71	486	292	264	264
query72	5940	3724	3737	3724
query73	1528	741	345	345
query74	8995	8844	8771	8771
query75	3803	3140	2715	2715
query76	4150	1187	747	747
query77	617	355	282	282
query78	10041	10185	9239	9239
query79	2348	814	595	595
query80	631	509	444	444
query81	478	253	218	218
query82	656	126	95	95
query83	176	168	153	153
query84	283	89	69	69
query85	771	353	301	301
query86	363	279	290	279
query87	4525	4717	4507	4507
query88	3575	2228	2206	2206
query89	408	315	280	280
query90	1919	210	208	208
query91	133	142	105	105
query92	77	61	61	61
query93	1342	1089	586	586
query94	663	440	391	391
query95	352	266	254	254
query96	484	565	273	273
query97	3312	3396	3243	3243
query98	239	205	199	199
query99	1416	1399	1278	1278
Total cold run time: 297327 ms
Total hot run time: 192012 ms

@doris-robot

doris-robot commented Mar 9, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.71 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 31f8cb770bd6a78fd3a582dd194777011d33895b, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.03
query3	0.23	0.06	0.06
query4	1.62	0.11	0.10
query5	0.56	0.53	0.55
query6	1.19	0.72	0.71
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.60	0.52	0.52
query10	0.57	0.63	0.58
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.62	0.61	0.60
query14	2.80	2.83	2.82
query15	0.93	0.85	0.84
query16	0.36	0.38	0.38
query17	1.01	1.01	1.01
query18	0.21	0.20	0.19
query19	1.88	1.87	1.94
query20	0.01	0.02	0.01
query21	15.36	0.88	0.57
query22	0.75	1.10	0.80
query23	14.83	1.39	0.68
query24	6.66	2.79	0.42
query25	0.30	0.19	0.11
query26	0.60	0.16	0.13
query27	0.06	0.05	0.04
query28	9.42	0.87	0.43
query29	12.57	3.85	3.25
query30	0.24	0.08	0.06
query31	2.84	0.59	0.38
query32	3.22	0.55	0.46
query33	2.98	3.06	3.01
query34	15.77	5.20	4.50
query35	4.50	4.56	4.54
query36	0.67	0.51	0.48
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.14	0.13
query41	0.09	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 104.29 s
Total hot run time: 30.71 s

@doris-robot

doris-robot commented Mar 9, 2025

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 62.67% (47/75) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 47.69% (12746/26725)
Line Coverage 37.02% (108468/293030)
Region Coverage 36.08% (55454/153705)
Branch Coverage 31.32% (27867/88980)

@jacktengg jacktengg closed this Mar 10, 2025
@jacktengg jacktengg reopened this Mar 10, 2025
@jacktengg jacktengg mentioned this pull request Mar 10, 2025
Closed
3 of 16 tasks
@jacktengg
jacktengg force-pushed the be-core-ut-datatypes branch from 31f8cb7 to 3331ed4 Compare March 10, 2025 01:55
@jacktengg

jacktengg commented Mar 10, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Mar 10, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 32973 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3331ed4b441352429574cda8672246bbff032837, data reload: false

------ Round 1 ----------------------------------
q1	17637	5263	5084	5084
q2	2061	298	167	167
q3	10410	1334	766	766
q4	10248	1039	541	541
q5	7771	2316	2415	2316
q6	207	171	138	138
q7	896	765	633	633
q8	9303	1301	1068	1068
q9	4971	4911	4871	4871
q10	6830	2335	1924	1924
q11	498	285	263	263
q12	355	351	231	231
q13	17782	3711	3084	3084
q14	242	230	212	212
q15	538	499	494	494
q16	643	627	603	603
q17	576	874	359	359
q18	6798	6473	6471	6471
q19	1685	975	543	543
q20	316	315	190	190
q21	2930	2220	2011	2011
q22	1082	1014	1004	1004
Total cold run time: 103779 ms
Total hot run time: 32973 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5207	5128	5172	5128
q2	236	323	230	230
q3	2150	2690	2329	2329
q4	1450	1795	1362	1362
q5	4256	4148	4126	4126
q6	211	172	124	124
q7	1965	1913	1807	1807
q8	2635	2564	2589	2564
q9	7151	7288	7108	7108
q10	3031	3276	2790	2790
q11	570	512	499	499
q12	733	745	596	596
q13	3483	3919	3285	3285
q14	292	298	280	280
q15	518	490	485	485
q16	655	676	649	649
q17	1141	1595	1387	1387
q18	7767	7562	7467	7467
q19	818	829	916	829
q20	2024	2022	1886	1886
q21	5540	4953	4821	4821
q22	1103	1082	1041	1041
Total cold run time: 52936 ms
Total hot run time: 50793 ms

@doris-robot

doris-robot commented Mar 10, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 191991 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3331ed4b441352429574cda8672246bbff032837, data reload: false

query1	1386	1030	1019	1019
query2	6227	1949	1887	1887
query3	11005	4506	4437	4437
query4	53269	25367	22997	22997
query5	5007	601	497	497
query6	335	199	199	199
query7	4909	515	297	297
query8	316	240	236	236
query9	5680	2541	2567	2541
query10	417	357	276	276
query11	15286	15086	14992	14992
query12	163	111	106	106
query13	1062	525	402	402
query14	10655	6554	6549	6549
query15	202	192	182	182
query16	7065	675	477	477
query17	1046	713	594	594
query18	1520	410	306	306
query19	196	197	170	170
query20	130	128	126	126
query21	206	126	110	110
query22	4443	4547	4510	4510
query23	34054	33316	33271	33271
query24	5889	2414	2433	2414
query25	453	480	393	393
query26	724	280	157	157
query27	2038	485	356	356
query28	2750	2463	2454	2454
query29	571	569	438	438
query30	276	227	190	190
query31	881	865	793	793
query32	73	62	66	62
query33	452	349	308	308
query34	762	856	508	508
query35	806	831	751	751
query36	960	996	881	881
query37	127	100	75	75
query38	4317	4242	4073	4073
query39	1508	1461	1489	1461
query40	212	129	111	111
query41	58	65	59	59
query42	135	115	112	112
query43	509	508	503	503
query44	1343	811	821	811
query45	182	180	172	172
query46	870	1056	666	666
query47	1926	1890	1807	1807
query48	393	421	319	319
query49	717	591	409	409
query50	733	764	425	425
query51	4287	4318	4277	4277
query52	108	107	100	100
query53	245	262	192	192
query54	503	512	423	423
query55	84	84	89	84
query56	272	280	263	263
query57	1184	1186	1151	1151
query58	248	243	234	234
query59	2496	2868	2849	2849
query60	297	271	261	261
query61	121	120	124	120
query62	781	740	684	684
query63	226	187	190	187
query64	1865	1032	694	694
query65	4559	4479	4540	4479
query66	770	397	313	313
query67	16028	15507	15177	15177
query68	7272	880	508	508
query69	544	307	269	269
query70	1167	1134	1133	1133
query71	493	314	286	286
query72	5955	3550	3821	3550
query73	1466	740	350	350
query74	9329	9144	8966	8966
query75	3891	3169	2721	2721
query76	4149	1217	764	764
query77	656	359	286	286
query78	10312	10328	9238	9238
query79	2111	874	591	591
query80	604	514	458	458
query81	479	257	220	220
query82	279	127	97	97
query83	172	179	150	150
query84	298	97	71	71
query85	936	364	307	307
query86	373	293	280	280
query87	4435	4552	4409	4409
query88	3741	2201	2198	2198
query89	417	316	290	290
query90	1780	209	207	207
query91	142	136	108	108
query92	73	59	58	58
query93	1921	1061	581	581
query94	633	416	269	269
query95	345	280	264	264
query96	479	592	276	276
query97	3338	3417	3265	3265
query98	230	203	198	198
query99	1340	1459	1246	1246
Total cold run time: 297678 ms
Total hot run time: 191991 ms

@doris-robot

doris-robot commented Mar 10, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.96 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3331ed4b441352429574cda8672246bbff032837, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.03	0.03
query3	0.23	0.06	0.07
query4	1.63	0.11	0.11
query5	0.57	0.54	0.54
query6	1.20	0.72	0.73
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.52	0.52
query10	0.59	0.60	0.58
query11	0.16	0.11	0.10
query12	0.14	0.12	0.11
query13	0.63	0.61	0.60
query14	2.64	2.68	2.66
query15	0.92	0.86	0.87
query16	0.38	0.38	0.38
query17	1.02	1.02	1.03
query18	0.20	0.20	0.19
query19	1.89	1.80	2.00
query20	0.01	0.01	0.01
query21	15.38	0.90	0.55
query22	0.75	1.13	0.62
query23	15.04	1.41	0.63
query24	6.81	1.80	1.02
query25	0.48	0.14	0.11
query26	0.66	0.16	0.13
query27	0.05	0.06	0.05
query28	9.46	0.83	0.44
query29	12.53	3.95	3.30
query30	0.25	0.09	0.07
query31	2.82	0.59	0.39
query32	3.23	0.55	0.47
query33	3.03	3.00	2.99
query34	15.60	5.12	4.54
query35	4.58	4.49	4.48
query36	0.68	0.50	0.48
query37	0.10	0.07	0.07
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.17	0.13	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.83 s
Total hot run time: 30.96 s

@doris-robot

doris-robot commented Mar 10, 2025

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 62.50% (40/64) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 47.70% (12748/26723)
Line Coverage 37.03% (108491/293013)
Region Coverage 36.09% (55464/153700)
Branch Coverage 31.33% (27873/88978)

yiguolei
yiguolei approved these changes Mar 11, 2025
View reviewed changes

@yiguolei yiguolei left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Mar 11, 2025
@github-actions

github-actions Bot commented Mar 11, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actions Bot commented Mar 11, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

RoanHeNaN
RoanHeNaN approved these changes Mar 11, 2025
View reviewed changes

@RoanHeNaN RoanHeNaN left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@hello-stephen
hello-stephen merged commit 9855f79 into apache:master Mar 12, 2025
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@jacktengg koarz
[ut](datatype) Add BE UT for some basic data types and serializers (a…
d92c1c7 
…pache#48788)

### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
DataTypeNumber, DataTypeDecimal, DataTypeString, DataTypeDateTimeV2
@morningman morningman mentioned this pull request Oct 27, 2026
Merged
3 of 12 tasks
morningman added a commit that referenced this pull request Oct 29, 2026
@morningman
[test](be) Drop the std::set::end() dereference in load_data_from_csv (…
49b0bee 
…#69695)

### What problem does this PR solve?

Issue Number: None

Related PR: #48788

Problem Summary:

**In short.** `load_data_from_csv()` in the BE test utilities
dereferences `std::set::end()` in an assertion. With libc++ (macOS) that
reads past the set on the stack, so under ASAN every BE unit test suite
that loads a CSV fixture aborts while loading its fixtures and takes the
rest of the `doris_be_test` run with it. This removes the assertion; it
only repeated the size check on the line above.

**Background.** #48788 added `load_data_from_csv()`
(`be/test/testutil/test_util.cpp`), which the column, data type and
SerDe unit tests use to load their CSV fixtures. It checks that the
SerDes, the columns and the requested column indexes line up:
`serders.size() == columns.size()`, `serders.size() == idxes.size()`,
and then `serders.size() == *idxes.end()`.

**The problem, and what it cost.** Dereferencing the end iterator of a
`std::set` is undefined behavior. In libstdc++ the end iterator points
at the tree header, whose value slot overlaps the element count, so on
Linux the last assertion compares against `idxes.size()` again and
passes. In libc++ it reads outside the set object: ASAN reports a
stack-buffer-overflow, and with `abort_on_error` the whole
`doris_be_test` process stops. The suites of all 24 test files that load
CSV fixtures (`DataTypeDateTimeV2SerDeTest`, `DataTypeNumberSerDeTest`,
`ColumnStringTest`, ...) cannot run on macOS, and a test filter that
includes one of them ends the run there. CI runs on Linux only, so it
never failed there.

**How this PR fixes it.** Remove the assertion. The size check right
above it already guarantees one index per SerDe. No production code
changes. #63474, open since May, carries the same removal among its
macOS build fixes; this PR takes only this line, so the test fix does
not wait for the rest.

**Results.**

| | macOS arm64, ASAN | Linux |
|---|---|---|
| Before | `DataTypeDateTimeV2SerDeTest` aborts in `SetUpTestSuite` with
a stack-buffer-overflow | passes (the assertion repeats the size check)
|
| After | the 24 CSV-fixture suites (364 tests) pass | unchanged |
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei yiguolei approved these changes
+1 more reviewer
@RoanHeNaN RoanHeNaN RoanHeNaN approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@jacktengg @hello-stephen @doris-robot @yiguolei @RoanHeNaN