branch-3.1: [opt](iceberg) support insert to branch #52751 #53703 by morningman · Pull Request #53693 · apache/doris

morningman · 2025-07-22T05:10:51Z

bp #52751 #53703

Thearas · 2025-07-22T05:10:57Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

morningman · 2025-07-22T05:14:52Z

run buildall

doris-robot · 2025-07-22T05:34:31Z

TPC-H: Total hot run time: 32527 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 455199d5d1a483c112bed76be0c1c271baf4bd32, data reload: false

------ Round 1 ----------------------------------
q1	17682	5517	5349	5349
q2	2042	275	158	158
q3	10593	1255	771	771
q4	10295	870	457	457
q5	8930	2387	2188	2188
q6	186	165	134	134
q7	899	732	608	608
q8	9314	1446	1181	1181
q9	5177	4930	4944	4930
q10	6777	2272	1817	1817
q11	465	274	246	246
q12	336	360	206	206
q13	17769	3648	3051	3051
q14	227	223	204	204
q15	518	468	470	468
q16	422	436	375	375
q17	609	898	363	363
q18	7122	6435	6237	6237
q19	1205	953	576	576
q20	340	356	207	207
q21	3300	2221	2016	2016
q22	1079	1007	985	985
Total cold run time: 105287 ms
Total hot run time: 32527 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5520	5477	5468	5468
q2	236	344	244	244
q3	2249	2656	2268	2268
q4	1340	1809	1335	1335
q5	4434	4934	5036	4934
q6	174	167	130	130
q7	2062	1965	1875	1875
q8	2596	2834	2737	2737
q9	7306	7261	7267	7261
q10	2990	3324	2750	2750
q11	563	506	503	503
q12	668	763	632	632
q13	3410	3822	3127	3127
q14	269	313	280	280
q15	517	479	459	459
q16	450	500	440	440
q17	1231	1740	1265	1265
q18	7581	7458	7326	7326
q19	820	1073	1156	1073
q20	2005	2042	1862	1862
q21	5376	4935	4673	4673
q22	1082	1042	1002	1002
Total cold run time: 52879 ms
Total hot run time: 51644 ms

doris-robot · 2025-07-22T05:46:53Z

TPC-DS: Total hot run time: 198098 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 455199d5d1a483c112bed76be0c1c271baf4bd32, data reload: false

query1	1281	923	901	901
query2	6255	2017	1981	1981
query3	10989	4500	4558	4500
query4	32558	23898	23768	23768
query5	4142	630	462	462
query6	267	192	176	176
query7	3983	522	337	337
query8	305	247	246	246
query9	9444	2616	2577	2577
query10	489	344	263	263
query11	17902	15403	15243	15243
query12	167	109	106	106
query13	1555	567	425	425
query14	8939	7378	7920	7378
query15	226	213	195	195
query16	7983	707	463	463
query17	1534	783	600	600
query18	2140	422	323	323
query19	216	204	185	185
query20	124	118	116	116
query21	209	134	108	108
query22	4473	4468	4422	4422
query23	35243	34054	34059	34054
query24	7769	2735	2725	2725
query25	513	497	438	438
query26	1160	296	175	175
query27	2259	497	371	371
query28	5200	2273	2207	2207
query29	721	604	470	470
query30	250	213	169	169
query31	1000	922	872	872
query32	74	58	57	57
query33	504	386	316	316
query34	780	851	561	561
query35	793	823	754	754
query36	1023	1064	960	960
query37	105	97	81	81
query38	4018	4063	4024	4024
query39	1535	1453	1443	1443
query40	202	120	106	106
query41	55	48	47	47
query42	119	103	106	103
query43	529	529	509	509
query44	1446	862	849	849
query45	184	182	171	171
query46	949	1104	670	670
query47	1973	1951	1910	1910
query48	418	449	354	354
query49	762	505	390	390
query50	707	706	459	459
query51	7415	7314	7065	7065
query52	105	103	97	97
query53	242	270	205	205
query54	581	569	485	485
query55	86	80	77	77
query56	298	278	261	261
query57	1269	1275	1198	1198
query58	234	216	230	216
query59	3253	3275	3092	3092
query60	289	286	266	266
query61	114	113	166	113
query62	809	734	694	694
query63	239	200	221	200
query64	4164	998	630	630
query65	3398	3366	3291	3291
query66	1006	418	314	314
query67	16189	15803	15535	15535
query68	6600	866	554	554
query69	504	307	265	265
query70	1182	1089	1117	1089
query71	408	297	271	271
query72	5776	3774	3879	3774
query73	645	748	361	361
query74	10366	9086	9217	9086
query75	3171	3173	2690	2690
query76	3209	1194	806	806
query77	495	375	287	287
query78	10346	10412	9647	9647
query79	3533	941	615	615
query80	695	531	438	438
query81	514	256	220	220
query82	661	118	89	89
query83	165	159	146	146
query84	252	100	81	81
query85	777	366	302	302
query86	400	310	317	310
query87	4273	4387	4206	4206
query88	5199	2435	2414	2414
query89	422	335	297	297
query90	1744	192	192	192
query91	136	143	109	109
query92	69	56	54	54
query93	2246	926	553	553
query94	701	413	303	303
query95	365	282	272	272
query96	500	619	282	282
query97	3183	3318	3129	3129
query98	223	208	193	193
query99	1555	1440	1304	1304
Total cold run time: 295478 ms
Total hot run time: 198098 ms

doris-robot · 2025-07-22T05:52:08Z

ClickBench: Total hot run time: 28.41 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 455199d5d1a483c112bed76be0c1c271baf4bd32, data reload: false

query1	0.04	0.03	0.04
query2	0.06	0.03	0.03
query3	0.24	0.07	0.06
query4	1.62	0.11	0.11
query5	0.50	0.53	0.51
query6	1.13	0.72	0.73
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.56	0.50	0.49
query10	0.56	0.55	0.55
query11	0.14	0.10	0.11
query12	0.14	0.11	0.11
query13	0.60	0.59	0.60
query14	0.79	0.79	0.80
query15	0.85	0.83	0.82
query16	0.37	0.38	0.38
query17	1.04	0.99	1.01
query18	0.23	0.22	0.21
query19	1.93	1.84	1.77
query20	0.01	0.01	0.01
query21	15.38	0.93	0.58
query22	0.73	0.74	0.56
query23	15.29	1.42	0.60
query24	3.12	1.38	0.64
query25	0.23	0.17	0.20
query26	0.24	0.15	0.14
query27	0.06	0.05	0.05
query28	13.46	1.04	0.44
query29	12.60	3.92	3.25
query30	0.24	0.09	0.06
query31	2.86	0.60	0.38
query32	3.23	0.54	0.46
query33	2.94	2.98	3.06
query34	16.63	5.18	4.47
query35	4.55	4.61	4.55
query36	0.63	0.49	0.47
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.03
query40	0.16	0.12	0.12
query41	0.07	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 103.52 s
Total hot run time: 28.41 s

Related Issue: apache#48285 Problem Summary: support insert to branch, like : ```sql insert into tb@branch(br1); ```

morningman · 2025-07-22T07:33:36Z

run buildall

doris-robot · 2025-07-22T07:55:55Z

TPC-H: Total hot run time: 32558 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3b5dcf8e075c7255c74d513e2fba8c13e92d71af, data reload: false

------ Round 1 ----------------------------------
q1	17622	5409	5413	5409
q2	2044	302	165	165
q3	10629	1286	714	714
q4	10283	868	443	443
q5	9480	2390	2179	2179
q6	188	159	132	132
q7	897	755	635	635
q8	9335	1440	1109	1109
q9	5816	4861	4927	4861
q10	6827	2263	1840	1840
q11	474	287	282	282
q12	348	354	207	207
q13	17767	3593	3044	3044
q14	226	236	219	219
q15	517	476	487	476
q16	422	437	373	373
q17	597	866	363	363
q18	7142	6480	6361	6361
q19	1290	958	556	556
q20	356	345	214	214
q21	2990	2164	1980	1980
q22	1062	1047	996	996
Total cold run time: 106312 ms
Total hot run time: 32558 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5565	5461	5454	5454
q2	239	328	242	242
q3	2284	2580	2289	2289
q4	1368	1843	1392	1392
q5	4399	4984	5139	4984
q6	165	164	125	125
q7	2060	1921	1745	1745
q8	2604	2830	2638	2638
q9	7170	7148	7090	7090
q10	3028	3316	2784	2784
q11	582	500	499	499
q12	670	770	580	580
q13	3401	3741	3151	3151
q14	287	309	299	299
q15	513	474	462	462
q16	440	478	432	432
q17	1196	1739	1229	1229
q18	7564	7446	7217	7217
q19	800	1082	1093	1082
q20	1983	2020	1852	1852
q21	5296	4826	4634	4634
q22	1125	1107	1086	1086
Total cold run time: 52739 ms
Total hot run time: 51266 ms

doris-robot · 2025-07-22T08:08:01Z

TPC-DS: Total hot run time: 190540 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3b5dcf8e075c7255c74d513e2fba8c13e92d71af, data reload: false

query1	971	368	357	357
query2	6515	1984	1993	1984
query3	6717	221	222	221
query4	34194	23696	23414	23414
query5	4340	618	449	449
query6	281	201	169	169
query7	4621	510	317	317
query8	299	253	236	236
query9	9640	2610	2598	2598
query10	499	353	265	265
query11	18154	15478	15196	15196
query12	155	112	105	105
query13	1654	561	428	428
query14	9755	7894	6559	6559
query15	234	193	181	181
query16	8140	634	493	493
query17	1641	772	593	593
query18	2469	430	346	346
query19	232	188	172	172
query20	120	117	110	110
query21	216	140	109	109
query22	4643	4608	4354	4354
query23	34630	33978	33841	33841
query24	7277	2610	2656	2610
query25	503	472	412	412
query26	1098	260	170	170
query27	2028	472	345	345
query28	5060	2165	2117	2117
query29	594	567	460	460
query30	265	187	160	160
query31	979	894	803	803
query32	91	62	61	61
query33	511	370	318	318
query34	754	853	519	519
query35	746	814	716	716
query36	1011	1028	966	966
query37	106	96	75	75
query38	3886	3884	3818	3818
query39	1490	1479	1441	1441
query40	212	120	102	102
query41	52	51	55	51
query42	122	109	105	105
query43	521	512	490	490
query44	1413	818	822	818
query45	189	178	172	172
query46	887	1049	693	693
query47	1867	1905	1815	1815
query48	435	451	349	349
query49	800	501	448	448
query50	661	667	413	413
query51	7152	7149	7118	7118
query52	104	102	94	94
query53	231	253	186	186
query54	546	540	473	473
query55	86	78	77	77
query56	278	256	267	256
query57	1236	1224	1170	1170
query58	239	217	223	217
query59	3011	3115	2937	2937
query60	309	281	267	267
query61	120	114	112	112
query62	802	745	670	670
query63	227	190	189	189
query64	4542	992	672	672
query65	3293	3190	3188	3188
query66	1050	416	305	305
query67	15940	15671	15511	15511
query68	7731	845	538	538
query69	491	310	270	270
query70	1179	1124	1101	1101
query71	473	312	256	256
query72	5624	3801	3788	3788
query73	642	747	354	354
query74	10282	9173	8946	8946
query75	3200	3167	2658	2658
query76	3267	1188	798	798
query77	495	374	277	277
query78	10353	10343	9594	9594
query79	3510	895	596	596
query80	716	520	439	439
query81	505	262	216	216
query82	646	126	91	91
query83	165	169	145	145
query84	246	108	88	88
query85	797	379	304	304
query86	394	307	309	307
query87	4331	4305	4201	4201
query88	5334	2419	2388	2388
query89	408	327	303	303
query90	1845	193	192	192
query91	143	147	106	106
query92	68	55	53	53
query93	2072	907	554	554
query94	696	428	309	309
query95	343	279	260	260
query96	487	615	301	301
query97	3224	3311	3198	3198
query98	238	221	198	198
query99	1473	1398	1325	1325
Total cold run time: 294222 ms
Total hot run time: 190540 ms

doris-robot · 2025-07-22T08:13:17Z

ClickBench: Total hot run time: 29.1 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3b5dcf8e075c7255c74d513e2fba8c13e92d71af, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.04
query3	0.24	0.07	0.07
query4	1.61	0.10	0.10
query5	0.51	0.52	0.51
query6	1.13	0.73	0.72
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.56	0.51	0.51
query10	0.57	0.57	0.56
query11	0.15	0.11	0.11
query12	0.15	0.12	0.10
query13	0.62	0.60	0.59
query14	0.80	0.80	0.80
query15	0.86	0.84	0.83
query16	0.38	0.39	0.38
query17	1.05	1.02	1.06
query18	0.25	0.23	0.22
query19	1.96	1.82	1.78
query20	0.02	0.02	0.02
query21	15.40	0.90	0.54
query22	0.73	0.80	0.72
query23	15.04	1.39	0.50
query24	3.04	1.71	1.03
query25	0.18	0.24	0.19
query26	0.16	0.14	0.14
query27	0.07	0.05	0.05
query28	14.19	1.00	0.43
query29	12.60	4.00	3.29
query30	0.26	0.09	0.06
query31	2.83	0.59	0.39
query32	3.22	0.54	0.46
query33	2.95	3.00	3.03
query34	16.65	5.22	4.54
query35	4.54	4.56	4.63
query36	0.63	0.50	0.48
query37	0.09	0.06	0.05
query38	0.04	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.14	0.13
query41	0.07	0.03	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.02
Total cold run time: 104.01 s
Total hot run time: 29.1 s

hello-stephen · 2025-07-22T08:41:15Z

FE UT Coverage Report

Increment line coverage 29.82% (17/57) 🎉
Increment coverage report
Complete coverage report

morningman requested a review from morrySnow as a code owner July 22, 2025 05:10

wuwenchi and others added 2 commits July 22, 2025 00:20

[opt](iceberg) support insert to branch (apache#52751)

6efd95c

Related Issue: apache#48285 Problem Summary: support insert to branch, like : ```sql insert into tb@branch(br1); ```

[fix](test) fix unstable iceberg_branch_insert_data.groovy

3b5dcf8

morningman force-pushed the 31_bp52751 branch from 455199d to 3b5dcf8 Compare July 22, 2025 07:20

morningman changed the title ~~[opt](iceberg) support insert to branch (#52751)~~ [opt](iceberg) support insert to branch (#52751)(#53703) Jul 22, 2025

morrySnow approved these changes Jul 22, 2025

View reviewed changes

morrySnow changed the title ~~[opt](iceberg) support insert to branch (#52751)(#53703)~~ branch-3.1: [opt](iceberg) support insert to branch #52751 #53703 Jul 22, 2025

morrySnow merged commit 2421a60 into apache:branch-3.1 Jul 22, 2025

Navigation Menu

Repository navigation

branch-3.1: [opt](iceberg) support insert to branch #52751 #53703 - #53693

branch-3.1: [opt](iceberg) support insert to branch #52751 #53703#53693
morrySnow merged 2 commits into
apache:branch-3.1from
morningman:31_bp52751

morningman commented Jul 22, 2025 •

edited

Loading

Uh oh!

Thearas commented Jul 22, 2025

Uh oh!

morningman commented Jul 22, 2025

Uh oh!

doris-robot commented Jul 22, 2025

Uh oh!

doris-robot commented Jul 22, 2025

Uh oh!

doris-robot commented Jul 22, 2025

Uh oh!

morningman commented Jul 22, 2025

Uh oh!

doris-robot commented Jul 22, 2025

Uh oh!

doris-robot commented Jul 22, 2025

Uh oh!

doris-robot commented Jul 22, 2025

Uh oh!

hello-stephen commented Jul 22, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

morningman commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Thearas commented Jul 22, 2025

Uh oh!

morningman commented Jul 22, 2025

Uh oh!

doris-robot commented Jul 22, 2025

Uh oh!

doris-robot commented Jul 22, 2025

Uh oh!

doris-robot commented Jul 22, 2025

Uh oh!

morningman commented Jul 22, 2025

Uh oh!

doris-robot commented Jul 22, 2025

Uh oh!

doris-robot commented Jul 22, 2025

Uh oh!

doris-robot commented Jul 22, 2025

Uh oh!

hello-stephen commented Jul 22, 2025

FE UT Coverage Report

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

morningman commented Jul 22, 2025 •

edited

Loading