Skip to content

[feat](Ann Index)Support create index and build index for ANN - #55586

Merged
airborne12 merged 9 commits into
apache:masterfrom
uchenily:create-build-ann-index
Nov 20, 2025
Merged

[feat](Ann Index)Support create index and build index for ANN#55586
airborne12 merged 9 commits into
apache:masterfrom
uchenily:create-build-ann-index

Conversation

@uchenily

@uchenily uchenily commented Sep 2, 2025

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

This PR add CREATE INDEX and BUILD INDEX sql syntax for ANN index

i.e.

CREATE INDEX [IF NOT EXISTS] <ann_index_name> 
             ON <table_name> (<column_name>)
             USING ANN
             [PROPERTIES ("<key>" = "<value>"[ , ...])]
             [COMMENT '<index_comment>']

BUILD INDEX <ann_index_name> ON <table_name> [partition_list]

Related PR: #54276

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

hello-stephen commented Sep 2, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@uchenily
uchenily force-pushed the create-build-ann-index branch from fc3e37b to 435208f Compare September 2, 2025 09:12
@uchenily

uchenily commented Sep 2, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Sep 2, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 34376 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 435208f9b26fe9a0d6bc6a03e0dbc292a90a22d6, data reload: false

------ Round 1 ----------------------------------
q1	17616	5308	5166	5166
q2	2046	325	217	217
q3	10266	1298	754	754
q4	10247	1049	524	524
q5	7569	2420	2423	2420
q6	190	172	144	144
q7	950	774	636	636
q8	9346	1442	1148	1148
q9	7095	5181	5181	5181
q10	6990	2443	1959	1959
q11	506	299	306	299
q12	389	366	237	237
q13	17778	3657	3019	3019
q14	253	241	238	238
q15	584	514	484	484
q16	442	431	393	393
q17	620	875	379	379
q18	7804	7271	7032	7032
q19	1096	965	570	570
q20	353	345	237	237
q21	4015	3196	2370	2370
q22	1081	1037	969	969
Total cold run time: 107236 ms
Total hot run time: 34376 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5252	5225	5223	5223
q2	260	341	235	235
q3	2169	2712	2320	2320
q4	1377	1780	1371	1371
q5	4273	4544	4654	4544
q6	224	180	140	140
q7	2076	1959	1824	1824
q8	2741	2694	2662	2662
q9	7458	7307	7489	7307
q10	3112	3369	2859	2859
q11	579	526	505	505
q12	720	838	650	650
q13	3551	4053	3313	3313
q14	293	299	296	296
q15	519	498	514	498
q16	473	517	448	448
q17	1215	1582	1485	1485
q18	7939	7575	7738	7575
q19	861	801	888	801
q20	1907	1970	1877	1877
q21	4759	4315	4402	4315
q22	1121	1031	1022	1022
Total cold run time: 52879 ms
Total hot run time: 51270 ms

@doris-robot

doris-robot commented Sep 2, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 187800 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 435208f9b26fe9a0d6bc6a03e0dbc292a90a22d6, data reload: false

query1	1057	423	448	423
query2	6579	1756	1761	1756
query3	6770	231	229	229
query4	26660	23171	23021	23021
query5	4376	661	512	512
query6	354	261	234	234
query7	4657	539	313	313
query8	311	265	261	261
query9	8655	2953	2948	2948
query10	519	370	309	309
query11	15440	15007	14787	14787
query12	184	130	126	126
query13	1693	579	471	471
query14	9462	6023	6055	6023
query15	229	205	184	184
query16	7743	691	546	546
query17	1328	792	670	670
query18	2084	460	363	363
query19	224	260	173	173
query20	138	130	128	128
query21	218	132	120	120
query22	4092	4087	4009	4009
query23	34053	33338	33054	33054
query24	8276	2402	2411	2402
query25	580	531	461	461
query26	834	291	174	174
query27	2646	509	360	360
query28	4355	2329	2310	2310
query29	718	618	496	496
query30	297	248	201	201
query31	915	802	751	751
query32	95	87	78	78
query33	576	405	393	393
query34	806	861	539	539
query35	828	815	761	761
query36	979	994	932	932
query37	131	115	94	94
query38	4085	4033	4018	4018
query39	1508	1430	1436	1430
query40	235	153	138	138
query41	68	68	63	63
query42	128	114	125	114
query43	514	530	474	474
query44	1388	888	870	870
query45	191	179	180	179
query46	879	1022	667	667
query47	1748	1827	1705	1705
query48	395	449	367	367
query49	723	505	410	410
query50	673	702	406	406
query51	4141	4101	4128	4101
query52	113	118	107	107
query53	247	275	206	206
query54	633	608	562	562
query55	102	91	95	91
query56	394	357	328	328
query57	1208	1199	1147	1147
query58	304	294	297	294
query59	2642	2735	2697	2697
query60	374	369	363	363
query61	173	164	175	164
query62	814	754	681	681
query63	274	200	199	199
query64	3633	1144	845	845
query65	4315	4261	4229	4229
query66	911	457	368	368
query67	15428	15248	15066	15066
query68	8940	942	645	645
query69	507	354	309	309
query70	1259	1138	1092	1092
query71	572	368	327	327
query72	5671	5188	4986	4986
query73	727	615	365	365
query74	9170	9246	8848	8848
query75	4183	3121	2670	2670
query76	3585	1205	782	782
query77	821	420	339	339
query78	9531	9883	8936	8936
query79	1624	875	607	607
query80	699	666	536	536
query81	489	263	229	229
query82	194	150	113	113
query83	297	263	259	259
query84	301	119	93	93
query85	842	476	433	433
query86	352	312	306	306
query87	4330	4281	4226	4226
query88	2888	2237	2244	2237
query89	405	338	296	296
query90	2071	235	238	235
query91	166	166	136	136
query92	85	82	75	75
query93	1137	998	650	650
query94	691	422	385	385
query95	418	345	336	336
query96	484	603	283	283
query97	2687	2710	2622	2622
query98	258	218	215	215
query99	1468	1448	1328	1328
Total cold run time: 273955 ms
Total hot run time: 187800 ms

@doris-robot

doris-robot commented Sep 2, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 32.89 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 435208f9b26fe9a0d6bc6a03e0dbc292a90a22d6, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.07	0.06
query3	0.26	0.09	0.09
query4	1.61	0.12	0.12
query5	0.45	0.43	0.42
query6	1.16	0.66	0.65
query7	0.04	0.03	0.03
query8	0.07	0.05	0.05
query9	0.61	0.54	0.53
query10	0.58	0.57	0.58
query11	0.17	0.12	0.12
query12	0.16	0.13	0.13
query13	0.64	0.63	0.62
query14	0.84	0.83	0.83
query15	0.89	0.85	0.89
query16	0.40	0.40	0.42
query17	1.03	1.08	1.03
query18	0.23	0.21	0.20
query19	1.97	1.80	1.88
query20	0.01	0.02	0.01
query21	15.40	0.95	0.58
query22	0.76	1.26	0.86
query23	14.74	1.41	0.65
query24	6.84	1.33	0.55
query25	0.55	0.16	0.14
query26	0.72	0.17	0.14
query27	0.06	0.05	0.06
query28	9.87	0.97	0.44
query29	12.61	3.92	3.27
query30	3.12	3.10	3.05
query31	2.83	0.60	0.38
query32	3.25	0.57	0.48
query33	3.04	3.05	3.08
query34	16.00	5.49	4.89
query35	4.98	4.91	4.98
query36	0.71	0.52	0.50
query37	0.11	0.08	0.08
query38	0.06	0.05	0.05
query39	0.04	0.04	0.03
query40	0.20	0.15	0.14
query41	0.09	0.03	0.04
query42	0.04	0.03	0.02
query43	0.04	0.04	0.04
Total cold run time: 107.33 s
Total hot run time: 32.89 s

@hello-stephen

hello-stephen commented Sep 2, 2025

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 41.00% (82/200) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 51.71% (17148/33160)
Line Coverage 37.23% (156753/421052)
Region Coverage 31.87% (119601/375298)
Branch Coverage 33.23% (52546/158133)

@hello-stephen

hello-stephen commented Sep 2, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 51.00% (102/200) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 70.62% (23006/32577)
Line Coverage 56.92% (239588/420925)
Region Coverage 52.31% (199189/380816)
Branch Coverage 53.95% (85849/159114)

@hello-stephen

hello-stephen commented Sep 2, 2025

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 90.00% (9/10) 🎉
Increment coverage report
Complete coverage report

@uchenily
uchenily marked this pull request as draft September 3, 2025 02:05
@uchenily

uchenily commented Sep 3, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Sep 3, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 34327 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 61868aaf6e70f5233f34e917325d07848c4b945f, data reload: false

------ Round 1 ----------------------------------
q1	17617	5213	5099	5099
q2	2003	327	213	213
q3	10258	1295	705	705
q4	10223	1030	536	536
q5	7538	2454	2381	2381
q6	187	169	148	148
q7	951	757	659	659
q8	9366	1341	1194	1194
q9	7054	5168	5152	5152
q10	6982	2395	1974	1974
q11	504	309	286	286
q12	387	366	225	225
q13	17791	3643	3034	3034
q14	251	244	223	223
q15	553	508	502	502
q16	441	434	386	386
q17	590	872	369	369
q18	7462	7142	7126	7126
q19	1099	962	556	556
q20	346	347	233	233
q21	3720	2626	2349	2349
q22	1065	1025	977	977
Total cold run time: 106388 ms
Total hot run time: 34327 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5162	5115	5095	5095
q2	257	340	232	232
q3	2148	2716	2344	2344
q4	1350	1812	1339	1339
q5	4235	4368	4606	4368
q6	216	170	132	132
q7	2102	1971	1847	1847
q8	2764	2725	2635	2635
q9	7356	7476	7332	7332
q10	3194	3305	2893	2893
q11	605	562	527	527
q12	694	867	680	680
q13	3593	3881	3246	3246
q14	286	323	276	276
q15	519	481	479	479
q16	432	504	612	504
q17	1165	1614	1403	1403
q18	7825	7794	7697	7697
q19	871	798	863	798
q20	2014	2070	1908	1908
q21	4853	4362	4259	4259
q22	1077	1055	1010	1010
Total cold run time: 52718 ms
Total hot run time: 51004 ms

@doris-robot

doris-robot commented Sep 3, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 187392 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 61868aaf6e70f5233f34e917325d07848c4b945f, data reload: false

query1	1050	480	426	426
query2	6571	1743	1720	1720
query3	6758	226	220	220
query4	26152	23468	22957	22957
query5	4470	683	556	556
query6	341	246	234	234
query7	4645	527	307	307
query8	331	272	252	252
query9	8696	2971	2972	2971
query10	516	357	318	318
query11	15451	15327	14793	14793
query12	174	124	116	116
query13	1669	554	423	423
query14	8738	5871	5798	5798
query15	214	194	181	181
query16	7141	718	504	504
query17	984	748	633	633
query18	2003	434	337	337
query19	203	196	212	196
query20	132	131	125	125
query21	214	131	115	115
query22	3987	4177	4158	4158
query23	34029	33224	33019	33019
query24	8274	2402	2412	2402
query25	575	512	466	466
query26	1253	284	171	171
query27	2726	517	357	357
query28	4383	2284	2266	2266
query29	841	608	491	491
query30	297	225	199	199
query31	878	821	734	734
query32	95	83	82	82
query33	604	399	359	359
query34	805	864	542	542
query35	824	830	769	769
query36	984	1018	949	949
query37	137	119	91	91
query38	4091	4028	4015	4015
query39	1513	1429	1422	1422
query40	224	143	131	131
query41	72	66	63	63
query42	131	116	124	116
query43	540	507	468	468
query44	1355	887	878	878
query45	181	184	168	168
query46	877	1019	662	662
query47	1803	1797	1774	1774
query48	390	433	320	320
query49	750	519	419	419
query50	654	695	413	413
query51	4212	4283	4147	4147
query52	124	121	114	114
query53	257	271	206	206
query54	647	630	562	562
query55	102	96	96	96
query56	362	363	352	352
query57	1224	1211	1134	1134
query58	304	291	280	280
query59	2578	2713	2561	2561
query60	378	360	365	360
query61	171	160	181	160
query62	804	754	645	645
query63	235	202	198	198
query64	4584	1153	892	892
query65	4255	4254	4211	4211
query66	1199	440	350	350
query67	15755	15132	15060	15060
query68	7674	921	597	597
query69	514	336	301	301
query70	1227	1183	1198	1183
query71	533	359	322	322
query72	6044	4998	4978	4978
query73	677	617	360	360
query74	9246	9237	9020	9020
query75	3126	3091	2623	2623
query76	3151	1168	751	751
query77	503	401	347	347
query78	9417	9733	9048	9048
query79	1611	865	589	589
query80	658	588	597	588
query81	509	262	233	233
query82	201	142	115	115
query83	269	270	262	262
query84	318	119	96	96
query85	851	460	428	428
query86	388	328	310	310
query87	4324	4278	4170	4170
query88	2815	2224	2225	2224
query89	409	332	294	294
query90	1959	230	222	222
query91	215	163	134	134
query92	99	74	73	73
query93	1534	982	647	647
query94	701	413	323	323
query95	422	336	335	335
query96	482	607	285	285
query97	2618	2723	2627	2627
query98	245	220	218	218
query99	1333	1430	1275	1275
Total cold run time: 270990 ms
Total hot run time: 187392 ms

@doris-robot

doris-robot commented Sep 3, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 32.41 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 61868aaf6e70f5233f34e917325d07848c4b945f, data reload: false

query1	0.05	0.04	0.05
query2	0.09	0.05	0.06
query3	0.24	0.08	0.08
query4	1.60	0.11	0.12
query5	0.45	0.42	0.42
query6	1.18	0.65	0.64
query7	0.03	0.02	0.03
query8	0.05	0.05	0.04
query9	0.60	0.54	0.52
query10	0.59	0.58	0.58
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.63	0.63	0.62
query14	0.80	0.89	0.81
query15	0.88	0.84	0.86
query16	0.41	0.41	0.39
query17	1.07	1.04	1.06
query18	0.22	0.21	0.20
query19	1.91	1.83	1.84
query20	0.01	0.01	0.01
query21	15.41	1.03	0.59
query22	0.81	1.22	0.77
query23	14.80	1.40	0.65
query24	6.87	1.14	0.43
query25	0.52	0.12	0.08
query26	0.63	0.18	0.14
query27	0.07	0.06	0.05
query28	9.86	0.94	0.42
query29	12.56	3.89	3.22
query30	3.15	3.00	2.95
query31	2.83	0.60	0.38
query32	3.24	0.57	0.49
query33	3.15	3.09	3.10
query34	16.15	5.43	4.88
query35	4.95	4.94	4.95
query36	0.71	0.52	0.51
query37	0.10	0.08	0.06
query38	0.06	0.04	0.05
query39	0.03	0.03	0.03
query40	0.19	0.16	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.05	0.03	0.03
Total cold run time: 107.38 s
Total hot run time: 32.41 s

@doris-robot

doris-robot commented Sep 3, 2025

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 46.00% (92/200) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 51.84% (17207/33194)
Line Coverage 37.28% (157188/421664)
Region Coverage 31.97% (120250/376105)
Branch Coverage 33.32% (52732/158258)

@uchenily
uchenily marked this pull request as ready for review September 3, 2025 11:53
@hello-stephen

hello-stephen commented Sep 3, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 56.00% (112/200) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 70.63% (23022/32597)
Line Coverage 56.96% (240026/421404)
Region Coverage 52.38% (199815/381484)
Branch Coverage 54.07% (86049/159151)

@hello-stephen

hello-stephen commented Sep 3, 2025

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 90.00% (9/10) 🎉
Increment coverage report
Complete coverage report

@uchenily

uchenily commented Sep 3, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Sep 3, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 34181 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d13c56a7b874a527dffb46dcc21c8c19717ba152, data reload: false

------ Round 1 ----------------------------------
q1	17624	5279	5126	5126
q2	1983	373	210	210
q3	10225	1306	754	754
q4	10235	1024	535	535
q5	7531	2484	2360	2360
q6	181	175	137	137
q7	966	774	638	638
q8	9347	1308	1142	1142
q9	7071	5096	5168	5096
q10	6976	2410	2005	2005
q11	492	298	302	298
q12	367	364	231	231
q13	17800	3663	3011	3011
q14	236	242	228	228
q15	564	497	482	482
q16	428	429	368	368
q17	604	886	358	358
q18	8160	7165	7022	7022
q19	1086	944	565	565
q20	348	344	236	236
q21	3948	3217	2390	2390
q22	1058	1034	989	989
Total cold run time: 107230 ms
Total hot run time: 34181 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5183	5111	5128	5111
q2	261	333	227	227
q3	2201	2722	2292	2292
q4	1382	1775	1344	1344
q5	4247	4527	4594	4527
q6	231	200	133	133
q7	2020	1941	1813	1813
q8	2613	2698	2757	2698
q9	7470	7385	7308	7308
q10	3134	3505	2847	2847
q11	584	538	497	497
q12	722	764	637	637
q13	3570	3894	3329	3329
q14	299	315	297	297
q15	513	519	453	453
q16	464	502	459	459
q17	1195	1544	1438	1438
q18	7751	7723	7775	7723
q19	829	886	903	886
q20	1971	2139	1881	1881
q21	5108	4661	4506	4506
q22	1105	1043	993	993
Total cold run time: 52853 ms
Total hot run time: 51399 ms

@hello-stephen

hello-stephen commented Sep 3, 2025

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 10.00% (1/10) 🎉
Increment coverage report
Complete coverage report

@doris-robot

doris-robot commented Sep 3, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 186565 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d13c56a7b874a527dffb46dcc21c8c19717ba152, data reload: false

query1	1047	457	396	396
query2	6559	1702	1710	1702
query3	6762	236	237	236
query4	26513	23639	23208	23208
query5	5071	699	496	496
query6	333	237	218	218
query7	4669	515	297	297
query8	303	246	236	236
query9	8654	2929	2921	2921
query10	509	347	290	290
query11	16070	15145	14724	14724
query12	171	120	113	113
query13	1669	553	420	420
query14	9672	6130	5849	5849
query15	225	200	176	176
query16	7679	661	476	476
query17	1325	743	617	617
query18	2029	419	363	363
query19	201	207	173	173
query20	133	127	121	121
query21	211	137	124	124
query22	3941	4254	4061	4061
query23	33849	32941	32932	32932
query24	8114	2366	2423	2366
query25	573	517	430	430
query26	1240	287	164	164
query27	2729	509	352	352
query28	4361	2267	2251	2251
query29	765	596	486	486
query30	291	226	200	200
query31	935	828	713	713
query32	90	82	78	78
query33	573	433	354	354
query34	787	847	518	518
query35	807	851	757	757
query36	973	1055	947	947
query37	130	111	92	92
query38	4048	4021	4054	4021
query39	1470	1422	1437	1422
query40	230	134	127	127
query41	69	63	65	63
query42	129	117	115	115
query43	508	510	481	481
query44	1363	881	860	860
query45	180	173	175	173
query46	867	1010	652	652
query47	1756	1809	1709	1709
query48	414	432	336	336
query49	785	510	403	403
query50	638	699	411	411
query51	4112	4077	4124	4077
query52	124	117	102	102
query53	247	280	204	204
query54	616	613	548	548
query55	102	94	89	89
query56	350	320	315	315
query57	1171	1213	1145	1145
query58	285	284	289	284
query59	2633	2609	2586	2586
query60	380	355	351	351
query61	172	184	202	184
query62	839	733	659	659
query63	226	197	196	196
query64	4455	1161	838	838
query65	4324	4238	4246	4238
query66	1096	437	358	358
query67	15529	15374	15140	15140
query68	5034	938	591	591
query69	504	341	297	297
query70	1226	1189	1107	1107
query71	551	355	327	327
query72	6094	5206	5283	5206
query73	704	652	359	359
query74	8937	9079	8704	8704
query75	3140	3100	2601	2601
query76	3186	1148	732	732
query77	485	410	343	343
query78	9700	9744	8924	8924
query79	2941	854	594	594
query80	1046	592	524	524
query81	529	358	226	226
query82	746	138	108	108
query83	261	267	253	253
query84	264	115	91	91
query85	867	466	435	435
query86	419	313	294	294
query87	4332	4193	4298	4193
query88	3937	2224	2259	2224
query89	413	340	302	302
query90	1972	231	235	231
query91	164	172	136	136
query92	96	78	78	78
query93	2596	1032	650	650
query94	847	395	328	328
query95	417	341	330	330
query96	496	575	282	282
query97	2635	2676	2584	2584
query98	254	219	213	213
query99	1375	1431	1354	1354
Total cold run time: 275249 ms
Total hot run time: 186565 ms

@doris-robot

doris-robot commented Sep 3, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 32.64 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d13c56a7b874a527dffb46dcc21c8c19717ba152, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.05	0.06
query3	0.25	0.09	0.09
query4	1.61	0.12	0.12
query5	0.44	0.42	0.40
query6	1.16	0.65	0.66
query7	0.04	0.02	0.03
query8	0.06	0.05	0.05
query9	0.61	0.52	0.53
query10	0.58	0.57	0.57
query11	0.16	0.11	0.11
query12	0.16	0.12	0.12
query13	0.63	0.63	0.62
query14	0.79	0.86	0.85
query15	0.88	0.83	0.86
query16	0.39	0.41	0.41
query17	1.06	1.07	1.04
query18	0.23	0.20	0.20
query19	1.96	1.87	1.76
query20	0.02	0.01	0.02
query21	15.40	0.97	0.58
query22	0.80	1.20	0.82
query23	14.81	1.40	0.63
query24	6.59	0.60	1.00
query25	0.48	0.19	0.12
query26	0.62	0.16	0.13
query27	0.06	0.05	0.05
query28	9.89	0.95	0.43
query29	12.55	3.88	3.23
query30	3.12	3.08	2.97
query31	2.83	0.59	0.37
query32	3.24	0.55	0.47
query33	3.07	3.20	3.12
query34	15.90	5.46	4.82
query35	4.95	4.93	4.95
query36	0.68	0.52	0.53
query37	0.10	0.07	0.08
query38	0.06	0.05	0.04
query39	0.04	0.03	0.03
query40	0.19	0.15	0.15
query41	0.09	0.04	0.03
query42	0.04	0.02	0.02
query43	0.04	0.04	0.04
Total cold run time: 106.73 s
Total hot run time: 32.64 s

@hello-stephen

hello-stephen commented Sep 3, 2025

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 62.84% (115/183) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 51.84% (17205/33190)
Line Coverage 37.28% (157218/421681)
Region Coverage 31.93% (120090/376084)
Branch Coverage 33.32% (52727/158252)

@hello-stephen

hello-stephen commented Sep 3, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 73.77% (135/183) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 70.68% (23037/32592)
Line Coverage 57.01% (240234/421413)
Region Coverage 52.29% (199470/381456)
Branch Coverage 54.06% (86039/159142)

@hello-stephen

hello-stephen commented Sep 3, 2025

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 90.00% (9/10) 🎉
Increment coverage report
Complete coverage report

RoanHeNaN
RoanHeNaN reviewed Sep 4, 2025
View reviewed changes
Comment thread be/src/olap/rowset/segment_v2/segment_iterator.cpp Resolved
Comment thread be/src/olap/rowset/segment_v2/segment_iterator.cpp Resolved
@uchenily
uchenily force-pushed the create-build-ann-index branch from d13c56a to 01b74f3 Compare September 5, 2025 02:00
@hello-stephen

hello-stephen commented Nov 14, 2025

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 69.06% (125/181) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.81% (18286/34623)
Line Coverage 38.20% (166370/435504)
Region Coverage 33.19% (129351/389687)
Branch Coverage 33.93% (55520/163621)

@hello-stephen

hello-stephen commented Nov 14, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 80.11% (145/181) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.58% (24353/34024)
Line Coverage 58.07% (253315/436187)
Region Coverage 53.50% (211456/395212)
Branch Coverage 54.78% (90234/164708)

@hello-stephen

hello-stephen commented Nov 14, 2025

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 90.00% (9/10) 🎉
Increment coverage report
Complete coverage report

@airborne12

airborne12 commented Nov 19, 2025

Copy link
Copy Markdown
Member

run buildall

@hello-stephen

hello-stephen commented Nov 19, 2025

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 69.06% (125/181) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.74% (18278/34659)
Line Coverage 38.12% (166172/435905)
Region Coverage 33.07% (129289/390949)
Branch Coverage 33.84% (55468/163900)

@uchenily

uchenily commented Nov 20, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Nov 20, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 34818 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5927b67472b01732739d8157771b0ad00109e082, data reload: false

------ Round 1 ----------------------------------
q1	17608	5049	4927	4927
q2	2061	315	202	202
q3	10275	1320	739	739
q4	10279	988	374	374
q5	8487	2540	2195	2195
q6	214	172	134	134
q7	946	770	616	616
q8	9345	1347	1051	1051
q9	7276	5426	5479	5426
q10	6874	2235	1796	1796
q11	489	308	284	284
q12	336	357	232	232
q13	17803	3674	3007	3007
q14	227	236	209	209
q15	585	509	495	495
q16	1030	983	940	940
q17	591	887	370	370
q18	7437	7232	7059	7059
q19	1401	935	566	566
q20	363	351	232	232
q21	3780	3288	2979	2979
q22	1094	1026	985	985
Total cold run time: 108501 ms
Total hot run time: 34818 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5064	4932	4926	4926
q2	324	400	326	326
q3	2222	2656	2272	2272
q4	1378	1796	1314	1314
q5	4349	4414	4497	4414
q6	224	174	134	134
q7	1994	1967	1815	1815
q8	2646	2613	2597	2597
q9	7593	7663	7594	7594
q10	3123	3302	2807	2807
q11	599	534	514	514
q12	682	809	616	616
q13	3464	4023	3223	3223
q14	297	312	269	269
q15	535	495	514	495
q16	1052	1101	1049	1049
q17	1224	1592	1380	1380
q18	7885	7834	7431	7431
q19	779	739	772	739
q20	1920	2003	1836	1836
q21	4736	4462	4259	4259
q22	1102	1090	998	998
Total cold run time: 53192 ms
Total hot run time: 51008 ms

@doris-robot

doris-robot commented Nov 20, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 188107 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5927b67472b01732739d8157771b0ad00109e082, data reload: false

query1	1005	424	384	384
query2	6554	1669	1714	1669
query3	6758	224	226	224
query4	26283	23228	22930	22930
query5	4455	685	519	519
query6	346	264	241	241
query7	4646	500	314	314
query8	311	266	274	266
query9	8673	2930	2919	2919
query10	521	351	314	314
query11	16010	15006	14856	14856
query12	183	129	123	123
query13	1690	596	460	460
query14	10731	9088	9058	9058
query15	197	187	174	174
query16	7366	655	487	487
query17	1218	746	636	636
query18	2006	412	324	324
query19	213	199	179	179
query20	133	123	124	123
query21	217	138	117	117
query22	3923	4089	3949	3949
query23	33982	33033	32915	32915
query24	8146	2368	2389	2368
query25	614	530	474	474
query26	1244	283	176	176
query27	2747	508	366	366
query28	4389	2256	2221	2221
query29	856	672	534	534
query30	307	228	206	206
query31	894	812	751	751
query32	98	87	84	84
query33	592	398	352	352
query34	784	867	534	534
query35	817	827	760	760
query36	946	970	889	889
query37	136	117	102	102
query38	3527	3535	3457	3457
query39	1486	1427	1404	1404
query40	251	141	134	134
query41	66	66	66	66
query42	131	120	124	120
query43	468	488	456	456
query44	1256	797	809	797
query45	197	189	172	172
query46	892	1004	648	648
query47	1784	1789	1707	1707
query48	406	432	329	329
query49	794	504	428	428
query50	670	691	402	402
query51	3861	3897	3879	3879
query52	121	125	113	113
query53	244	270	203	203
query54	347	342	316	316
query55	99	96	97	96
query56	373	370	351	351
query57	1169	1191	1129	1129
query58	311	297	297	297
query59	2588	2643	2555	2555
query60	421	368	351	351
query61	163	159	162	159
query62	796	736	644	644
query63	224	197	197	197
query64	4518	1163	871	871
query65	4023	3897	3929	3897
query66	1196	432	336	336
query67	15246	15064	14805	14805
query68	8234	957	634	634
query69	493	332	297	297
query70	1339	1328	1306	1306
query71	480	345	333	333
query72	6074	4881	4960	4881
query73	676	595	367	367
query74	8860	9213	9019	9019
query75	3893	3256	2801	2801
query76	3628	1148	725	725
query77	829	407	350	350
query78	9613	9843	8827	8827
query79	2104	857	620	620
query80	675	597	541	541
query81	500	258	223	223
query82	233	164	145	145
query83	274	278	257	257
query84	254	118	97	97
query85	900	496	459	459
query86	347	319	310	310
query87	3680	3719	3634	3634
query88	3542	2249	2259	2249
query89	387	322	298	298
query90	2104	237	238	237
query91	171	164	132	132
query92	93	79	71	71
query93	1587	1042	693	693
query94	678	440	349	349
query95	446	335	343	335
query96	494	600	285	285
query97	2875	2971	2843	2843
query98	259	227	220	220
query99	1328	1381	1252	1252
Total cold run time: 274669 ms
Total hot run time: 188107 ms

@doris-robot

doris-robot commented Nov 20, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 27.87 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5927b67472b01732739d8157771b0ad00109e082, data reload: false

query1	0.05	0.05	0.05
query2	0.09	0.05	0.04
query3	0.26	0.08	0.09
query4	1.60	0.11	0.12
query5	0.26	0.24	0.26
query6	1.18	0.65	0.65
query7	0.03	0.02	0.02
query8	0.05	0.05	0.04
query9	0.57	0.52	0.51
query10	0.58	0.57	0.58
query11	0.16	0.12	0.11
query12	0.15	0.12	0.13
query13	0.62	0.61	0.61
query14	1.01	0.99	1.00
query15	0.84	0.83	0.84
query16	0.39	0.38	0.38
query17	1.00	1.03	1.04
query18	0.22	0.20	0.20
query19	1.93	1.86	1.83
query20	0.02	0.04	0.01
query21	15.44	0.20	0.14
query22	4.99	0.08	0.05
query23	15.65	0.27	0.10
query24	2.91	0.79	0.93
query25	0.07	0.05	0.06
query26	0.14	0.15	0.13
query27	0.05	0.05	0.06
query28	5.18	1.14	0.97
query29	12.60	3.99	3.24
query30	0.29	0.13	0.11
query31	2.83	0.59	0.37
query32	3.23	0.54	0.49
query33	3.06	3.04	3.03
query34	15.84	5.16	4.56
query35	4.57	4.54	4.57
query36	0.69	0.50	0.49
query37	0.10	0.06	0.06
query38	0.07	0.05	0.03
query39	0.03	0.03	0.03
query40	0.18	0.14	0.13
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 99.11 s
Total hot run time: 27.87 s

@hello-stephen

hello-stephen commented Nov 20, 2025

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 69.06% (125/181) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.74% (18280/34660)
Line Coverage 38.14% (166241/435905)
Region Coverage 33.08% (129341/390945)
Branch Coverage 33.86% (55492/163896)

@hello-stephen

hello-stephen commented Nov 20, 2025

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 80.11% (145/181) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.44% (24332/34061)
Line Coverage 57.90% (252792/436575)
Region Coverage 53.00% (210143/396522)
Branch Coverage 54.52% (89946/164975)

@hello-stephen

hello-stephen commented Nov 20, 2025

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 90.00% (9/10) 🎉
Increment coverage report
Complete coverage report

airborne12
airborne12 approved these changes Nov 20, 2025
View reviewed changes

@airborne12 airborne12 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

github-actions Bot commented Nov 20, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions Bot added approved Indicates a PR has been approved by one committer. reviewed labels Nov 20, 2025
@github-actions

github-actions Bot commented Nov 20, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@airborne12
airborne12 merged commit 09fc3fd into apache:master Nov 20, 2025
27 of 28 checks passed
airborne12 pushed a commit to airborne12/apache-doris that referenced this pull request Dec 3, 2025
@uchenily @airborne12
[feat](Ann Index)Support create index and build index for ANN (apache…
47f8a3b 
…#55586)

This PR add `CREATE INDEX` and `BUILD INDEX` sql syntax for ANN index

i.e.

```sql
CREATE INDEX [IF NOT EXISTS] <ann_index_name>
             ON <table_name> (<column_name>)
             USING ANN
             [PROPERTIES ("<key>" = "<value>"[ , ...])]
             [COMMENT '<index_comment>']

BUILD INDEX <ann_index_name> ON <table_name> [partition_list]
```

Related PR: apache#54276
@airborne12 airborne12 mentioned this pull request Dec 3, 2025
Merged
yiguolei pushed a commit that referenced this pull request Dec 3, 2025
@airborne12
branch-4.0: [feat](Ann Index)Support create index and build index for…
4d49ccb 
… ANN #55586 (#58651)

cherry pick from #55586
nagisa-kunhah pushed a commit to nagisa-kunhah/doris that referenced this pull request Dec 14, 2025
@uchenily @nagisa-kunhah
[feat](Ann Index)Support create index and build index for ANN (apache…
4fdb8f4 
…#55586)

### What problem does this PR solve?

This PR add `CREATE INDEX` and `BUILD INDEX` sql syntax for ANN index 

i.e.

```sql
CREATE INDEX [IF NOT EXISTS] <ann_index_name> 
             ON <table_name> (<column_name>)
             USING ANN
             [PROPERTIES ("<key>" = "<value>"[ , ...])]
             [COMMENT '<index_comment>']

BUILD INDEX <ann_index_name> ON <table_name> [partition_list]
```


Related PR: apache#54276
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@airborne12 airborne12 airborne12 approved these changes
@gavinchou gavinchou Awaiting requested review from gavinchou gavinchou is a code owner
@dataroaring dataroaring Awaiting requested review from dataroaring
@w41ter w41ter Awaiting requested review from w41ter
+1 more reviewer
@RoanHeNaN RoanHeNaN RoanHeNaN approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/4.0.2-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@uchenily @hello-stephen @doris-robot @airborne12 @RoanHeNaN @yiguolei