Skip to content

[chore](audit) Optimize the SQL (insert into values) length in audit logs - #37894

Merged
gavinchou merged 7 commits into
apache:masterfrom
yagagagaga:chore_20240716
Jul 23, 2024
Merged

[chore](audit) Optimize the SQL (insert into values) length in audit logs#37894
gavinchou merged 7 commits into
apache:masterfrom
yagagagaga:chore_20240716

Conversation

@yagagagaga

@yagagagaga yagagagaga commented Jul 16, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

Proposed changes

  • Add annotation information to the insert into value statement
  • Let the SQL length in the audit log be controlled by GlobalVariable.auditPluginMaxSqlLength

Here is the result of this PR:
image

If the length of sql greater than GlobalVariable.auditPluginMaxSqlLength, the sql will be truncate but it still have a comment if the sql is insert into value

set global audit_plugin_max_sql_length=30;
insert into test.t values (1, 'Tom'), (2, 'Jelly'), (3, 'Damn');
image

@gavinchou

gavinchou commented Jul 16, 2024

Copy link
Copy Markdown
Contributor

Add example what the insert into values will look like after this PR

@gavinchou
gavinchou requested a review from morrySnow July 16, 2024 07:38
@gavinchou gavinchou changed the title [chore](audit) Optimize the sql length in audit logs [chore](audit) Optimize the SQL (insert into values) length in audit logs Jul 16, 2024
morrySnow
morrySnow previously approved these changes Jul 17, 2024
View reviewed changes
@github-actions

github-actions Bot commented Jul 17, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Jul 17, 2024
@github-actions

github-actions Bot commented Jul 17, 2024

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

gavinchou
gavinchou requested changes Jul 17, 2024
View reviewed changes
Comment thread regression-test/suites/audit/test_audit_log_behavior.groovy Resolved
Comment thread fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java Resolved
@github-actions github-actions Bot removed the approved Indicates a PR has been approved by one committer. label Jul 17, 2024
@yagagagaga

yagagagaga commented Jul 22, 2024

Copy link
Copy Markdown
Contributor Author

run buildall

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Jul 22, 2024
@github-actions

github-actions Bot commented Jul 22, 2024

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@doris-robot

doris-robot commented Jul 22, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 39980 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e9a6cc71dffb9f21128219682416a2a3d04124a1, data reload: false

------ Round 1 ----------------------------------
q1	17642	4347	4278	4278
q2	2014	195	186	186
q3	10507	1274	1122	1122
q4	10195	854	789	789
q5	7514	2696	2683	2683
q6	228	139	139	139
q7	982	608	613	608
q8	9221	2080	2061	2061
q9	8896	6585	6621	6585
q10	8817	3769	3804	3769
q11	464	250	241	241
q12	466	225	223	223
q13	17765	2987	2971	2971
q14	274	240	243	240
q15	532	481	466	466
q16	490	384	374	374
q17	974	734	657	657
q18	8090	7633	7431	7431
q19	8335	1444	1361	1361
q20	697	328	338	328
q21	4912	3185	3247	3185
q22	347	283	297	283
Total cold run time: 119362 ms
Total hot run time: 39980 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4423	4252	4248	4248
q2	364	284	276	276
q3	3105	2887	2964	2887
q4	1982	1767	1672	1672
q5	5597	5546	5502	5502
q6	228	138	138	138
q7	2274	1855	1800	1800
q8	3281	3461	3438	3438
q9	8768	8977	8777	8777
q10	4126	3800	3879	3800
q11	613	508	536	508
q12	808	644	615	615
q13	17195	3181	3217	3181
q14	307	293	297	293
q15	511	502	490	490
q16	498	432	425	425
q17	1823	1550	1535	1535
q18	8332	7919	7841	7841
q19	1806	1749	1574	1574
q20	2657	1916	1866	1866
q21	5139	4765	4959	4765
q22	809	517	523	517
Total cold run time: 74646 ms
Total hot run time: 56148 ms

@doris-robot

doris-robot commented Jul 22, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 173896 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e9a6cc71dffb9f21128219682416a2a3d04124a1, data reload: false

query1	919	371	360	360
query2	6368	1939	1836	1836
query3	6649	206	220	206
query4	23573	17610	17313	17313
query5	3592	470	477	470
query6	245	199	172	172
query7	4577	285	288	285
query8	229	186	186	186
query9	8564	2438	2423	2423
query10	431	285	268	268
query11	12095	9991	10023	9991
query12	123	82	82	82
query13	1655	380	391	380
query14	10142	7751	7616	7616
query15	226	167	167	167
query16	7379	483	474	474
query17	1143	563	537	537
query18	1900	283	288	283
query19	199	162	157	157
query20	91	83	82	82
query21	207	133	126	126
query22	4338	4115	3989	3989
query23	34023	33788	33630	33630
query24	10947	2999	2960	2960
query25	734	403	408	403
query26	714	157	160	157
query27	2311	277	279	277
query28	6272	2071	2045	2045
query29	1002	630	635	630
query30	286	155	159	155
query31	932	746	775	746
query32	105	54	55	54
query33	664	345	326	326
query34	889	496	503	496
query35	919	767	792	767
query36	1156	961	976	961
query37	138	77	84	77
query38	2966	2884	2812	2812
query39	903	797	807	797
query40	194	123	117	117
query41	44	45	42	42
query42	119	97	98	97
query43	508	483	458	458
query44	1063	715	711	711
query45	194	161	158	158
query46	1107	723	707	707
query47	1847	1779	1792	1779
query48	377	289	311	289
query49	857	397	406	397
query50	774	398	388	388
query51	6777	6684	6570	6570
query52	114	87	91	87
query53	354	289	286	286
query54	885	443	449	443
query55	76	71	71	71
query56	280	273	263	263
query57	1139	1061	1053	1053
query58	256	240	264	240
query59	2856	2652	2550	2550
query60	291	295	289	289
query61	98	91	92	91
query62	812	650	643	643
query63	320	294	289	289
query64	9191	2232	1709	1709
query65	3176	3153	3140	3140
query66	812	328	325	325
query67	15494	15028	15054	15028
query68	4596	536	530	530
query69	553	335	358	335
query70	1160	1072	1148	1072
query71	440	273	274	273
query72	9099	5560	5682	5560
query73	749	331	322	322
query74	6076	5703	5646	5646
query75	3923	2663	2676	2663
query76	2914	937	956	937
query77	613	307	298	298
query78	9697	9066	9191	9066
query79	3080	532	537	532
query80	2788	468	532	468
query81	600	228	219	219
query82	771	145	128	128
query83	309	175	167	167
query84	263	98	83	83
query85	2098	318	305	305
query86	474	337	305	305
query87	3285	3117	3077	3077
query88	4009	2442	2475	2442
query89	475	401	381	381
query90	1851	199	198	198
query91	135	104	100	100
query92	62	49	48	48
query93	3217	508	498	498
query94	1234	298	298	298
query95	415	321	320	320
query96	614	281	272	272
query97	3259	3074	3009	3009
query98	273	199	201	199
query99	1561	1271	1229	1229
Total cold run time: 280214 ms
Total hot run time: 173896 ms

@doris-robot

doris-robot commented Jul 22, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.17 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e9a6cc71dffb9f21128219682416a2a3d04124a1, data reload: false

query1	0.04	0.04	0.03
query2	0.09	0.04	0.04
query3	0.22	0.04	0.04
query4	1.69	0.08	0.07
query5	0.50	0.48	0.48
query6	1.13	0.72	0.75
query7	0.01	0.01	0.02
query8	0.04	0.04	0.04
query9	0.54	0.48	0.49
query10	0.53	0.55	0.54
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.60	0.58	0.59
query14	0.77	0.79	0.78
query15	0.87	0.81	0.82
query16	0.36	0.37	0.36
query17	0.98	1.02	0.96
query18	0.23	0.22	0.23
query19	1.82	1.65	1.71
query20	0.02	0.01	0.01
query21	15.39	0.75	0.64
query22	4.02	6.70	2.41
query23	18.28	1.37	1.44
query24	2.06	0.25	0.21
query25	0.15	0.09	0.08
query26	0.31	0.22	0.21
query27	0.45	0.23	0.23
query28	13.24	1.04	1.02
query29	12.56	3.35	3.38
query30	0.25	0.06	0.06
query31	2.88	0.39	0.39
query32	3.28	0.50	0.48
query33	2.91	2.99	2.90
query34	17.06	4.37	4.36
query35	4.41	4.44	4.37
query36	0.65	0.48	0.47
query37	0.19	0.16	0.15
query38	0.15	0.15	0.15
query39	0.04	0.04	0.03
query40	0.15	0.12	0.12
query41	0.09	0.04	0.04
query42	0.05	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.36 s
Total hot run time: 31.17 s

@yagagagaga

yagagagaga commented Jul 22, 2024

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Jul 22, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 40111 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c4658686e77f475dc7114aff54392b5274db5b9f, data reload: false

------ Round 1 ----------------------------------
q1	7208	4375	4268	4268
q2	836	195	195	195
q3	2733	1126	1159	1126
q4	8178	819	883	819
q5	2803	2880	2966	2880
q6	221	143	137	137
q7	957	595	604	595
q8	2519	2062	2079	2062
q9	6621	6525	6599	6525
q10	3894	3816	3752	3752
q11	379	247	244	244
q12	383	232	225	225
q13	18442	2980	2966	2966
q14	277	230	243	230
q15	516	484	472	472
q16	468	385	375	375
q17	951	710	694	694
q18	8071	7468	7348	7348
q19	1584	1531	1437	1437
q20	592	323	326	323
q21	4959	3150	3400	3150
q22	528	290	288	288
Total cold run time: 73120 ms
Total hot run time: 40111 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4276	4255	4234	4234
q2	360	272	268	268
q3	3041	2762	2754	2754
q4	1875	1620	1575	1575
q5	5322	5321	5296	5296
q6	214	129	128	128
q7	2123	1754	1719	1719
q8	3186	3357	3310	3310
q9	8626	8755	8794	8755
q10	4199	3849	3862	3849
q11	603	510	507	507
q12	828	644	672	644
q13	17870	3142	3174	3142
q14	332	307	308	307
q15	550	505	488	488
q16	508	455	442	442
q17	1800	1503	1517	1503
q18	8107	7848	7696	7696
q19	1713	1551	1502	1502
q20	2072	1879	1856	1856
q21	5105	4912	4793	4793
q22	610	494	522	494
Total cold run time: 73320 ms
Total hot run time: 55262 ms

@doris-robot

doris-robot commented Jul 22, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 174611 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c4658686e77f475dc7114aff54392b5274db5b9f, data reload: false

query1	924	385	376	376
query2	6262	1976	1868	1868
query3	6652	208	212	208
query4	27742	17749	17519	17519
query5	3897	474	484	474
query6	269	172	157	157
query7	4552	301	287	287
query8	235	206	198	198
query9	8356	2454	2418	2418
query10	424	281	272	272
query11	10740	10387	10376	10376
query12	128	80	81	80
query13	1607	372	355	355
query14	9903	7190	7819	7190
query15	219	163	163	163
query16	7703	510	449	449
query17	1131	540	519	519
query18	1850	290	310	290
query19	196	146	150	146
query20	90	82	79	79
query21	202	125	123	123
query22	4448	4332	4224	4224
query23	33981	33736	33760	33736
query24	7234	2929	2868	2868
query25	584	398	386	386
query26	726	152	154	152
query27	2035	277	275	275
query28	3791	2092	2080	2080
query29	874	640	629	629
query30	227	161	151	151
query31	982	753	796	753
query32	100	61	61	61
query33	501	379	339	339
query34	927	501	492	492
query35	851	740	759	740
query36	1139	943	922	922
query37	139	82	86	82
query38	2880	2860	2780	2780
query39	922	863	841	841
query40	216	121	126	121
query41	48	44	44	44
query42	125	103	100	100
query43	532	479	492	479
query44	1118	761	749	749
query45	185	168	169	168
query46	1106	738	729	729
query47	1942	1855	1823	1823
query48	375	306	292	292
query49	838	434	422	422
query50	769	392	397	392
query51	6711	6716	6627	6627
query52	101	88	94	88
query53	350	282	287	282
query54	559	456	453	453
query55	76	72	70	70
query56	281	270	277	270
query57	1116	1049	1034	1034
query58	248	242	245	242
query59	3072	2695	2610	2610
query60	308	274	266	266
query61	95	91	93	91
query62	769	627	630	627
query63	333	297	290	290
query64	9116	2274	1780	1780
query65	3166	3111	3121	3111
query66	814	333	329	329
query67	15592	15271	14995	14995
query68	4549	541	536	536
query69	494	357	334	334
query70	1175	1156	1136	1136
query71	369	282	278	278
query72	6928	5562	5750	5562
query73	745	329	324	324
query74	6155	5785	5703	5703
query75	3359	2699	2701	2699
query76	2137	938	886	886
query77	447	305	307	305
query78	9684	9875	9018	9018
query79	2486	520	513	513
query80	2472	462	473	462
query81	593	222	216	216
query82	738	135	145	135
query83	290	169	163	163
query84	266	93	86	86
query85	1765	310	298	298
query86	491	308	316	308
query87	3261	3106	3090	3090
query88	3869	2444	2509	2444
query89	479	396	380	380
query90	1765	195	192	192
query91	134	120	103	103
query92	69	50	50	50
query93	2790	492	497	492
query94	1221	282	279	279
query95	408	315	315	315
query96	610	278	274	274
query97	3213	3082	3046	3046
query98	208	201	193	193
query99	1696	1249	1256	1249
Total cold run time: 270250 ms
Total hot run time: 174611 ms

@doris-robot

doris-robot commented Jul 22, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 30.8 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c4658686e77f475dc7114aff54392b5274db5b9f, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.05	0.04
query3	0.22	0.04	0.05
query4	1.68	0.08	0.07
query5	0.49	0.48	0.47
query6	1.12	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.48
query10	0.54	0.54	0.56
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.59	0.58	0.58
query14	0.77	0.77	0.82
query15	0.85	0.82	0.81
query16	0.38	0.37	0.37
query17	1.01	0.98	1.04
query18	0.23	0.23	0.22
query19	1.78	1.70	1.68
query20	0.01	0.01	0.03
query21	15.39	0.77	0.64
query22	4.75	6.14	2.25
query23	18.27	1.34	1.19
query24	2.06	0.23	0.22
query25	0.14	0.08	0.09
query26	0.31	0.22	0.21
query27	0.45	0.23	0.23
query28	13.35	1.02	1.00
query29	12.68	3.42	3.36
query30	0.26	0.06	0.05
query31	2.87	0.40	0.38
query32	3.28	0.47	0.47
query33	2.87	2.85	2.99
query34	16.88	4.33	4.35
query35	4.38	4.43	4.40
query36	0.67	0.47	0.46
query37	0.18	0.16	0.16
query38	0.15	0.16	0.15
query39	0.05	0.04	0.04
query40	0.14	0.12	0.12
query41	0.10	0.04	0.04
query42	0.06	0.05	0.04
query43	0.06	0.05	0.04
Total cold run time: 110.06 s
Total hot run time: 30.8 s

@yagagagaga

yagagagaga commented Jul 23, 2024

Copy link
Copy Markdown
Contributor Author

run buildall

@yagagagaga

yagagagaga commented Jul 23, 2024

Copy link
Copy Markdown
Contributor Author

/run buildall

@doris-robot

doris-robot commented Jul 23, 2024

Copy link
Copy Markdown
TPC-H: Total hot run time: 39447 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c4658686e77f475dc7114aff54392b5274db5b9f, data reload: false

------ Round 1 ----------------------------------
q1	17610	4308	4227	4227
q2	2018	189	183	183
q3	10463	1181	1139	1139
q4	10183	784	745	745
q5	7561	2675	2634	2634
q6	217	135	135	135
q7	942	604	594	594
q8	9221	2025	2066	2025
q9	8738	6492	6555	6492
q10	8860	3772	3819	3772
q11	444	233	238	233
q12	395	225	222	222
q13	17759	2957	3002	2957
q14	278	229	242	229
q15	504	468	488	468
q16	485	383	377	377
q17	954	709	772	709
q18	7942	7514	7195	7195
q19	6651	1399	1404	1399
q20	673	317	322	317
q21	4801	3115	3227	3115
q22	345	280	288	280
Total cold run time: 117044 ms
Total hot run time: 39447 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4393	4203	4226	4203
q2	361	262	273	262
q3	2972	2791	2874	2791
q4	1986	1680	1663	1663
q5	5678	5517	5478	5478
q6	223	130	132	130
q7	2164	1874	1878	1874
q8	3270	3389	3392	3389
q9	8751	8728	8858	8728
q10	4063	3850	3745	3745
q11	563	522	499	499
q12	817	679	653	653
q13	17309	3155	3208	3155
q14	315	283	296	283
q15	552	473	489	473
q16	486	434	433	433
q17	1815	1515	1500	1500
q18	8109	7984	7860	7860
q19	1729	1615	1518	1518
q20	2459	1848	1835	1835
q21	9051	4696	4847	4696
q22	570	522	503	503
Total cold run time: 77636 ms
Total hot run time: 55671 ms

@doris-robot

doris-robot commented Jul 23, 2024

Copy link
Copy Markdown
TPC-DS: Total hot run time: 173399 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c4658686e77f475dc7114aff54392b5274db5b9f, data reload: false

query1	908	372	352	352
query2	6415	1983	1824	1824
query3	6649	207	216	207
query4	20992	17409	17311	17311
query5	3685	463	478	463
query6	263	164	174	164
query7	4594	294	280	280
query8	233	191	192	191
query9	8598	2385	2356	2356
query10	432	285	273	273
query11	11009	10132	10165	10132
query12	119	84	83	83
query13	1641	369	369	369
query14	10389	7820	7474	7474
query15	219	173	168	168
query16	7614	485	461	461
query17	1164	547	542	542
query18	1718	292	279	279
query19	197	158	152	152
query20	89	81	85	81
query21	211	136	133	133
query22	4307	4006	4165	4006
query23	33980	33735	33679	33679
query24	11089	2931	2955	2931
query25	615	406	403	403
query26	726	152	152	152
query27	2276	273	272	272
query28	6231	2031	2007	2007
query29	919	741	611	611
query30	255	151	153	151
query31	965	752	748	748
query32	98	54	59	54
query33	694	355	321	321
query34	877	472	477	472
query35	895	767	763	763
query36	1136	990	968	968
query37	139	81	79	79
query38	2940	2861	2801	2801
query39	873	880	846	846
query40	196	120	112	112
query41	44	45	43	43
query42	114	99	97	97
query43	510	476	468	468
query44	1045	707	720	707
query45	193	162	157	157
query46	1066	757	718	718
query47	1895	1772	1791	1772
query48	362	287	291	287
query49	832	404	420	404
query50	759	387	390	387
query51	6743	6675	6666	6666
query52	101	94	87	87
query53	352	286	281	281
query54	880	437	437	437
query55	75	73	72	72
query56	310	269	259	259
query57	1173	1047	1055	1047
query58	251	255	253	253
query59	2827	2538	2610	2538
query60	286	277	319	277
query61	94	98	91	91
query62	788	643	643	643
query63	305	286	286	286
query64	9111	2232	1683	1683
query65	3168	3092	3091	3091
query66	775	325	325	325
query67	15510	15045	14781	14781
query68	5624	528	523	523
query69	699	449	351	351
query70	1212	1173	1114	1114
query71	480	278	280	278
query72	8727	5640	5690	5640
query73	788	320	323	320
query74	6074	5725	5581	5581
query75	4293	2671	2681	2671
query76	3922	947	943	943
query77	678	306	305	305
query78	9997	9027	9001	9001
query79	8127	520	516	516
query80	1708	489	478	478
query81	587	224	221	221
query82	1502	131	134	131
query83	350	163	159	159
query84	276	83	88	83
query85	1464	298	322	298
query86	437	323	280	280
query87	3284	3120	3067	3067
query88	4953	2359	2369	2359
query89	519	388	397	388
query90	1791	199	201	199
query91	128	99	99	99
query92	59	49	49	49
query93	6576	500	488	488
query94	1126	264	266	264
query95	407	317	318	317
query96	612	271	268	268
query97	3216	3050	3033	3033
query98	215	194	191	191
query99	1629	1277	1256	1256
Total cold run time: 287379 ms
Total hot run time: 173399 ms

@doris-robot

doris-robot commented Jul 23, 2024

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.06 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c4658686e77f475dc7114aff54392b5274db5b9f, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.08
query5	0.50	0.47	0.48
query6	1.14	0.73	0.71
query7	0.02	0.02	0.01
query8	0.06	0.04	0.05
query9	0.54	0.48	0.48
query10	0.54	0.55	0.53
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.61	0.59	0.58
query14	0.75	0.77	0.80
query15	0.86	0.82	0.81
query16	0.36	0.34	0.36
query17	0.98	1.02	0.99
query18	0.22	0.21	0.23
query19	1.77	1.68	1.71
query20	0.02	0.01	0.01
query21	15.41	0.77	0.67
query22	3.91	7.05	2.54
query23	18.27	1.31	1.21
query24	2.07	0.24	0.20
query25	0.16	0.10	0.09
query26	0.31	0.22	0.22
query27	0.46	0.25	0.23
query28	13.60	1.02	1.00
query29	12.89	3.27	3.29
query30	0.25	0.06	0.06
query31	2.86	0.40	0.40
query32	3.27	0.49	0.47
query33	2.87	2.96	2.89
query34	17.10	4.34	4.44
query35	4.43	4.46	4.38
query36	0.66	0.46	0.47
query37	0.18	0.16	0.15
query38	0.16	0.15	0.14
query39	0.04	0.04	0.04
query40	0.15	0.13	0.13
query41	0.10	0.04	0.05
query42	0.05	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.94 s
Total hot run time: 31.06 s

@gavinchou
gavinchou merged commit 63db4e0 into apache:master Jul 23, 2024
dataroaring pushed a commit that referenced this pull request Jul 24, 2024
@yagagagaga @dataroaring
[chore](audit) Optimize the SQL (insert into values) length in audit …
9a8776f 
…logs (#37894)

## Proposed changes

- Add annotation information to the `insert into value` statement
- Let the SQL length in the audit log be controlled by
`GlobalVariable.auditPluginMaxSqlLength`

Here is the result of this PR:
<img width="1661" alt="image"
src="https://github.com/user-attachments/assets/b48466b5-6fa7-4d0d-b3a5-d3b9b3444162">


If the length of sql greater than
`GlobalVariable.auditPluginMaxSqlLength`, the sql will be truncate but
it still have a comment if the sql is `insert into value`

```sql
set global audit_plugin_max_sql_length=30;
insert into test.t values (1, 'Tom'), (2, 'Jelly'), (3, 'Damn');
```

<img width="1647" alt="image"
src="https://github.com/user-attachments/assets/00602adf-3f46-471e-b7b1-c1c73bc2be2a">
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Sep 10, 2024
@morrySnow
[chore](audit) let line comment work well
767b5ab 
should replace new line character with '\n', not ' '

intro by apache#37894
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Sep 10, 2024
@morrySnow
[chore](audit) let line comment work well
c22e0ee 
should replace new line character with '\n', not ' '

intro by apache#37894
@morrySnow morrySnow mentioned this pull request Sep 10, 2024
Merged
yagagagaga added a commit to yagagagaga/doris that referenced this pull request Oct 21, 2024
@yagagagaga
[chore](audit) Optimize the SQL (insert into values) length in audit …
aa823de 
…logs (apache#37894)
yagagagaga added a commit to yagagagaga/doris that referenced this pull request Oct 21, 2024
@yagagagaga
[chore](audit) Optimize the SQL (insert into values) length in audit …
01a93a9 
…logs (apache#37894)
@yagagagaga yagagagaga mentioned this pull request Oct 21, 2024
Merged
yiguolei pushed a commit that referenced this pull request Oct 22, 2024
@yagagagaga
[cherry-pick][chore](audit) Optimize the SQL (insert into values) len…
104d427 
…gth in audit logs (#37894) and let line comment work well (#40599) (#42186)

## Proposed changes

cherry-pick from master #37894 and #40599

<!--Describe your changes.-->
@yagagagaga
yagagagaga deleted the chore_20240716 branch February 25, 2025 02:34
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@yagagagaga @HappenLee
[chore](audit) Optimize the SQL (insert into values) length in audit …
c7a7aca 
…logs (apache#37894)

## Proposed changes

- Add annotation information to the `insert into value` statement
- Let the SQL length in the audit log be controlled by
`GlobalVariable.auditPluginMaxSqlLength`

Here is the result of this PR:
<img width="1661" alt="image"
src="https://github.com/user-attachments/assets/b48466b5-6fa7-4d0d-b3a5-d3b9b3444162">


If the length of sql greater than
`GlobalVariable.auditPluginMaxSqlLength`, the sql will be truncate but
it still have a comment if the sql is `insert into value`

```sql
set global audit_plugin_max_sql_length=30;
insert into test.t values (1, 'Tom'), (2, 'Jelly'), (3, 'Damn');
```

<img width="1647" alt="image"
src="https://github.com/user-attachments/assets/00602adf-3f46-471e-b7b1-c1c73bc2be2a">
@morningman morningman mentioned this pull request Nov 3, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gavinchou gavinchou gavinchou approved these changes
@luwei16 luwei16 luwei16 approved these changes
@morrySnow morrySnow morrySnow left review comments
+1 more reviewer
@LiBinfeng-01 LiBinfeng-01 LiBinfeng-01 approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.7-merged dev/3.0.1-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@yagagagaga @gavinchou @doris-robot @luwei16 @LiBinfeng-01 @morrySnow @liaoxin01 @wm1581066 @dataroaring