Skip to content

[fix](third party) fix hang when destroy of rdkafka instances#44913

Merged
liaoxin01 merged 1 commit intoapache:masterfrom
sollhui:rdlibkafka_fix_patch
Dec 3, 2024
Merged

[fix](third party) fix hang when destroy of rdkafka instances#44913
liaoxin01 merged 1 commit intoapache:masterfrom
sollhui:rdlibkafka_fix_patch

Conversation

@sollhui
Copy link
Copy Markdown
Contributor

@sollhui sollhui commented Dec 3, 2024 •
edited
Loading

What problem does this PR solve?

Related PR: confluentinc/librdkafka#4724

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link
Copy Markdown

doris-robot commented Dec 3, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@sollhui
Copy link
Copy Markdown
Contributor Author

sollhui commented Dec 3, 2024

run buildall

@doris-robot
Copy link
Copy Markdown

doris-robot commented Dec 3, 2024

TeamCity be ut coverage result:
Function Coverage: 38.51% (10007/25988)
Line Coverage: 29.50% (83779/284018)
Region Coverage: 28.63% (43104/150566)
Branch Coverage: 25.23% (21904/86818)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3d140c546ade6d73217b709807074c47cf1d15a0_3d140c546ade6d73217b709807074c47cf1d15a0/report/index.html

@doris-robot
Copy link
Copy Markdown

doris-robot commented Dec 3, 2024

TPC-H: Total hot run time: 40169 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3d140c546ade6d73217b709807074c47cf1d15a0, data reload: false

------ Round 1 ----------------------------------
q1	17594	7757	7363	7363
q2	2039	185	175	175
q3	10532	1151	1195	1151
q4	10235	735	807	735
q5	7588	2760	2716	2716
q6	235	145	147	145
q7	996	619	613	613
q8	9255	1916	1942	1916
q9	6707	6483	6532	6483
q10	6974	2321	2332	2321
q11	476	264	259	259
q12	414	215	215	215
q13	17754	3035	2986	2986
q14	259	212	220	212
q15	569	534	542	534
q16	660	581	578	578
q17	985	584	532	532
q18	7580	6694	6857	6694
q19	1341	943	954	943
q20	462	182	177	177
q21	4087	3111	3243	3111
q22	387	310	315	310
Total cold run time: 107129 ms
Total hot run time: 40169 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7247	7263	7256	7256
q2	332	228	232	228
q3	2915	2865	3028	2865
q4	2157	1891	1835	1835
q5	5612	5744	5664	5664
q6	219	139	140	139
q7	2209	1785	1855	1785
q8	3475	3600	3561	3561
q9	9090	8983	9082	8983
q10	3615	3546	3541	3541
q11	604	518	509	509
q12	816	624	625	624
q13	11585	3210	3222	3210
q14	302	278	267	267
q15	572	521	520	520
q16	671	647	643	643
q17	1808	1591	1592	1591
q18	7790	7635	7611	7611
q19	1636	1476	1427	1427
q20	2053	1788	1818	1788
q21	5381	5445	5279	5279
q22	643	569	553	553
Total cold run time: 70732 ms
Total hot run time: 59879 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Dec 3, 2024

TPC-DS: Total hot run time: 191211 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3d140c546ade6d73217b709807074c47cf1d15a0, data reload: false

query1	996	381	372	372
query2	6529	2077	2050	2050
query3	6710	205	212	205
query4	33663	23508	23579	23508
query5	4397	455	446	446
query6	289	186	182	182
query7	4621	302	312	302
query8	301	233	231	231
query9	9649	2694	2695	2694
query10	469	251	251	251
query11	18261	15240	15314	15240
query12	159	102	105	102
query13	1690	402	406	402
query14	10188	7234	7470	7234
query15	317	196	190	190
query16	8151	458	453	453
query17	1829	580	557	557
query18	2144	336	286	286
query19	363	147	147	147
query20	120	110	115	110
query21	204	100	99	99
query22	4658	4395	4285	4285
query23	35003	34294	34187	34187
query24	11247	2398	2464	2398
query25	643	369	373	369
query26	1587	149	151	149
query27	2808	272	279	272
query28	8066	2460	2424	2424
query29	956	405	398	398
query30	293	179	143	143
query31	1024	823	846	823
query32	98	55	55	55
query33	774	290	285	285
query34	976	500	531	500
query35	897	746	716	716
query36	1102	921	950	921
query37	145	72	69	69
query38	4336	4206	4190	4190
query39	1459	1435	1471	1435
query40	280	97	99	97
query41	45	43	44	43
query42	116	97	96	96
query43	524	502	484	484
query44	1299	792	793	792
query45	191	164	163	163
query46	1128	722	719	719
query47	1949	1867	1871	1867
query48	410	301	318	301
query49	1234	394	384	384
query50	808	397	374	374
query51	7321	7299	7098	7098
query52	99	90	90	90
query53	258	185	177	177
query54	1222	408	398	398
query55	81	78	77	77
query56	257	242	238	238
query57	1265	1123	1129	1123
query58	223	214	217	214
query59	3145	2903	2991	2903
query60	262	234	239	234
query61	115	111	111	111
query62	868	667	665	665
query63	216	178	179	178
query64	5176	660	632	632
query65	3251	3203	3252	3203
query66	1458	310	311	310
query67	16097	15741	15514	15514
query68	5280	554	552	552
query69	420	244	253	244
query70	1201	1067	1123	1067
query71	339	243	251	243
query72	6432	4109	4077	4077
query73	765	357	362	357
query74	10379	9079	9035	9035
query75	3679	2705	2677	2677
query76	3106	1122	1091	1091
query77	625	269	271	269
query78	10400	9479	9431	9431
query79	1595	593	598	593
query80	1141	415	435	415
query81	509	227	238	227
query82	926	117	113	113
query83	248	141	143	141
query84	240	118	74	74
query85	1322	298	299	298
query86	365	303	288	288
query87	4724	4543	4473	4473
query88	3376	2225	2178	2178
query89	407	294	303	294
query90	2062	188	186	186
query91	136	103	102	102
query92	57	50	51	50
query93	1137	548	544	544
query94	1099	300	292	292
query95	370	245	252	245
query96	608	283	276	276
query97	2902	2704	2709	2704
query98	221	196	203	196
query99	1572	1314	1321	1314
Total cold run time: 304063 ms
Total hot run time: 191211 ms

@doris-robot
Copy link
Copy Markdown

doris-robot commented Dec 3, 2024

ClickBench: Total hot run time: 32.66 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3d140c546ade6d73217b709807074c47cf1d15a0, data reload: false

query1	0.04	0.04	0.03
query2	0.06	0.04	0.03
query3	0.24	0.07	0.07
query4	1.62	0.11	0.10
query5	0.43	0.40	0.41
query6	1.14	0.67	0.66
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.58	0.51	0.51
query10	0.55	0.56	0.54
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.61	0.59
query14	2.80	2.85	2.86
query15	0.90	0.83	0.84
query16	0.39	0.39	0.39
query17	1.04	1.00	1.07
query18	0.22	0.21	0.21
query19	1.90	1.83	1.95
query20	0.01	0.01	0.01
query21	15.36	0.59	0.58
query22	2.61	2.52	1.57
query23	17.07	0.82	0.75
query24	3.09	1.21	1.67
query25	0.28	0.20	0.14
query26	0.31	0.14	0.15
query27	0.04	0.05	0.04
query28	10.09	1.09	1.07
query29	12.58	3.22	3.20
query30	0.25	0.07	0.06
query31	2.86	0.39	0.37
query32	3.26	0.46	0.46
query33	2.96	3.01	3.07
query34	16.89	4.49	4.49
query35	4.51	4.51	4.50
query36	0.67	0.48	0.47
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.18 s
Total hot run time: 32.66 s

dataroaring
dataroaring approved these changes Dec 3, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring added usercase Important user case type label dev/2.1.x dev/3.0.x labels Dec 3, 2024
@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Dec 3, 2024
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Dec 3, 2024

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Dec 3, 2024

PR approved by anyone and no changes requested.

liaoxin01
liaoxin01 approved these changes Dec 3, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@liaoxin01 liaoxin01 merged commit 802ba71 into apache:master Dec 3, 2024
github-actions Bot pushed a commit that referenced this pull request Dec 3, 2024
@sollhui
[fix](third party) fix hang when destroy of rdkafka instances (#44913)
7432610 
Related PR: confluentinc/librdkafka#4724
@github-actions github-actions Bot mentioned this pull request Dec 3, 2024
Merged
github-actions Bot pushed a commit that referenced this pull request Dec 3, 2024
@sollhui
[fix](third party) fix hang when destroy of rdkafka instances (#44913)
74e685b 
Related PR: confluentinc/librdkafka#4724
@github-actions github-actions Bot mentioned this pull request Dec 3, 2024
Merged
dataroaring pushed a commit that referenced this pull request Dec 5, 2024
@github-actions @sollhui
branch-3.0: [fix](third party) fix hang when destroy of rdkafka insta…
bbfb363 
…nces #44913 (#44950)

Cherry-picked from #44913

Co-authored-by: hui lai <[email protected]>
dataroaring pushed a commit that referenced this pull request Dec 5, 2024
@sollhui @dataroaring
[fix](third party) fix hang when destroy of rdkafka instances (#44913)
983edb6 
Related PR: confluentinc/librdkafka#4724
yiguolei pushed a commit that referenced this pull request Dec 9, 2024
@github-actions @sollhui @hello-stephen
branch-2.1: [fix](third party) fix hang when destroy of rdkafka insta…
a8b5125 
…nces #44913 (#44951)

Cherry-picked from #44913

Co-authored-by: hui lai <[email protected]>
Co-authored-by: Dongyang Li <[email protected]>
hubgeter pushed a commit to hubgeter/doris that referenced this pull request Mar 12, 2025
@sollhui
(selectdb-cloud) fix hang when destroy of rdkafka instances (apache#4…
629d28c 
…4913) (apache#3706)

pick apache#44913
deardeng pushed a commit to deardeng/incubator-doris that referenced this pull request Dec 19, 2025
@github-actions @sollhui @hello-stephen
branch-2.1: [fix](third party) fix hang when destroy of rdkafka insta…
b86e538 
…nces apache#44913 (apache#44951)

Cherry-picked from apache#44913

Co-authored-by: hui lai <[email protected]>
Co-authored-by: Dongyang Li <[email protected]>
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Apr 24, 2026
@sollhui @HappenLee
[fix](third party) fix hang when destroy of rdkafka instances (apache…
96597e8 
…#44913)

Related PR: confluentinc/librdkafka#4724
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@liaoxin01 liaoxin01 liaoxin01 approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@sollhui @doris-robot @liaoxin01 @dataroaring @yiguolei