Make unit tests to only use dynamically allocated ports #5486

merlimat · 2019-10-28T21:32:43Z

Motivation

Use dynamically allocated ports in unit tests execution to ensure no port conflicts between multiple threads/processes.

aahmed-se · 2019-10-28T23:35:37Z

run java8 tests

jiazhai

great to have this feature.

jiazhai · 2019-10-29T03:17:26Z

run java8 tests
run integration tests

codelipenghui

👍

codelipenghui · 2019-10-29T10:50:17Z

run java8 tests

aahmed-se · 2019-10-30T19:21:47Z

run java8 tests.

aahmed-se · 2019-10-30T21:19:48Z

run java8 tests

…RunTest

merlimat · 2020-01-05T01:12:56Z

run java8 tests

tuteng · 2020-01-05T02:19:32Z

run java8 tests

tuteng · 2020-01-05T03:03:40Z

run java8 tests

tuteng · 2020-01-05T03:57:30Z

run java8 tests

tuteng · 2020-01-05T05:30:03Z

run java8 tests

tuteng · 2020-01-05T08:25:34Z

Unit testing seems to have some problems:

2020-01-05\T\06:21:26.542 [ERROR] testAuthentication(org.apache.pulsar.broker.authentication.ProxySaslAuthenticationTest)  Time elapsed: 33.912 s  <<< FAILURE!
org.apache.pulsar.client.api.PulsarClientException: java.util.concurrent.ExecutionException: org.apache.pulsar.client.api.PulsarClientException: Connection already closed
	at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:717)
	at org.apache.pulsar.client.impl.ProducerBuilderImpl.create(ProducerBuilderImpl.java:93)
	at org.apache.pulsar.broker.authentication.ProxySaslAuthenticationTest.testAuthentication(ProxySaslAuthenticationTest.java:255)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:54)
	at org.testng.internal.InvokeMethodRunnable.run(InvokeMethodRunnable.java:44)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

*Motivation* A regression was introduced in apache#5486. If websocket service as running as part of pulsar standalone, the cluster data is set with null service urls. This causes service url is not set correctly in the pulsar client and an illegal argument exception ("Param serviceUrl must not be blank.") will be thrown. *Modifications* 1. Pass `null` when constructing the websocket service. So the local cluster data can be refreshed when creating pulsar client. 2. Set the cluster data after both broker service and web service started and ports are allocated. *Test* Verified locally and need to figure out how to automate this in integration tests

*Motivation* Fixes #5997 Fixes #6079 A regression was introduced in #5486. If websocket service as running as part of pulsar standalone, the cluster data is set with null service urls. This causes service url is not set correctly in the pulsar client and an illegal argument exception ("Param serviceUrl must not be blank.") will be thrown. *Modifications* 1. Pass `null` when constructing the websocket service. So the local cluster data can be refreshed when creating pulsar client. 2. Set the cluster data after both broker service and web service started and ports are allocated.

) *Motivation* Fixes apache#5997 Fixes apache#6079 A regression was introduced in apache#5486. If websocket service as running as part of pulsar standalone, the cluster data is set with null service urls. This causes service url is not set correctly in the pulsar client and an illegal argument exception ("Param serviceUrl must not be blank.") will be thrown. *Modifications* 1. Pass `null` when constructing the websocket service. So the local cluster data can be refreshed when creating pulsar client. 2. Set the cluster data after both broker service and web service started and ports are allocated.

) *Motivation* Fixes apache#5997 Fixes apache#6079 A regression was introduced in apache#5486. If websocket service as running as part of pulsar standalone, the cluster data is set with null service urls. This causes service url is not set correctly in the pulsar client and an illegal argument exception ("Param serviceUrl must not be blank.") will be thrown. *Modifications* 1. Pass `null` when constructing the websocket service. So the local cluster data can be refreshed when creating pulsar client. 2. Set the cluster data after both broker service and web service started and ports are allocated. (cherry picked from commit 49a9897)

The two failures for this have been fixed by: - 8df1f9c Do not use timed receive in BrokerClientIntegrationTest.testUnsupportedBatchMessageConsumer (apache#5333) - f141efa Make unit tests to only use dynamically allocated ports (apache#5486)

*Motivation* Fixes #5997 Fixes #6079 A regression was introduced in #5486. If websocket service as running as part of pulsar standalone, the cluster data is set with null service urls. This causes service url is not set correctly in the pulsar client and an illegal argument exception ("Param serviceUrl must not be blank.") will be thrown. *Modifications* 1. Pass `null` when constructing the websocket service. So the local cluster data can be refreshed when creating pulsar client. 2. Set the cluster data after both broker service and web service started and ports are allocated. (cherry picked from commit 49a9897)

) *Motivation* Fixes apache#5997 Fixes apache#6079 A regression was introduced in apache#5486. If websocket service as running as part of pulsar standalone, the cluster data is set with null service urls. This causes service url is not set correctly in the pulsar client and an illegal argument exception ("Param serviceUrl must not be blank.") will be thrown. *Modifications* 1. Pass `null` when constructing the websocket service. So the local cluster data can be refreshed when creating pulsar client. 2. Set the cluster data after both broker service and web service started and ports are allocated. (cherry picked from commit 49a9897)

* Make unit tests to only use dynamically allocated ports * Fixed proxy tests * Fixed zk test utils after merge * Fixed multi host client test * Fix for testConcurrentConsumerReceiveWhileReconnect * Increased timeout on testPulsarSourceLocalRunWithFile * Fixed newer test to also avoid port manager * Fixed test race condition with thread starting in PulsarFunctionLocalRunTest * Fixed ProxyWithAuthorizationNegTest * Fixed ProxySaslAuthenticationTest

) *Motivation* Fixes apache#5997 Fixes apache#6079 A regression was introduced in apache#5486. If websocket service as running as part of pulsar standalone, the cluster data is set with null service urls. This causes service url is not set correctly in the pulsar client and an illegal argument exception ("Param serviceUrl must not be blank.") will be thrown. *Modifications* 1. Pass `null` when constructing the websocket service. So the local cluster data can be refreshed when creating pulsar client. 2. Set the cluster data after both broker service and web service started and ports are allocated.

Make unit tests to only use dynamically allocated ports

679b8af

merlimat added the area/test label Oct 28, 2019

merlimat added this to the 2.5.0 milestone Oct 28, 2019

merlimat requested review from aahmed-se, ivankelly, jerrypeng, rdhabalia and sijie October 28, 2019 21:32

merlimat self-assigned this Oct 28, 2019

aahmed-se approved these changes Oct 28, 2019

View reviewed changes

Fixed proxy tests

f9cb6fb

jiazhai approved these changes Oct 29, 2019

View reviewed changes

codelipenghui approved these changes Oct 29, 2019

View reviewed changes

sijie modified the milestones: 2.5.0, 2.6.0 Nov 25, 2019

merlimat added 8 commits January 4, 2020 06:44

Merge remote-tracking branch 'apache/master' into unit-tests-port-0

718bcd7

Fixed zk test utils after merge

1998ff6

Fixed multi host client test

41002de

Fix for testConcurrentConsumerReceiveWhileReconnect

61f4979

Increased timeout on testPulsarSourceLocalRunWithFile

ae9565b

Fixed newer test to also avoid port manager

066531f

Fixed test race condition with thread starting in PulsarFunctionLocal…

0a19b50

…RunTest

Fixed ProxyWithAuthorizationNegTest

5cc35d5

Fixed ProxySaslAuthenticationTest

898ab48

merlimat merged commit f141efa into apache:master Jan 5, 2020

merlimat deleted the unit-tests-port-0 branch January 5, 2020 20:44

sijie modified the milestones: 2.6.0, 2.5.1, 2.5.0 Jan 6, 2020

This was referenced Jan 21, 2020

Cannot create producer via websocket in standalone mode #6079

Closed

websocket meet 404 when create producer in standalone mode #5997

Closed

sijie mentioned this pull request Jan 21, 2020

[Websocket] Websocket doesn't set the correct cluster data #6102

Merged

Make unit tests to only use dynamically allocated ports #5486

Make unit tests to only use dynamically allocated ports #5486

Uh oh!

Conversation

merlimat commented Oct 28, 2019

Motivation

Uh oh!

aahmed-se commented Oct 28, 2019

Uh oh!

jiazhai left a comment

Choose a reason for hiding this comment

Uh oh!

jiazhai commented Oct 29, 2019

Uh oh!

codelipenghui left a comment

Choose a reason for hiding this comment

Uh oh!

codelipenghui commented Oct 29, 2019

Uh oh!

aahmed-se commented Oct 30, 2019

Uh oh!

aahmed-se commented Oct 30, 2019

Uh oh!

merlimat commented Jan 5, 2020

Uh oh!

tuteng commented Jan 5, 2020

Uh oh!

tuteng commented Jan 5, 2020

Uh oh!

tuteng commented Jan 5, 2020

Uh oh!

tuteng commented Jan 5, 2020

Uh oh!

tuteng commented Jan 5, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants