This repository was archived by the owner on Mar 3, 2023. It is now read-only.
download/upload multiple files with thread pool #26
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Yan Zhang <[email protected]>
Contributor
Contributor
Author
|
Thanks for the information, I just signed the CLA. And a scanned copy has been sent to |
Contributor
|
@mosabua could you perform a release on Maven Central ASAP so that I can consume the new build in m2e, pretty please? |
eclipsewebmaster
pushed a commit
to eclipse-m2e/m2e-core
that referenced
this pull request
Jun 30, 2020
aether-connector-okhttp 0.17.6.999 is a pseudo stable build manually deployed to Bintray, until 0.17.7 is published to Maven Central (hopefully before 2020-03-RC1). That build contains takari/aether-connector-okhttp#26 Change-Id: Icb2c3cf7d99702f9838d52b10a1e1b31b84a0ef6 Signed-off-by: Fred Bricon <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes #25
By default, if there are more than one
GetTaskorPutTask, it uses a ThreadPool to execute. Otherwise the task is executed directly. The default max of threads is 5, and it can be set throughmaven.aertifact.threads, which aligns with Maven CLI, see implemetation in maven-resolverBelow is a comparison of the performance. TL;DR: it saves 41s by this PR, when downloading 67 .jar files.
CPU: Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz, 2 cores 4 threads.
OS: macOS Catalina 10.15.3
RAM: 8G
Steps:
AetherRepositoryConnector.get()call which was to download 67 artifacts.Mono-threaded
click to view logs
Multi-threaded (5 threads by default)
click to view logs
18:49:09.969549 Method 'io.takari.aether.connector.AetherRepositoryConnector.get()' entered 18:49:10.092469 Start: org/springframework/boot/spring-boot-starter-web/2.2.4.RELEASE/spring-boot-starter-web-2.2.4.RELEASE.jar 18:49:10.206528 Start: org/springframework/boot/spring-boot-starter/2.2.4.RELEASE/spring-boot-starter-2.2.4.RELEASE.jar 18:49:10.326366 Start: org/springframework/boot/spring-boot/2.2.4.RELEASE/spring-boot-2.2.4.RELEASE.jar 18:49:10.450838 Start: org/springframework/boot/spring-boot-autoconfigure/2.2.4.RELEASE/spring-boot-autoconfigure-2.2.4.RELEASE.jar 18:49:10.565010 Start: org/springframework/boot/spring-boot-starter-logging/2.2.4.RELEASE/spring-boot-starter-logging-2.2.4.RELEASE.jar 18:49:10.844090 End: org/springframework/boot/spring-boot-starter-web/2.2.4.RELEASE/spring-boot-starter-web-2.2.4.RELEASE.jar 18:49:10.956967 Start: ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar 18:49:11.459868 End: org/springframework/boot/spring-boot-starter/2.2.4.RELEASE/spring-boot-starter-2.2.4.RELEASE.jar 18:49:11.586152 Start: ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar 18:49:11.932689 End: org/springframework/boot/spring-boot-starter-logging/2.2.4.RELEASE/spring-boot-starter-logging-2.2.4.RELEASE.jar 18:49:12.058379 Start: org/apache/logging/log4j/log4j-to-slf4j/2.12.1/log4j-to-slf4j-2.12.1.jar 18:49:13.004132 End: ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar 18:49:13.117093 Start: org/apache/logging/log4j/log4j-api/2.12.1/log4j-api-2.12.1.jar 18:49:13.183003 End: org/apache/logging/log4j/log4j-to-slf4j/2.12.1/log4j-to-slf4j-2.12.1.jar 18:49:13.356291 Start: org/slf4j/jul-to-slf4j/1.7.30/jul-to-slf4j-1.7.30.jar 18:49:13.796988 End: ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar 18:49:13.945719 Start: jakarta/annotation/jakarta.annotation-api/1.3.5/jakarta.annotation-api-1.3.5.jar Unable to evaluate the expression "String.format("%s End: %s", java.time.LocalTime.now(), path)" : Object has been collected Unable to evaluate the expression "String.format("%s End: %s", java.time.LocalTime.now(), path)" : Object has been collected 18:49:14.190490 End: org/slf4j/jul-to-slf4j/1.7.30/jul-to-slf4j-1.7.30.jar 18:49:14.405423 End: org/apache/logging/log4j/log4j-api/2.12.1/log4j-api-2.12.1.jar 18:49:14.407579 Start: org/yaml/snakeyaml/1.25/snakeyaml-1.25.jar 18:49:14.409297 Start: org/springframework/boot/spring-boot-starter-json/2.2.4.RELEASE/spring-boot-starter-json-2.2.4.RELEASE.jar 18:49:14.624653 Start: com/fasterxml/jackson/core/jackson-databind/2.10.2/jackson-databind-2.10.2.jar 18:49:14.773289 End: jakarta/annotation/jakarta.annotation-api/1.3.5/jakarta.annotation-api-1.3.5.jar 18:49:14.820753 Start: com/fasterxml/jackson/core/jackson-annotations/2.10.2/jackson-annotations-2.10.2.jar 18:49:15.002625 Start: com/fasterxml/jackson/core/jackson-core/2.10.2/jackson-core-2.10.2.jar 18:49:15.408197 End: org/yaml/snakeyaml/1.25/snakeyaml-1.25.jar 18:49:15.514662 Start: com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.10.2/jackson-datatype-jdk8-2.10.2.jar 18:49:15.660148 End: org/springframework/boot/spring-boot-starter-json/2.2.4.RELEASE/spring-boot-starter-json-2.2.4.RELEASE.jar 18:49:15.758737 Start: com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.10.2/jackson-datatype-jsr310-2.10.2.jar 18:49:16.074558 End: com/fasterxml/jackson/core/jackson-annotations/2.10.2/jackson-annotations-2.10.2.jar 18:49:16.116495 End: com/fasterxml/jackson/core/jackson-core/2.10.2/jackson-core-2.10.2.jar 18:49:16.267152 Start: com/fasterxml/jackson/module/jackson-module-parameter-names/2.10.2/jackson-module-parameter-names-2.10.2.jar 18:49:16.333148 Start: org/springframework/boot/spring-boot-starter-tomcat/2.2.4.RELEASE/spring-boot-starter-tomcat-2.2.4.RELEASE.jar 18:49:16.424444 End: com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.10.2/jackson-datatype-jdk8-2.10.2.jar 18:49:16.477405 End: com/fasterxml/jackson/core/jackson-databind/2.10.2/jackson-databind-2.10.2.jar 18:49:16.593745 End: com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.10.2/jackson-datatype-jsr310-2.10.2.jar 18:49:16.695396 Start: org/apache/tomcat/embed/tomcat-embed-core/9.0.30/tomcat-embed-core-9.0.30.jar 18:49:16.741506 Start: org/apache/tomcat/embed/tomcat-embed-el/9.0.30/tomcat-embed-el-9.0.30.jar 18:49:16.880739 Start: org/apache/tomcat/embed/tomcat-embed-websocket/9.0.30/tomcat-embed-websocket-9.0.30.jar 18:49:16.961335 End: com/fasterxml/jackson/module/jackson-module-parameter-names/2.10.2/jackson-module-parameter-names-2.10.2.jar 18:49:17.103348 Start: org/springframework/boot/spring-boot-starter-validation/2.2.4.RELEASE/spring-boot-starter-validation-2.2.4.RELEASE.jar 18:49:17.199138 End: org/springframework/boot/spring-boot-starter-tomcat/2.2.4.RELEASE/spring-boot-starter-tomcat-2.2.4.RELEASE.jar 18:49:17.405159 Start: jakarta/validation/jakarta.validation-api/2.0.2/jakarta.validation-api-2.0.2.jar 18:49:17.999827 End: org/springframework/boot/spring-boot-starter-validation/2.2.4.RELEASE/spring-boot-starter-validation-2.2.4.RELEASE.jar 18:49:18.099158 Start: org/hibernate/validator/hibernate-validator/6.0.18.Final/hibernate-validator-6.0.18.Final.jar 18:49:18.138279 End: org/apache/tomcat/embed/tomcat-embed-el/9.0.30/tomcat-embed-el-9.0.30.jar 18:49:18.198136 End: org/apache/tomcat/embed/tomcat-embed-websocket/9.0.30/tomcat-embed-websocket-9.0.30.jar 18:49:18.407345 End: jakarta/validation/jakarta.validation-api/2.0.2/jakarta.validation-api-2.0.2.jar 18:49:18.580660 Start: org/jboss/logging/jboss-logging/3.4.1.Final/jboss-logging-3.4.1.Final.jar 18:49:18.604969 Start: com/fasterxml/classmate/1.5.1/classmate-1.5.1.jar 18:49:18.776157 Start: org/springframework/spring-web/5.2.3.RELEASE/spring-web-5.2.3.RELEASE.jar 18:49:19.036148 End: org/apache/tomcat/embed/tomcat-embed-core/9.0.30/tomcat-embed-core-9.0.30.jar 18:49:19.198550 Start: org/springframework/spring-beans/5.2.3.RELEASE/spring-beans-5.2.3.RELEASE.jar 18:49:19.294051 End: org/hibernate/validator/hibernate-validator/6.0.18.Final/hibernate-validator-6.0.18.Final.jar 18:49:19.512792 End: org/jboss/logging/jboss-logging/3.4.1.Final/jboss-logging-3.4.1.Final.jar 18:49:19.760341 Start: org/springframework/spring-webmvc/5.2.3.RELEASE/spring-webmvc-5.2.3.RELEASE.jar 18:49:19.792146 End: com/fasterxml/classmate/1.5.1/classmate-1.5.1.jar 18:49:20.079080 Start: org/springframework/spring-aop/5.2.3.RELEASE/spring-aop-5.2.3.RELEASE.jar 18:49:20.175970 End: org/springframework/spring-web/5.2.3.RELEASE/spring-web-5.2.3.RELEASE.jar 18:49:20.241481 End: org/springframework/spring-beans/5.2.3.RELEASE/spring-beans-5.2.3.RELEASE.jar 18:49:20.307129 Start: org/springframework/spring-context/5.2.3.RELEASE/spring-context-5.2.3.RELEASE.jar 18:49:20.619873 Start: org/springframework/spring-expression/5.2.3.RELEASE/spring-expression-5.2.3.RELEASE.jar 18:49:20.621381 Start: org/springframework/boot/spring-boot-starter-test/2.2.4.RELEASE/spring-boot-starter-test-2.2.4.RELEASE.jar 18:49:20.818995 End: org/springframework/spring-webmvc/5.2.3.RELEASE/spring-webmvc-5.2.3.RELEASE.jar 18:49:20.982064 Start: org/springframework/boot/spring-boot-test/2.2.4.RELEASE/spring-boot-test-2.2.4.RELEASE.jar 18:49:21.200623 End: org/springframework/spring-aop/5.2.3.RELEASE/spring-aop-5.2.3.RELEASE.jar 18:49:21.297930 Start: org/springframework/boot/spring-boot-test-autoconfigure/2.2.4.RELEASE/spring-boot-test-autoconfigure-2.2.4.RELEASE.jar 18:49:21.409021 End: org/springframework/boot/spring-boot-starter-test/2.2.4.RELEASE/spring-boot-starter-test-2.2.4.RELEASE.jar 18:49:21.519048 Start: com/jayway/jsonpath/json-path/2.4.0/json-path-2.4.0.jar 18:49:21.599228 End: org/springframework/spring-context/5.2.3.RELEASE/spring-context-5.2.3.RELEASE.jar 18:49:21.618259 End: org/springframework/spring-expression/5.2.3.RELEASE/spring-expression-5.2.3.RELEASE.jar 18:49:21.783472 Start: net/minidev/json-smart/2.3/json-smart-2.3.jar 18:49:21.832114 Start: net/minidev/accessors-smart/1.2/accessors-smart-1.2.jar 18:49:21.986881 End: org/springframework/boot/spring-boot-test/2.2.4.RELEASE/spring-boot-test-2.2.4.RELEASE.jar 18:49:22.080938 Start: org/ow2/asm/asm/5.0.4/asm-5.0.4.jar 18:49:22.181650 End: org/springframework/boot/spring-boot-test-autoconfigure/2.2.4.RELEASE/spring-boot-test-autoconfigure-2.2.4.RELEASE.jar 18:49:22.273535 Start: org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.jar 18:49:22.383942 End: com/jayway/jsonpath/json-path/2.4.0/json-path-2.4.0.jar 18:49:22.475840 Start: jakarta/xml/bind/jakarta.xml.bind-api/2.3.2/jakarta.xml.bind-api-2.3.2.jar 18:49:22.620575 End: net/minidev/json-smart/2.3/json-smart-2.3.jar 18:49:22.624802 End: net/minidev/accessors-smart/1.2/accessors-smart-1.2.jar 18:49:22.802604 Start: jakarta/activation/jakarta.activation-api/1.2.1/jakarta.activation-api-1.2.1.jar 18:49:22.844466 Start: org/junit/jupiter/junit-jupiter/5.5.2/junit-jupiter-5.5.2.jar 18:49:22.863655 End: org/ow2/asm/asm/5.0.4/asm-5.0.4.jar 18:49:23.057122 End: org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.jar 18:49:23.137004 Start: org/junit/jupiter/junit-jupiter-api/5.5.2/junit-jupiter-api-5.5.2.jar 18:49:23.212418 Start: org/apiguardian/apiguardian-api/1.1.0/apiguardian-api-1.1.0.jar 18:49:23.241599 End: jakarta/xml/bind/jakarta.xml.bind-api/2.3.2/jakarta.xml.bind-api-2.3.2.jar 18:49:23.433187 Start: org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar 18:49:23.634035 End: jakarta/activation/jakarta.activation-api/1.2.1/jakarta.activation-api-1.2.1.jar 18:49:23.731517 Start: org/junit/platform/junit-platform-commons/1.5.2/junit-platform-commons-1.5.2.jar 18:49:23.832759 End: org/junit/jupiter/junit-jupiter/5.5.2/junit-jupiter-5.5.2.jar 18:49:23.925243 Start: org/junit/jupiter/junit-jupiter-params/5.5.2/junit-jupiter-params-5.5.2.jar 18:49:24.036507 End: org/junit/jupiter/junit-jupiter-api/5.5.2/junit-jupiter-api-5.5.2.jar 18:49:24.037959 End: org/apiguardian/apiguardian-api/1.1.0/apiguardian-api-1.1.0.jar 18:49:24.210190 Start: org/junit/jupiter/junit-jupiter-engine/5.5.2/junit-jupiter-engine-5.5.2.jar 18:49:24.242036 Start: org/junit/platform/junit-platform-engine/1.5.2/junit-platform-engine-1.5.2.jar 18:49:24.334151 End: org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar 18:49:24.484436 End: org/junit/platform/junit-platform-commons/1.5.2/junit-platform-commons-1.5.2.jar 18:49:24.569342 Start: org/mockito/mockito-junit-jupiter/3.1.0/mockito-junit-jupiter-3.1.0.jar 18:49:24.681138 Start: org/assertj/assertj-core/3.13.2/assertj-core-3.13.2.jar 18:49:24.800929 End: org/junit/jupiter/junit-jupiter-params/5.5.2/junit-jupiter-params-5.5.2.jar 18:49:25.038581 End: org/junit/jupiter/junit-jupiter-engine/5.5.2/junit-jupiter-engine-5.5.2.jar 18:49:25.188363 Start: org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar 18:49:25.222889 End: org/junit/platform/junit-platform-engine/1.5.2/junit-platform-engine-1.5.2.jar 18:49:25.360239 Start: org/mockito/mockito-core/3.1.0/mockito-core-3.1.0.jar 18:49:25.504301 Start: net/bytebuddy/byte-buddy/1.10.6/byte-buddy-1.10.6.jar 18:49:26.103765 End: org/mockito/mockito-junit-jupiter/3.1.0/mockito-junit-jupiter-3.1.0.jar 18:49:26.209608 Start: net/bytebuddy/byte-buddy-agent/1.10.6/byte-buddy-agent-1.10.6.jar 18:49:26.615590 End: org/hamcrest/hamcrest/2.1/hamcrest-2.1.jar 18:49:26.720174 Start: org/objenesis/objenesis/2.6/objenesis-2.6.jar 18:49:27.250985 End: org/mockito/mockito-core/3.1.0/mockito-core-3.1.0.jar 18:49:27.353775 Start: org/skyscreamer/jsonassert/1.5.0/jsonassert-1.5.0.jar 18:49:27.546627 End: net/bytebuddy/byte-buddy-agent/1.10.6/byte-buddy-agent-1.10.6.jar 18:49:27.670716 Start: com/vaadin/external/google/android-json/0.0.20131108.vaadin1/android-json-0.0.20131108.vaadin1.jar 18:49:27.849478 End: org/objenesis/objenesis/2.6/objenesis-2.6.jar 18:49:27.886109 End: org/assertj/assertj-core/3.13.2/assertj-core-3.13.2.jar 18:49:28.028873 Start: org/springframework/spring-core/5.2.3.RELEASE/spring-core-5.2.3.RELEASE.jar 18:49:28.086815 End: org/skyscreamer/jsonassert/1.5.0/jsonassert-1.5.0.jar 18:49:28.115193 Start: org/springframework/spring-jcl/5.2.3.RELEASE/spring-jcl-5.2.3.RELEASE.jar 18:49:28.267322 End: net/bytebuddy/byte-buddy/1.10.6/byte-buddy-1.10.6.jar 18:49:28.304931 End: com/vaadin/external/google/android-json/0.0.20131108.vaadin1/android-json-0.0.20131108.vaadin1.jar 18:49:28.354630 Start: org/springframework/spring-test/5.2.3.RELEASE/spring-test-5.2.3.RELEASE.jar 18:49:28.567477 Start: org/xmlunit/xmlunit-core/2.6.3/xmlunit-core-2.6.3.jar 18:49:28.646651 Start: org/springframework/spring/2.5.6/spring-2.5.6.jar 18:49:29.024848 End: org/springframework/spring-jcl/5.2.3.RELEASE/spring-jcl-5.2.3.RELEASE.jar 18:49:29.087806 End: org/springframework/spring-core/5.2.3.RELEASE/spring-core-5.2.3.RELEASE.jar 18:49:29.211370 Start: commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar 18:49:29.448004 End: org/springframework/spring-test/5.2.3.RELEASE/spring-test-5.2.3.RELEASE.jar 18:49:29.515209 End: org/xmlunit/xmlunit-core/2.6.3/xmlunit-core-2.6.3.jar 18:49:29.984356 End: commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar 18:49:30.178170 End: org/springframework/spring/2.5.6/spring-2.5.6.jar 18:49:30.279528 Method breakpoint reached. Method 'io.takari.aether.connector.AetherRepositoryConnector.get()' is about to exit