Use Apache HTTP client as default AWS HTTP client #7118
Description
Feature Request / Improvement
We have been using the URLConnection HTTP client as default in https://github.com/apache/iceberg/blob/master/aws/src/main/java/org/apache/iceberg/aws/AwsProperties.java#L359, but there has been complaints about it having issue with S3 connection when throughput is high, it will transiently fail with:
java.io.UncheckedIOException: javax.net.ssl.SSLException: Connection reset
at software.amazon.awssdk.utils.FunctionalUtils.asRuntimeException(FunctionalUtils.java:180)
at software.amazon.awssdk.utils.FunctionalUtils.lambda$safeSupplier$4(FunctionalUtils.java:110)
at software.amazon.awssdk.utils.FunctionalUtils.invokeSafely(FunctionalUtils.java:136)
at software.amazon.awssdk.http.urlconnection.UrlConnectionHttpClient$RequestCallable.lambda$call$1(UrlConnectionHttpClient.java:212)
Upgrading the AWS version might have fixed the specific issue, but overall it feels better to use Apache client which is more mature.
Any thoughts about changing this default? @amogh-jahagirdar @singhpk234 @rajarshisarkar @nastra @danielcweeks
Query engine
None