You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit cd58118
Browse filesBrowse the repository at this point in the historyBrowse files
See the [Comet Benchmarking Guide](https://datafusion.apache.org/comet/contributor-guide/benchmarking.html) for more details.
Spark 4.1.3 is slower than Spark 4.2.0 on this cluster (1,336 s vs 994 s), so the speedups are larger than with
58
+
Spark 4.2.0. Most of the difference is the S3 input stream: Spark 4.1.3 (Hadoop 3.4.2) reads with the `classic`
59
+
S3A stream by default, while Spark 4.2.0 (Hadoop 3.5.0) defaults to the S3 Analytics Accelerator. Spark 4.2.0 with
60
+
`spark.hadoop.fs.s3a.input.stream.type=classic` took 1,294 s. See
61
+
[#7451](https://github.com/apache/datafusion-comet/issues/7451) for the investigation.
0 commit comments