Skip to content

Commit cd58118

Browse files
committed
Merge remote-tracking branch 'origin/main' into txt-file-support
2 parents e9f2edb + eab3ba6 commit cd58118

23 files changed

Lines changed: 311 additions & 72 deletions

File tree

‎.github/workflows/codeql.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,11 +52,11 @@ jobs:
5252
persist-credentials: false
5353

5454
- name: Initialize CodeQL
55-
uses: github/codeql-action/init@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4
55+
uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
5656
with:
5757
languages: actions
5858

5959
- name: Perform CodeQL Analysis
60-
uses: github/codeql-action/analyze@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4
60+
uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4
6161
with:
6262
category: "/language:actions"

‎README.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -42,17 +42,17 @@ pipeline, operators and expressions execute as Rust code (via the [Apache DataFu
4242
or as JVM code that operates directly on Arrow batches. Comet integrates with the Spark ecosystem
4343
without requiring any code changes.
4444

45-
**Comet provides a ~1.8x speedup over Spark 4.2 for TPC-DS @ SF 1000 (1TB), resulting in ~45% cost savings.**
45+
**Comet provides a ~2x speedup over Spark 4.2 for TPC-DS @ SF 1000 (1TB), resulting in ~50% cost savings.**
4646

47-
That speedup gives you a choice: finish the same Spark workload in just over half the time on the cluster you already
48-
have, or match your current Spark performance on just over half the resources. Either way, the gain translates directly into
47+
That speedup gives you a choice: finish the same Spark workload in half the time on the cluster you already
48+
have, or match your current Spark performance on half the resources. Either way, the gain translates directly into
4949
lower cloud bills, reduced on-prem capacity, and lower energy usage, with no changes to your existing Spark SQL,
5050
DataFrame, or PySpark code. Comet runs on commodity hardware: no GPUs, FPGAs, or other specialized accelerators are
5151
required, so the savings come from better utilization of the infrastructure you already run on.
5252

53-
![](docs/source/_static/images/benchmark-results/1.1.0/tpcds_allqueries.png)
53+
![](docs/source/_static/images/benchmark-results/1.2.0/tpcds_allqueries.png)
5454

55-
![](docs/source/_static/images/benchmark-results/1.1.0/tpcds_queries_speedup_rel.png)
55+
![](docs/source/_static/images/benchmark-results/1.2.0/tpcds_queries_speedup_rel.png)
5656

5757
See the [Comet Benchmarking Guide](https://datafusion.apache.org/comet/contributor-guide/benchmarking.html) for more details.
5858

‎benchmarks/tpc/generate-comparison.py‎

Lines changed: 12 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -104,11 +104,8 @@ def generate_query_rel_speedup_chart(baseline, comparison, label1: str, label2:
104104
for query in common_queries:
105105
a = np.median(np.array(get_durations(baseline, str(query))))
106106
b = np.median(np.array(get_durations(comparison, str(query))))
107-
if a > b:
108-
speedup = a/b-1
109-
else:
110-
speedup = -(1/(a/b)-1)
111-
results.append(("q" + str(query), round(speedup*100, 0)))
107+
# Speedup as a multiple: 2.0 means twice as fast, 0.5 means twice as slow.
108+
results.append(("q" + str(query), a/b))
112109

113110
results = sorted(results, key=lambda x: -x[1])
114111

@@ -120,28 +117,27 @@ def generate_query_rel_speedup_chart(baseline, comparison, label1: str, label2:
120117
else:
121118
fig, ax = plt.subplots(figsize=(35, 10))
122119

123-
# Create bar chart
124-
bars = ax.bar(queries, speedups, color='skyblue')
120+
# Draw the bars from 1.0x (same speed) up for speedups and down for slowdowns
121+
bars = ax.bar(queries, [s - 1 for s in speedups], bottom=1, color='skyblue')
125122

126123
# Add text annotations
127124
for bar, speedup in zip(bars, speedups):
128-
yval = bar.get_height()
129-
if yval >= 0:
130-
ax.text(bar.get_x() + bar.get_width() / 2.0, min(800, yval+5), f'{yval:.0f}%', va='bottom', ha='center', fontsize=8,
125+
if speedup >= 1:
126+
ax.text(bar.get_x() + bar.get_width() / 2.0, speedup + 0.02, f'{speedup:.2f}x', va='bottom', ha='center', fontsize=8,
131127
color='blue', rotation=90)
132128
else:
133-
ax.text(bar.get_x() + bar.get_width() / 2.0, yval, f'{yval:.0f}%', va='top', ha='center', fontsize=8,
129+
ax.text(bar.get_x() + bar.get_width() / 2.0, speedup - 0.02, f'{speedup:.2f}x', va='top', ha='center', fontsize=8,
134130
color='blue', rotation=90)
135131

136132
# Add title and labels
137133
ax.set_title(label2 + " speedup over " + label1 + " (" + title + ")")
138-
ax.set_ylabel('Speedup Percentage (100% speedup = 2x faster)')
134+
ax.set_ylabel('Speedup (2x = twice as fast, below 1x = slower)')
139135
ax.set_xlabel('Query')
140136

141-
# Customize the y-axis to handle both positive and negative values better
142-
ax.axhline(0, color='black', linewidth=0.8)
143-
min_value = (min(speedups) // 100) * 100
144-
max_value = ((max(speedups) // 100) + 1) * 100 + 50
137+
# Customize the y-axis to handle both speedups and slowdowns
138+
ax.axhline(1, color='black', linewidth=0.8)
139+
min_value = max(0.0, np.floor((min(min(speedups), 1.0) - 0.25) * 4) / 4)
140+
max_value = max(speedups) * 1.1 + 0.25
145141
ax.set_ylim(min_value, max_value)
146142

147143
# Show grid for better readability
26.3 KB
Loading
123 KB
Loading
25 KB
Loading
61.9 KB
Loading
92.2 KB
Loading
126 KB
Loading

‎docs/source/contributor-guide/benchmark-results/tpc-ds.md‎

Lines changed: 51 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -19,37 +19,76 @@ under the License.
1919

2020
# TPC-DS
2121

22-
The following benchmarks were performed on an EKS cluster (`r6i.24xlarge` instances with EBS storage) with data stored in S3,
23-
using Apache Spark 4.2.0 (Scala 2.13) and Comet 1.1.0. Each query was run twice and the mean time is reported.
22+
The following benchmarks were performed on an EKS cluster (`r6i` instances with EBS storage) with data stored in S3,
23+
using Apache Spark 4.2.0 (Scala 2.13), Comet 1.1.0 and Comet 1.2.0. The tables are Parquet files compressed with zstd at
24+
scale factor 1000 (1 TB), and all three runs read the same files. Each query was run twice and the mean time is reported.
25+
26+
Across all 103 queries, Comet 1.2.0 finished in 488 seconds, Comet 1.1.0 in 601 seconds and Spark in 994 seconds,
27+
so Comet 1.2.0 is 2.0x faster than Spark and 1.2x faster than Comet 1.1.0.
2428

2529
## Benchmark Results
2630

2731
Total time to run all queries (lower is better).
2832

29-
![](../../_static/images/benchmark-results/1.1.0/tpcds_allqueries.png)
33+
![](../../_static/images/benchmark-results/1.2.0/tpcds_allqueries.png)
3034

3135
Per-query breakdown showing the relative performance of Spark and Comet.
3236

33-
![](../../_static/images/benchmark-results/1.1.0/tpcds_queries_compare.png)
37+
![](../../_static/images/benchmark-results/1.2.0/tpcds_queries_compare.png)
38+
39+
How much Comet 1.2.0 accelerates each query over Spark in relative terms.
40+
41+
![](../../_static/images/benchmark-results/1.2.0/tpcds_queries_speedup_rel.png)
3442

35-
How much Comet accelerates each query in relative terms.
43+
How much Comet 1.2.0 accelerates each query over Spark in absolute terms.
3644

37-
![](../../_static/images/benchmark-results/1.1.0/tpcds_queries_speedup_rel.png)
45+
![](../../_static/images/benchmark-results/1.2.0/tpcds_queries_speedup_abs.png)
3846

39-
How much Comet accelerates each query in absolute terms.
47+
## Spark 4.1.3
4048

41-
![](../../_static/images/benchmark-results/1.1.0/tpcds_queries_speedup_abs.png)
49+
The same tables and settings were also run with Apache Spark 4.1.3 (Scala 2.13).
50+
51+
| | Total time (103 queries) | Speedup over Spark |
52+
| ----------- | ------------------------ | ------------------ |
53+
| Spark 4.1.3 | 1,336 s | |
54+
| Comet 1.1.0 | 613 s | 2.18x |
55+
| Comet 1.2.0 | 532 s | 2.51x |
56+
57+
Spark 4.1.3 is slower than Spark 4.2.0 on this cluster (1,336 s vs 994 s), so the speedups are larger than with
58+
Spark 4.2.0. Most of the difference is the S3 input stream: Spark 4.1.3 (Hadoop 3.4.2) reads with the `classic`
59+
S3A stream by default, while Spark 4.2.0 (Hadoop 3.5.0) defaults to the S3 Analytics Accelerator. Spark 4.2.0 with
60+
`spark.hadoop.fs.s3a.input.stream.type=classic` took 1,294 s. See
61+
[#7451](https://github.com/apache/datafusion-comet/issues/7451) for the investigation.
62+
63+
Total time to run all queries (lower is better).
64+
65+
![](../../_static/images/benchmark-results/1.2.0/spark-4.1.3/tpcds_allqueries.png)
66+
67+
How much Comet 1.2.0 accelerates each query over Spark 4.1.3 in relative terms.
68+
69+
![](../../_static/images/benchmark-results/1.2.0/spark-4.1.3/tpcds_queries_speedup_rel.png)
4270

4371
## Configuration
4472

73+
Only settings that differ from the defaults are listed. Comet 1.1.0 and Comet 1.2.0 used the same settings.
74+
4575
Common:
4676

4777
```properties
78+
spark.driver.memory=128G
79+
spark.driver.maxResultSize=16g
4880
spark.executor.instances=32
4981
spark.executor.cores=16
50-
spark.memory.fraction=0.6
5182
spark.memory.storageFraction=0.2
83+
spark.sql.shuffle.partitions=512
84+
spark.sql.broadcastTimeout=900
85+
spark.sql.sources.ignoreDataLocality=true
86+
spark.hadoop.parquet.io.stats.enabled=false
87+
spark.hadoop.fs.s3a.connection.maximum=5000
88+
spark.hadoop.fs.s3a.threads.max=20
5289
# Kubernetes CPU constraints
90+
spark.kubernetes.driver.request.cores=32
91+
spark.kubernetes.driver.limit.cores=32
5392
spark.kubernetes.executor.request.cores=8
5493
spark.kubernetes.executor.limit.cores=8
5594
```
@@ -68,5 +107,8 @@ spark.executor.memory=64G
68107
spark.executor.memoryOverhead=10G
69108
spark.memory.offHeap.enabled=true
70109
spark.memory.offHeap.size=32G
110+
spark.plugins=org.apache.spark.CometPlugin
111+
spark.shuffle.manager=org.apache.spark.sql.comet.execution.shuffle.CometShuffleManager
112+
spark.sql.extensions=org.apache.comet.CometSparkSessionExtensions
71113
spark.comet.exec.memoryPool.fraction=0.8
72114
```

0 commit comments

Comments
 (0)