This guide reruns the single-node benchmark behind the numbers in the Apache Hudi Python/Rust quick start. It compares hudi-rs, delta-rs and PyIceberg on the same workloads, through each library's Python API, and checks every end state for correctness.
Everything is pinned:
- Harness:
benchmark/upsertat hudi-rs commit543642771a1440f8470053831463052e190b3923. It generates the data, runs every case in its own processes, gates on machine load, checks correctness and records timings. - Libraries:
requirements-lock.txtin the code gist: hudi 0.6.0, deltalake 1.6.6, pyiceberg 0.12.0, pyarrow 26.0.0, and every transitive dependency. - delta-rs and PyIceberg cases:
compare_cases.pyin the same gist (revisiond94a68449a1d), loaded into the harness with--extra-cases.
| Case | Library | Table |
|---|---|---|
hudi-mor |
hudi-rs | merge-on-read, record-level index on |
hudi-cow |
hudi-rs | copy-on-write, record-level index on (the HudiTable.create defaults) |
hudi-cow-noindex |
hudi-rs | copy-on-write, record-level index off |
delta |
deltalake | defaults |
iceberg |
pyiceberg | defaults, SQL catalog over sqlite |
Each library keeps its own defaults for file size and compression: hudi-rs 120 MiB files and zstd, delta-rs 100 MiB files and snappy, PyIceberg 512 MiB of in-memory Arrow per file and zstd. The only options set are the ones a workload needs: the record key and ordering field for hudi-rs and the identifier fields for PyIceberg's upsert. delta-rs's change data feed stays off, its default. No table is partitioned.
Each operation is the call the library documents for it:
| hudi-rs | delta-rs | PyIceberg | |
|---|---|---|---|
| Load | HudiTable.create(...), then append(table) |
write_deltalake(path, table) |
create_table(...), then append(table) |
| Upsert | upsert(table) |
merge(source, "t.k = s.k").when_matched_update_all().when_not_matched_insert_all().execute() |
upsert(table, join_cols=[k]) |
| Delete by key | delete_keys(keys) |
merge(keys, "t.k = s.k").when_matched_delete().execute() |
delete(delete_filter=In(k, keys)) |
| Read | read(), data columns only |
to_pyarrow_table() |
scan().to_arrow() |
References: delta-rs merging tables
and deleting rows,
PyIceberg API. delta-rs documents two ways to delete by key;
the merge form was faster on these workloads (2.9 s against 4.2 s per uuid delete commit), so it is
the one used. Every read returns a fully materialized Arrow table with the same data columns; Hudi's
five _hoodie_* metadata columns are not read.
Each scenario generates its base table and every commit's input from a fixed seed before anything is timed, so every library receives the same rows. A run loads the base table, applies 5 upsert commits and 5 delete commits, then reads the whole table back.
| Scenario | Models | Base rows | Each commit |
|---|---|---|---|
rag-chunks |
Re-syncing a retrieval corpus (text + 384-float embedding) | 1M | upsert 10,000 chunks, delete 1,000 |
training-set |
Curating a fine-tuning set | 1M | upsert 10,000 examples, delete 1,000 |
uuid |
Random-key upserts | 10M | upsert 10,000 existing + 1,000 new keys, delete 1,000 |
nyc-taxi |
Correcting a public dataset (NYC yellow taxi, January 2022) | 2.5M | upsert 1% of trips, delete 0.1% |
tpcds-web-returns |
A warehouse fact table, composite key | 0.7M | upsert 1% matched + 10% new, delete 5% |
training-set also times an incremental read and a time-travel read after every commit. Those reads
are not in the published table and are not like-for-like: hudi-rs and PyIceberg read the commit's
inserted and updated rows, while delta-rs, with its change data feed off, compares the two snapshots.
Turning the feed on did not change delta-rs's load, commit or read times on this workload.
Keys are random, so every copy-on-write commit touches nearly every file. hudi-rs resolves an upsert
by its ordering field (the later updated_at wins); delta-rs and PyIceberg overwrite unconditionally.
The end states agree because every input is newer than what it replaces.
- Timed: only the library call: the load, each commit, the full read. Opening the table, reading the commit inputs, correctness checks and file listings are outside the timer.
- Isolation: each phase (load, upserts, deletes, read) runs in a fresh process, and each of the three repetitions rebuilds the table from scratch.
- Cleanup: after each commit, outside the timer, the harness deletes data files the table's
current snapshot no longer references (
--reclaim, on by default), so copy-on-write runs do not keep a copy per commit. It is off ontraining-set, whose time-travel reads need the old files. No library's own cleaner or vacuum runs. - Correctness: every commit's inserted, updated and deleted row counts are checked against the plan, and every end state is compared row by row with a Parquet copy of the expected table. A case that fails either check reports no numbers.
- Quiet machine: before every timed call the harness checks that other processes kept at most
--max-loadcores busy over the last 10 seconds (the benchmark's own CPU is left out) and that at least--min-mem-gibof memory is free (measured while the phase holds its data, so it is headroom beyond the benchmark's own use). It waits up to--settle-timeoutseconds for load to drop, then refuses: the case is recorded asrefusedwith no numbers. Too little free memory refuses at once; close applications and rerun that case. Both readings are stored next to every timing. - Reported numbers: per case, the median over the 3 repetitions of load and read time, and of
each repetition's mean upsert and delete commit time.
summarize.pyprints min to max in parentheses and notes any timing whose variation across repetitions exceeds 10%.
The published numbers were taken on one Apple M3 Max laptop (14 cores, 36 GiB, local SSD), macOS 26.6,
with --max-load 6 --min-mem-gib 10 --settle-timeout 1800 --case-timeout 3600. A case whose load,
read, or per-repetition commit median varied by more than 10% across repetitions (ignoring under
20 ms), or with any single run over 1.5x its mean, was rerun, and the run with the least variation is
reported. That rerun rule is not part of the harness.
Needs uv (it provides Python 3.13), git, curl and network access (the
harness downloads the NYC taxi file and DuckDB's TPC-DS extension). About 20 GB of free disk on a real
disk; set UPSERT_BENCH_WORK_DIR if your temp directory is small or memory-backed.
git clone https://github.com/apache/hudi-rs && cd hudi-rs
git checkout 543642771a1440f8470053831463052e190b3923
curl -fsSLO https://gist.githubusercontent.com/yihua/514f656c2adcf9869ef41c265a36948a/raw/d94a68449a1df78d4f36f9b0ff97af865b02c75e/requirements-lock.txt
curl -fsSLO https://gist.githubusercontent.com/yihua/514f656c2adcf9869ef41c265a36948a/raw/d94a68449a1df78d4f36f9b0ff97af865b02c75e/compare_cases.py
benchmark/upsert/run.sh setup
uv pip install --python benchmark/upsert/.venv/bin/python -r requirements-lock.txtA smoke pass first. Smoke runs use small data, run 3 repetitions each, and only warn on a busy machine:
for s in uuid tpcds-web-returns nyc-taxi rag-chunks training-set; do
benchmark/upsert/run.sh smoke --scenario "$s" \
--cases hudi-mor,hudi-cow,hudi-cow-noindex,delta,iceberg --extra-cases compare_cases.py
doneThen full scale, one case per invocation, one at a time. Close other applications first. The limits
below are the ones the published numbers used; the harness defaults (--max-load 2 --min-mem-gib 16)
are stricter and refuse on most desktops.
mkdir -p results
for s in uuid tpcds-web-returns nyc-taxi rag-chunks training-set; do
for c in hudi-mor hudi-cow hudi-cow-noindex delta iceberg; do
benchmark/upsert/run.sh full --scenario "$s" --cases "$c" --repeat 3 \
--max-load 6 --min-mem-gib 10 --settle-timeout 1800 --case-timeout 3600 \
--extra-cases compare_cases.py --out "results/$s-$c.json"
done
done
benchmark/upsert/run.sh summarize results/*.json --markdownEach result file records the machine, the hudi-rs commit, every library version (deltalake and
pyiceberg under each case's config), the exact call behind every operation, and the load and memory
readings taken before each timed call. --force turns refusals into warnings, but summarize.py
then renders those results only with --allow-loaded.
Known issue: on macOS, PyIceberg 0.12 crashes on the first tpcds-web-returns upsert. pyarrow
overflows a worker thread's stack while building the per-key filter for the composite key. The
harness records the case as failed. Linux gives threads a larger stack, so the outcome there may
differ.