Inside DataFusion, Polars and DuckDB, with each engine's own answers.

ArrowMetal runs inside three query engines. Each one plans the query as it always does; ArrowMetal takes the parts of the plan measured faster on the Apple GPU, hands back what the engine would have returned, and leaves every other node to the engine. Every figure below comes from a run recorded in the repository, linked from its card.

use std::sync::Arc;
use datafusion::{datasource::MemTable, prelude::*};
use datafusion_arrowmetal::{
    session_context, ArrowMetalConfig, ArrowMetalRule};

let rule = ArrowMetalRule::new(ArrowMetalConfig::default());
let ctx = session_context(SessionConfig::new(), rule.clone());
let t = MemTable::try_new(batch.schema(), vec![vec![batch]])?;
ctx.register_table("t", Arc::new(t))?;
ctx.sql("SELECT * FROM t ORDER BY amount DESC")
    .await?.show().await?;      // as written
println!("{}", rule.report()); // TAKEN / LEFT, why

import polars as pl, arrowmetal as am

engine = am.MetalEngine()  # the measured defaults
out = (orders.lazy()
         .join(customers.lazy(), on="customer_id")
         .sort("amount", descending=True)
         .collect(engine=engine))  # Polars' answer
print(engine.last_report)  # Metal or Polars, per node

LOAD
  'duckdb-extension/build/arrowmetal_rewrite.duckdb_extension';

SELECT region, sum(amount), count(*), max(amount)
FROM sales GROUP BY region;          -- as written

SELECT * FROM arrowmetal_rewrites(); -- each decision
SET arrowmetal_rewrite = 'off';      -- plain DuckDB

New in 0.6.1.

Released on 2026-11-08: pip install arrowmetal, or arrowmetal = "0.6.1" and datafusion-arrowmetal = "0.6.1" on crates.io.

• You analyse data on a Mac with Polars, pyarrow, DuckDB or pandas, the scans and group-bys are the part you would hand to the GPU, and the columns can stay there between calls.
• You build a tool or a service on Apple silicon and want the CPU back for the application while the GPU does the columns.
• You maintain an engine and want an Apple-silicon GPU backend that speaks Arrow at every edge, without writing Metal.

Install, load, query.

Built and tested on an M4 Max; the wheel is checked in a clean virtual environment, with pyarrow as its only dependency. The Swift package resolves from the v0.6.1 tag. The ⧉ mark copies a block or a command.

import pyarrow as pa, arrowmetal as am

cols = {k: am.array(t[k]) for k in t.column_names}
region, amount = am.col("region"), am.col("amount")
q = am.filter((region == 2) & (amount > 100)).sum(amount)
am.query(cols, q)     # one Metal kernel, one round trip
cols["amount"].top_k(100).to_arrow()   # back to pyarrow

# Cargo.toml
[dependencies]
datafusion = { version = "=55.1.0", default-features = false, features = ["sql"] }
datafusion-arrowmetal = "0.6.1"
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }

import polars as pl, arrowmetal as am

df = pl.read_parquet("orders.parquet")
engine = am.MetalEngine()      # Polars' own answers
top = (df.lazy().sort("amount", descending=True)
         .head(100).collect(engine=engine))
print(engine.last_report)     # what ran on Metal, and why

import arrowmetal as am

con = am.duckdb_connect()   # DuckDB + rewrite extension
con.sql("CREATE TABLE sales AS "
        "SELECT * FROM 'sales.parquet'")
con.sql("""SELECT region, sum(amount), count(*)
           FROM sales GROUP BY region""").show()
# as written; eligible aggregates run on the GPU

import pandas as pd, arrowmetal as am
import arrowmetal.pandas_bridge       # adds .am

df = pd.read_parquet("trades.parquet",
                     dtype_backend="pyarrow")
df["price"].am.sum()
df["symbol"].am.contains("BRK")
df.am.groupby("symbol").sum("size")

// Package.swift
.package(url: "https://github.com/singhpratech/ArrowMetal",
         from: "0.6.1")

import ArrowMetal
let ctx    = try MetalContext()
let amount = try MetalArray<Int64>(values, context: ctx)
let mask   = try amount.compare(.gt, 100)
let total  = try amount.filter(where: mask).sum()

Unified memory changes the arithmetic of GPU analytics.

On a discrete GPU the data has to cross a bus before any kernel runs, so only long jobs pay off. On Apple silicon the Arrow buffer is already where the GPU reads, so a 3 millisecond scan is a 3 millisecond scan.

GPU dataframes existed for NVIDIA. We could not find Arrow compute on Metal.

Apache Arrow defines ARROW_DEVICE_METAL in its C Device Data Interface, which the project still marks experimental, and its nanoarrow library implements it as memory: page-aligned Metal buffers, allocation and copies, tested on Apple silicon runners, with no compute on them. ArrowMetal is the compute. Here is what we found, and what each one does on an Apple GPU, as of September 2026. If something is missing from this table, tell us and it will be corrected.

Claims about other projects come from their public documentation at the time of writing; corrections welcome as an issue on the repository.

Upstream tracker: what the matrix and the bindings found in other projects, reported with a reproduction, and where each report stands.

Twenty findings across six projects: eighteen reported to the projects, as our own issues or as evidence added to an existing one, seven of them fixed upstream; one pyarrow crash withheld until it reproduces; one Metal compiler abort worked around in the kernels, with the reproducer in our own issue tracker. Each divergence the matrix carries has a test in the repository that fails on the day upstream fixes it, so the workaround can be removed. Nothing here is a complaint about another project: every entry is a divergence the matrix has to carry.

The full tracker, with the evidence behind each row and where every report stands, is the Upstream tracker page in Docs and docs/UPSTREAM.md in the repository.

Against each library's fastest idiom, on every row that has one.

One representative row per operation from the full matrix, wall-clock milliseconds on an Apple M4 Max with 16 CPU cores, the best of up to five calls after a warm-up (a slow row is repeated fewer times, never fewer than twice; the count is in the CSV). Every CPU library is measured twice on the same data — its plain eager idiom, and the most parallel idiom it has for the same answer: Polars through pl.LazyFrame on the in-memory or streaming engine, pyarrow through its Acero query engine as a plan over 16 record batches (pandas' threaded paths, numexpr and numba, were not installed for this run; numpy's ufuncs are single-threaded). Each library column below shows the faster of its two, tagged with which; the small number after each library's wall time is the cores that call used, and the Compare tab shows every idiom side by side. Speedup is the fastest CPU idiom of any library divided by ArrowMetal's time; rows are sorted by it. Regenerated from the CSV on every build; the full table is the Compare tab.

A dash means the library has no equivalent operation or was not measured for that row; where pandas has no idiom, the numpy one is shown and marked, and a small lazy or acero tag means that library's parallel idiom was the faster of its two. Polars' lazy engine used a median of 11.5 of the 16 cores across this run and pyarrow's Acero 11.1, against 1.0 for the eager idioms on most families; the per-row figure is in every cell. Every row is on the Compare tab with the processor time each call consumed. Bold speedups are at or above the project's 3x bar.

312 compute functions, audited one by one.

Every function name in the Arrow 25 compute list plus the grouped aggregates, each executed through call_function against pyarrow and recorded as GPU, host, partial or missing. The counts come from the registry the tests run; the status is what runs, not what was intended.

Kernels at the bottom, an engine on top, one package.

Python and Swift natively; Rust, Go, TypeScript and R over one C header.

One library, one C header. The Python and Swift packages call the same kernels; every other language reaches them through include/arrowmetal.h with no Swift runtime in sight, which is how the Rust plugin and the C++ extension were built. Each binding is tested against its own ecosystem's kernels on the same data and carries measured timings in its doc, the rows where the ecosystem's own kernel is ahead included. Anything marked not started is an invitation.

Each binding's doc carries its own measured timing table with the method stated, the rows where the ecosystem's own kernel is ahead included.