{"@attributes":{"version":"2.0"},"channel":{"title":"cdelmonte.dev","link":"https:\/\/cdelmonte.dev\/","description":"Recent content on cdelmonte.dev","image":{"title":"cdelmonte.dev","url":"https:\/\/cdelmonte.dev\/og-default.png","link":"https:\/\/cdelmonte.dev\/og-default.png"},"generator":"Hugo -- 0.147.7","language":"en-us","lastBuildDate":"Mon, 12 Oct 2026 00:00:00 +0000","item":[{"title":"Why Sorted Parquet Files Do Not Guarantee an Ordered Scan in DataFusion","link":"https:\/\/cdelmonte.dev\/explainers\/sorted-files-ordered-scan\/","pubDate":"Mon, 12 Oct 2026 00:00:00 +0000","guid":"https:\/\/cdelmonte.dev\/explainers\/sorted-files-ordered-scan\/","description":"A purchases table, one Parquet file per day, every file sorted by customer: you would expect a GROUP BY customer to use that order, yet EXPLAIN often shows it ignored. This explainer follows the ordering from the file scan to the aggregate in Apache DataFusion, shows where it gets lost, how to read it in the plan, and what you can change."},{"title":"A Small Compiler for Explaining Delta Lake Pruning","link":"https:\/\/cdelmonte.dev\/essays\/a-small-compiler-for-explaining-delta-lake-pruning\/","pubDate":"Sat, 04 Jul 2026 00:00:00 +0000","guid":"https:\/\/cdelmonte.dev\/essays\/a-small-compiler-for-explaining-delta-lake-pruning\/","description":"How delta-explain turns one predicate and the Delta log into a trustworthy explanation of file pruning, through shared representation, safe rewrites, and conservative measurement."},{"title":"From Dictionary Indices to Integers: Parquet Bit Unpacking Explained","link":"https:\/\/cdelmonte.dev\/explainers\/parquet-bit-unpacking-explained\/","pubDate":"Wed, 10 Jun 2026 00:00:00 +0000","guid":"https:\/\/cdelmonte.dev\/explainers\/parquet-bit-unpacking-explained\/","description":"Parquet rarely stores a column’s values directly. Dictionary encoding keeps the distinct values once and represents the column through small bit-packed integer indices. This explainer walks through dictionary encoding, how the indices are bit-packed, and the bit-unpacking step that reconstructs them into an int[] before every dictionary lookup."},{"title":"Unpacking Parquet: Explicit SIMD, Scalar Baselines, and What HotSpot Makes of Them","link":"https:\/\/cdelmonte.dev\/deep-dives\/unpacking-parquet-explicit-simd-and-scalar-baselines\/","pubDate":"Tue, 09 Jun 2026 00:00:00 +0000","guid":"https:\/\/cdelmonte.dev\/deep-dives\/unpacking-parquet-explicit-simd-and-scalar-baselines\/","description":"On the JVM, optimizing a hot kernel is not only about writing faster code: it is also about understanding how much the result depends on the machine code HotSpot derives from the scalar loop. Using Parquet bit-unpacking as a concrete case, the piece shows that a SIMD speedup depends on which scalar baseline C2 is handed, when explicit vectorization is actually justified, and why a more specialized scalar routine is not necessarily faster."},{"title":"Where Spark Changes Shape: UnsafeRow, the JVM, and the Relocation of Portability","link":"https:\/\/cdelmonte.dev\/essays\/where-spark-changes-shape\/","pubDate":"Tue, 26 May 2026 00:00:00 +0000","guid":"https:\/\/cdelmonte.dev\/essays\/where-spark-changes-shape\/","description":"Run df.explain on a Parquet scan and you find ColumnarToRow, an operator whose only job is to change the data’s shape. It is the seam where two architectural eras meet, and a record of how portability in analytical systems has relocated from the JVM runtime to the data format and the query plan."},{"title":"Anti-patterns in Catalyst rules","link":"https:\/\/cdelmonte.dev\/deep-dives\/anti-patterns-in-catalyst-rules\/","pubDate":"Sun, 26 Apr 2026 00:00:00 +0000","guid":"https:\/\/cdelmonte.dev\/deep-dives\/anti-patterns-in-catalyst-rules\/","description":"Six concrete anti-patterns I encountered building a real Catalyst extension: from the wrong rule type for throws, to mutable state under AQE and Spark Connect, to JVM bootstrap traps in PySpark."},{"title":"The Hidden DSL in Catalyst","link":"https:\/\/cdelmonte.dev\/deep-dives\/the-hidden-dsl-in-catalyst\/","pubDate":"Sun, 26 Apr 2026 00:00:00 +0000","guid":"https:\/\/cdelmonte.dev\/deep-dives\/the-hidden-dsl-in-catalyst\/","description":"How Spark’s internal rewriting framework, Catalyst, exposes an embedded DSL with a public extension surface, the same one Delta Lake and Iceberg use to plug into the optimizer pipeline."},{"title":"delta-explain: Making Delta Lake Pruning Visible","link":"https:\/\/cdelmonte.dev\/projects\/delta-explain-making-delta-pruning-visible\/","pubDate":"Mon, 13 Apr 2026 00:00:00 +0000","guid":"https:\/\/cdelmonte.dev\/projects\/delta-explain-making-delta-pruning-visible\/","description":"Partition pruning and data skipping are invisible by default. delta-explain reads the Delta log directly and shows, step by step, how a WHERE predicate narrows down candidate files, with no engine required."},{"title":"The Disaggregation of the Lakehouse Stack","link":"https:\/\/cdelmonte.dev\/essays\/the-disaggregation-of-the-lakehouse-stack\/","pubDate":"Sun, 08 Mar 2026 00:00:00 +0000","guid":"https:\/\/cdelmonte.dev\/essays\/the-disaggregation-of-the-lakehouse-stack\/","description":"How Delta Kernel, Arrow, and pluggable execution are disaggregating the lakehouse stack. The lakehouse stack is not converging on a new dominant engine. It is converging on a layered architecture in which protocol, data representation, and query execution are increasingly isolated behind stable interfaces."},{"title":"Where Data System Abstractions Break: A Semiotic Reading","link":"https:\/\/cdelmonte.dev\/essays\/where-data-system-abstractions-break-semiotic-reading\/","pubDate":"Wed, 04 Mar 2026 00:00:00 +0000","guid":"https:\/\/cdelmonte.dev\/essays\/where-data-system-abstractions-break-semiotic-reading\/","description":"Many of the most surprising performance pathologies in modern data systems are semiotic failures: structural divergences between what an interface signifies and what the underlying system does."},{"title":"Fixing Skewed Nested Joins in Spark with Asymmetric Salting","link":"https:\/\/cdelmonte.dev\/deep-dives\/fixing-skewed-nested-joins-spark-asymmetric-salting\/","pubDate":"Mon, 01 Dec 2025 00:00:00 +0000","guid":"https:\/\/cdelmonte.dev\/deep-dives\/fixing-skewed-nested-joins-spark-asymmetric-salting\/","description":"In large-scale Spark pipelines, skew can occur when a single key carries a disproportionately large nested payload. Asymmetric salting offers a targeted solution: explode, salt, join in parallel, and optionally re-aggregate."},{"title":"Spark Is Not Just Lazy. Spark Compiles Dataflow.","link":"https:\/\/cdelmonte.dev\/deep-dives\/spark-is-not-lazy-spark-compiles-dataflow\/","pubDate":"Mon, 03 Nov 2025 00:00:00 +0000","guid":"https:\/\/cdelmonte.dev\/deep-dives\/spark-is-not-lazy-spark-compiles-dataflow\/","description":"Why calling Spark ’lazy’ is technically reductive, and how thinking of it as a dataflow compiler changes the way you design pipelines."},{"title":"Two Algorithms, One Intuition: Shunting Yard and Pratt Parsing","link":"https:\/\/cdelmonte.dev\/deep-dives\/shunting-yard-and-pratt-parsing\/","pubDate":"Mon, 06 Oct 2025 00:00:00 +0000","guid":"https:\/\/cdelmonte.dev\/deep-dives\/shunting-yard-and-pratt-parsing\/","description":"Parsing arithmetic expressions looks simple\u2026 until precedence enters the picture. Two classic algorithms, Dijkstra’s Shunting Yard and Pratt’s Top-Down Operator Precedence, provide radically different answers that reveal the same underlying intuition."},{"title":"Inside GKE Workload Identity: How Kubernetes Identities Become GCP Service Accounts","link":"https:\/\/cdelmonte.dev\/deep-dives\/kubernetes-gcp-service-accounts-workload-identity\/","pubDate":"Thu, 01 Aug 2024 00:00:00 +0000","guid":"https:\/\/cdelmonte.dev\/deep-dives\/kubernetes-gcp-service-accounts-workload-identity\/","description":"GKE Behind the Scenes: Understanding the Interaction Between Kubernetes and GCP Service Accounts Through The Metadata Server."}]}}