Debugging

Deep Dive into Spark SQL Metrics (Part 1): Types, Full Reference, and What They Mean

Part 1 of a 3-part deep dive into Apache Spark’s SQL metrics system. Covers the 5 metric types, a complete reference of 100+ metrics across all operators, and how to read the numbers in the Spark UI.

Introducing spark-advisor: An AI-Powered Spark Performance Engineer

spark-advisor is an agent skill that turns your AI coding assistant into a Spark performance engineer — diagnosing slow jobs, detecting skew, comparing benchmark runs, and producing actionable tuning recommendations.