{"@attributes":{"version":"2.0"},"channel":{"title":"Sql on Kent Yao","link":"https:\/\/yaooqinn.github.io\/tags\/sql\/","description":"Recent content in Sql on Kent Yao","generator":"Hugo -- 0.157.0","language":"en-us","lastBuildDate":"Thu, 02 Apr 2026 00:00:00 +0000","item":[{"title":"Deep Dive into Spark SQL Metrics (Part 5): Gluten Metrics Internals","link":"https:\/\/yaooqinn.github.io\/posts\/spark\/sql-metrics-part5-gluten-internals\/","pubDate":"Thu, 02 Apr 2026 00:00:00 +0000","guid":"https:\/\/yaooqinn.github.io\/posts\/spark\/sql-metrics-part5-gluten-internals\/","description":"Part 5 of the SQL Metrics deep dive. How Gluten maps Substrait plan nodes to Velox operators, aggregates metrics across pipelines, walks the MetricsUpdaterTree, and handles aggregation sub-phases and shuffle metrics."},{"title":"Deep Dive into Spark SQL Metrics (Part 6): Metrics In Action \u2014 TPC-DS q99 with Gluten","link":"https:\/\/yaooqinn.github.io\/posts\/spark\/sql-metrics-part6-in-action\/","pubDate":"Thu, 02 Apr 2026 00:00:00 +0000","guid":"https:\/\/yaooqinn.github.io\/posts\/spark\/sql-metrics-part6-in-action\/","description":"Part 6 of the SQL Metrics series. A real-world walkthrough of TPC-DS q99 at SF10000 with Gluten\/Velox, reading every metric to understand what happened during execution."},{"title":"Deep Dive into Spark SQL Metrics (Part 1): Types, Full Reference, and What They Mean","link":"https:\/\/yaooqinn.github.io\/posts\/spark\/understanding-sql-metrics\/","pubDate":"Wed, 01 Apr 2026 00:00:00 +0000","guid":"https:\/\/yaooqinn.github.io\/posts\/spark\/understanding-sql-metrics\/","description":"Part 1 of a 3-part deep dive into Apache Spark’s SQL metrics system. Covers the 5 metric types, a complete reference of 100+ metrics across all operators, and how to read the numbers in the Spark UI."},{"title":"Deep Dive into Spark SQL Metrics (Part 2): Internals and How AQE Uses Them","link":"https:\/\/yaooqinn.github.io\/posts\/spark\/sql-metrics-part2-internals\/","pubDate":"Wed, 01 Apr 2026 00:00:00 +0000","guid":"https:\/\/yaooqinn.github.io\/posts\/spark\/sql-metrics-part2-internals\/","description":"Part 2 of the SQL Metrics deep dive. How metrics flow from tasks to driver, and how Adaptive Query Execution uses shuffle statistics to rewrite plans at runtime."},{"title":"Deep Dive into Spark SQL Metrics (Part 3): Extension APIs, UI, and REST API","link":"https:\/\/yaooqinn.github.io\/posts\/spark\/sql-metrics-part3-extension-api\/","pubDate":"Wed, 01 Apr 2026 00:00:00 +0000","guid":"https:\/\/yaooqinn.github.io\/posts\/spark\/sql-metrics-part3-extension-api\/","description":"Part 3 of the SQL Metrics deep dive. How to extend Spark with custom metrics via the DataSource V2 API, how the UI renders them, and how to query metrics programmatically."},{"title":"Deep Dive into Spark SQL Metrics (Part 4): How Gluten Extends the Metrics System","link":"https:\/\/yaooqinn.github.io\/posts\/spark\/sql-metrics-part4-gluten\/","pubDate":"Wed, 01 Apr 2026 00:00:00 +0000","guid":"https:\/\/yaooqinn.github.io\/posts\/spark\/sql-metrics-part4-gluten\/","description":"Part 4 of the SQL Metrics deep dive. How Apache Gluten bridges native Velox\/ClickHouse metrics back to Spark’s SQL Metrics framework, adding 60+ metrics that vanilla Spark doesn’t have."},{"title":"The SQL Execution Detail Page Finally Shows You What Your Jobs Are Doing","link":"https:\/\/yaooqinn.github.io\/posts\/spark\/sql-execution-page-modernization\/","pubDate":"Sat, 14 Mar 2026 00:00:00 +0000","guid":"https:\/\/yaooqinn.github.io\/posts\/spark\/sql-execution-page-modernization\/","description":"The SQL execution detail page in Spark’s Web UI used to show jobs as comma-separated IDs. Now it has a full Associated Jobs table with status, duration, stage progress, and task progress bars \u2014 so you can debug SQL queries without clicking through each job individually."},{"title":"Rethinking SQL Plan Visualization in Apache Spark","link":"https:\/\/yaooqinn.github.io\/posts\/spark\/sql-plan-visualization\/","pubDate":"Thu, 05 Mar 2026 00:00:00 +0000","guid":"https:\/\/yaooqinn.github.io\/posts\/spark\/sql-plan-visualization\/","description":"The Spark SQL plan visualization just got a major upgrade \u2014 compact node labels, clickable metric panels, and edge row counts that make join explosions immediately visible."}]}}