You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Spark 4.2: segment-tree window metrics unavailable under CometWindowExec #4964
Spark 4.2 adds a segment-tree window-aggregation optimization to WindowExec, along with metrics such as "number of segment-tree frames prepared" and "number of segment-tree fallback frames prepared", plus an eligibility allowlist.
Comet replaces WindowExec with CometWindowExec, which has no segment-tree implementation and does not expose those metrics. The new test suites read the metric from a Spark Window node in the executed plan and fail with "No Window node in plan" when Comet is enabled.
Affected Spark SQL test suites (currently marked IgnoreCometSuite in dev/diffs/4.2.0.diff):
Activity
test: resolve Spark 4.2 SQL test failures and file specific tracking …
build: enable Spark SQL tests for released Spark 4.2.0 (apache#4950)