You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Comet throws RuntimeException instead of SparkException for invalid row index column type #3886
The Spark SQL test `ParquetRowIndexSuite: invalid row index column type - vectorized reader` expects a `SparkException` with condition `FAILED_READ_FILE` when the row index temporary column is declared with the wrong type (e.g., `StringType` instead of `LongType`).
Spark's vectorized reader wraps the `RuntimeException` from `findRowIndexColumnIndexInSchema` in a `SparkException` via `QueryExecutionErrors.cannotReadFilesError()`. Comet's `NativeBatchReader` lets the `RuntimeException` propagate unwrapped.
Steps to reproduce
Run the Spark SQL test suite `ParquetRowIndexSuite` with Comet enabled against Spark 4.0.1.
Expected behavior
`SparkException` with condition starting with `FAILED_READ_FILE` should be thrown, matching Spark's native behavior.
Activity
fix: skip invalid row index column type test for native_datafusion scan
liupoyi-1031 commentedon Sep 19, 2026
take