Repository navigation
[Bug] parse_url raises on a malformed URL when ANSI mode is off, Spark returns NULL #43
Description
Describe the bug
Spark's ParseUrl has failOnError = SQLConf.get.ansiEnabled (urlExpressions.scala#L211-L219). A malformed URL returns NULL unless ANSI mode is on, and then it raises INVALID_URL. try_parse_url never raises.
parse_url in datafusion-spark always raises the ANSI style error (parse_url.rs#L110-L113) and does not read datafusion.execution.enable_ansi_mode. With ANSI mode off, which is the DataFusion default, the result differs from Spark.
To Reproduce
Spark 4.0.0:
SELECT parse_url('inva lid://spark.apache.org/path?query=1', 'QUERY')
-- NULL (spark.sql.ansi.enabled=false)
-- [INVALID_URL] The url is invalid: inva lid://spark.apache.org/path?query=1.
-- Use `try_parse_url` to tolerate invalid URL and return NULL instead. (spark.sql.ansi.enabled=true)DataFusion: parse_url.slt#L346-L347 expects the error without enabling ANSI mode.
Expected behavior
Read enable_ansi_mode like the other ANSI aware functions in the crate (for example round): return NULL for a malformed URL when it is off and raise INVALID_URL when it is on. This is a small part of apache/datafusion#23929.
Additional context
Found by running the queries in datafusion/sqllogictest/test_files/spark on a local Spark and comparing the results with the expected values in the files, which is the kind of check proposed in #7. Spark results come from Apple's build of Spark 4.0.0 (4.0.0.65-apple) with a UTC session and spark.sql.ansi.enabled=false unless stated. The Spark source links point at the v4.0.0 tag. DataFusion results are the sqllogictest expectations at 9dec77584, which CI enforces. Anything marked as not run was derived from reading the code.
Activity
TechWizard9999 commentedon Nov 5, 2026
Hi @kosiew @comphead , I'd like to work on this. My plan is to read enable_ansi_mode in parse_url like round does return Null when off and update parse_url.slt.
This is my appraoch in my mind !
Please assign this issue to me
Thanks!!
kosiew commentedon Nov 6, 2026
@TechWizard9999
Assigned.
TechWizard9999 commentedon Nov 6, 2026
Hi @kosiew @comphead , A small doubt like can I work on 2 issues at the same time as the review takes time right ?
Please guide me through this
Thanks!!
comphead commentedon Nov 6, 2026
would be nice to fix this, its related to apache/datafusion-comet#7330
TechWizard9999 commentedon Nov 7, 2026
Sure, will take that
@comphead
TechWizard9999 commentedon Nov 7, 2026
Hi @comphead thanks for the pointer to #7330. I'd like to work on this,I have a blocker first.
The installation guide lists only Linux and Apple Silicon macOS for building from source. I don't have a Linux machine and I can't install WSL so I can't build or run the Comet tests locally , is ther eany way to run those CI tests if tehre is any method(github actions ,workflow ) please let me know
Thanks!!