JSONL File Scan reads a JSON null as the text "null" #8674

What happened?

The JSONL File Scan reads a JSON null as the text "null". One null in a column is enough to make the whole column STRING, so a column of numbers comes out as text and its missing value is the word null rather than an empty cell. A sort, a filter or any arithmetic downstream then works on the wrong type.

The cause is JSONUtils.JSONToMap: a null is a value node, so it takes the child.asText() branch, and NullNode.asText() returns "null". Schema inference then fails to parse "null" as a number and widens the column to STRING.

Two assertions in JSONUtilsSpec expect "null" today. They were added in #4716, whose description says the spec pins the current behavior so that a fix has to update it on purpose.

How to reproduce?

1. Upload a JSONL file with these two lines:

   {"a":1}
   {"a":null}
   

2. Add a JSONL File Scan that reads it and run the workflow.
3. Column a is inferred as STRING, and the two rows hold the text "1" and the text "null". Expected: INTEGER, with 1 and an empty value.

Version/Branch

1.4.0-incubating-SNAPSHOT (main)

Commit Hash (Optional)

53052c4

What browsers are you seeing the problem on?

No response

Relevant log output

Schema[Attribute[name=a, type=string]]
row 1: '1' (String)
row 2: 'null' (String)