[Python] pyarrow.json.read_json ignores nullable=False in explicit_schema parse_options

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major
    • None
    • Affects Version/s: None
    • Component/s: Python
    • None

      Reproducible example:

      import json
      import pyarrow.json as pj
      import pyarrow as pa
      
      s = {"id": "value", "nested": {"value": 1}}
      
      with open("issue.json", "w") as write_file:
          json.dump(s, write_file, indent=4)
      
      schema = pa.schema([
          pa.field("id", pa.string(), nullable=False),
          pa.field("nested", pa.struct([pa.field("value", pa.int64(), nullable=False)]))
      ])
      
      table = pj.read_json('issue.json', parse_options=pj.ParseOptions(explicit_schema=schema))
      
      print(schema)
      # id: string not null
      # nested: struct<value: int64 not null>
      #   child 0, value: int64 not null 
      print(table.schema)
      # id: string
      # nested: struct<value: int64>
      #   child 0, value: int64

            Assignee:
            Unassigned
            Reporter:
            Alenka Frim
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: