Skip to content

Support reading json string into binary data type. #701

@zijie0

Description

@zijie0

Describe the bug

When decoding json string into binary data type, arrow-rs will return JsonError("Binary type is not supported").

To Reproduce

Running the following test could reproduce the error.

#[test]
fn test_json_read_binary_structs() {
    let schema = Schema::new(vec![Field::new("c1", DataType::Binary, true)]);
    let decoder = Decoder::new(Arc::new(schema), 1024, None);
    let batch = decoder
        .next_batch(
            &mut vec![
                Ok(serde_json::json!({
                    "c1": "₁₂₃",
                })),
                Ok(serde_json::json!({
                    "c1": "foo",
                })),
            ]
            .into_iter(),
        )
        .unwrap()
        .unwrap();
    let data = batch.columns().iter().collect::<Vec<_>>();

    let schema = Schema::new(vec![Field::new("c1", DataType::Binary, true)]);
    let binary_values = BinaryArray::from(vec![
        "\u{2081}\u{2082}\u{2083}".as_bytes(),
        "foo".as_bytes(),
    ]);
    let expected_batch =
        RecordBatch::try_new(Arc::new(schema), vec![Arc::new(binary_values)])
            .unwrap();
    let expected_data = expected_batch.columns().iter().collect::<Vec<_>>();

    assert_eq!(data, expected_data);
    assert_eq!(batch.num_columns(), 1);
    assert_eq!(batch.num_rows(), 2);
}

Expected behavior

The above test should pass.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions