Cannot query ARRAY columns using Arrow Flight JDBC Driver

[mez]

When using 3rd-party SQL tools with the Apache Arrow Flight JDBC driver, like DBeaver or Alation Compose for example, I see consistent error messages indicating the driver does not support columns with data type defined as ARRAY.

Here’s DBeaver’s result when querying an ARRAY column using the sql “SELECT loc FROM Samples.samples.dremio.com.zips.json” in Dremio - error is “IndexOutOfBoundsException”:

When querying a different column (also defined as an ARRAY) in Alation’s “Compose” sql tool, we see the error “DataTables warning: table id=DataTables_Table_0 - Requested unknown parameter ‘22’ for row 0, column 22”. This SQL tool uses the same Arrow driver.

After some digging, it seems like the arrow driver does not support the Array column type.

The ask:

1. I would really like to see array support added to the Arrow driver. So would my clients.

2. I would love to hear of any workarounds in the meantime.

A question:

- Since the Arrow Flight driver is built and maintained by Apache, how can users and partners request features and influence the roadmap?

Thanks.

[balaji.ramaswamy]

@mez Goof find, I am able to reproduce the issue but a few other struct columns work, while a different LIST column failed. Let me check this and get back to you

[mez]

Thanks @balaji.ramaswamy I really appreciate you looking into this!

[mez]

@balaji.ramaswamy Happy new year! Did your checks come up with anything? Perhaps driver support for arrays is on the roadmap this year?

[Helder_Gregorio]

Hi @mez,

This is an issue with the arrow flight jdbc driver. The driver was not respecting the JDBC spec. I’ve already committed a fix to the community GH-964: Fix IndexOutOfBoundsException in Array.getResultSet() for JDBC clients by xborder · Pull Request #965 · apache/arrow-java · GitHub.