Skip to content

[FLIGHT][GO] CommandGetTables metadata being ignored #2434

@xborder

Description

@xborder

What would you like help with?

Hi,

I'm testing a connector that uses ADBC Go driver against a flight server. While I can obtain information about catalogs, schemas and tables, when it comes to the columns everything is empty.

I believe the issue is with the GO driver and compliance with FlightSQL but I'd like to get some validation on my interpretation.
FlightSQL specification states the following for CommandGetTables response:

 ....

  The returned Arrow schema will be:
  <
   catalog_name: utf8,
   db_schema_name: utf8,
   table_name: utf8 not null,
   table_type: utf8 not null,
   [optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
                                            it is serialized as an IPC message.)
  >
  Fields on table_schema may contain the following metadata:
   - ARROW:FLIGHT:SQL:CATALOG_NAME      - Table's catalog name
   - ARROW:FLIGHT:SQL:DB_SCHEMA_NAME    - Database schema name
   - ARROW:FLIGHT:SQL:TABLE_NAME        - Table name
   - ARROW:FLIGHT:SQL:TYPE_NAME         - The data source-specific name for the data type of the column.
   - ARROW:FLIGHT:SQL:PRECISION         - Column precision/size
   - ARROW:FLIGHT:SQL:SCALE             - Column scale/decimal digits if applicable
   - ARROW:FLIGHT:SQL:IS_AUTO_INCREMENT - "1" indicates if the column is auto incremented, "0" otherwise.
   - ARROW:FLIGHT:SQL:IS_CASE_SENSITIVE - "1" indicates if the column is case-sensitive, "0" otherwise.
   - ARROW:FLIGHT:SQL:IS_READ_ONLY      - "1" indicates if the column is read only, "0" otherwise.
   - ARROW:FLIGHT:SQL:IS_SEARCHABLE     - "1" indicates if the column is searchable via WHERE clause, "0" otherwise.
....
  • I can see this information being returned to the driver so the server I'm testing against is respecting the specification.
  • Regardless of the server returns, the result of GetObjects doesn't have type information about the existing columns
  • While debugging the driver I think the issue is in how the column information is being processed (in [shared_utils.go](https://github.com/apache/arrow-adbc/blob/c3149c0746d09877709e7625b7199e102d6d9c94/go/adbc/driver/internal/shared_utils.go#L378))
  • If there's the columns contain metadata, there is no code looking for the metadata keys described in the specification
  • JAVA ADBC seems to have a different implementation where these metadata keys are used

Questions:

  1. Does this mean that GO driver currently is not respecting the FlightSQL spec?
  2. If this is the case what should it look for in the metadata? The ARROW:FLIGHT:SQL: keys described in the spec or the xdbc_ keys? Should it look for ARROW:FLIGHT:SQL: and fallback to xdbc_?
  3. Where can I get more information about these xdbc_ metadata fields?

Thank you

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions