Skip to content

feat(csharp/drivers/bigquery): add BigQuery ADBC driver - #1192

Merged
CurtHagenlocher merged 44 commits into
apache:mainfrom
davidhcoe:dev/bigquerydriver
Oct 23, 2023
Merged

CurtHagenlocher merged 44 commits into
apache:mainfrom
davidhcoe:dev/bigquerydriver

Conversation

@davidhcoe

Copy link
Copy Markdown
Contributor
  • introduces ADBC support for connecting to BigQuery
  • minor fixes to other projects to align with the pattern

@davidhcoe
davidhcoe requested a review from lidavidm as a code owner October 11, 2023 19:44
@github-actions

Copy link
Copy Markdown

⚠️ Please follow the Conventional Commits format in CONTRIBUTING.md for PR titles.

@davidhcoe
davidhcoe marked this pull request as ready for review October 12, 2023 13:49
@davidhcoe

Copy link
Copy Markdown
Contributor Author

@lidavidm -- this is ready now. There was some strangeness with the line endings that the pre-commit check in kept fighting me on (even though the local one would pass). It seems the answer is to have the .csproj files with CRLF and the rest have LF for some reason.

@lidavidm

Copy link
Copy Markdown
Member

I had thought csproj wanted CRLF, but we can adjust that, sorry 😅

@CurtHagenlocher CurtHagenlocher left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main feedback is about the generation of schema queries.

Comment thread csharp/src/Client/AdbcCommand.cs Resolved
public override Schema GetTableSchema(string catalog, string dbSchema, string tableName)
{
string query = string.Format("SELECT * FROM `{0}`.`{1}`.INFORMATION_SCHEMA.COLUMNS WHERE table_name = '{2}'",
catalog, dbSchema, tableName);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity (and not because it should block anything) have you investigated whether the schema hierarchy can be combined into a single query? BigQuery has pretty decent support for structured columns, so it ought to be possible to format things together in the query and reduce round trips. (My assumption is that this would be faster than non-concurrent round trips; that would need to be tested, of course.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CurtHagenlocher CurtHagenlocher left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked a little more closely in a few places and spotted things I missed the first time. In addition to the SQL injection risk I hinted at earlier (without explicitly calling out) I think the lack of precision and scale on numeric results makes them unusable and there are two places where null scalar values wouldn't get reported.

Comment thread csharp/src/Client/AdbcCommand.cs Resolved
Comment thread csharp/src/Drivers/BigQuery/readme.md Resolved
Comment thread csharp/src/Drivers/BigQuery/readme.md Resolved
@CurtHagenlocher
CurtHagenlocher merged commit b3883cf into apache:main Oct 23, 2023
@lidavidm lidavidm added this to the ADBC Libraries 0.8.0 milestone Oct 23, 2023
@davidhcoe
davidhcoe deleted the dev/bigquerydriver branch April 17, 2024 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants