Skip to content

Conversation

@birschick-bq
Copy link
Contributor

@birschick-bq birschick-bq commented Apr 2, 2025

This change fixes a problem when connecting to a Databricks data source using the adbc.hive.host and adbc.hive.path instead of uri.

Databricks connections always require an encrypted connection, so the default of false for option adbc.http_options.tls.enabled does not correctly apply to a Databricks connection. This change makes the default True for Databricks and all HTTP-based drivers. Callers can provide a uri with http: scheme or set the adbc.http_options.tls.enabled to False to disable TLS communication.

Further, there are improvements in the test environment to support all options in the http_options.tls.* namespace

@github-actions github-actions bot added this to the ADBC Libraries 18 milestone Apr 2, 2025
@birschick-bq birschick-bq marked this pull request as draft April 2, 2025 21:23
@birschick-bq birschick-bq marked this pull request as ready for review April 2, 2025 21:41
@CurtHagenlocher
Copy link
Contributor

My overall feedback before going into any individual details is that I don't think this change is correct. It might be true that there are Spark services such as Databricks or HDInsight (does that even still exist) which are always managed and for which these options are not worthwhile, but it's also true that anyone can stand up a Spark instance for which they might need to set these properties. It does not make sense to limit this driver in ways that won't work for that case.

Copy link
Contributor

@CurtHagenlocher CurtHagenlocher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick turnaround! As per my other comment, I don't think this is the right approach to solving the problem.

@birschick-bq birschick-bq marked this pull request as draft April 3, 2025 00:06
@birschick-bq birschick-bq changed the title fix(csharp/src/Drivers/Apache): Set tls enabled to true for Databrick driver, by default fix(csharp/src/Drivers/Apache): Set tls enabled to true all HTTP-based drivers, by default Apr 3, 2025
@birschick-bq birschick-bq marked this pull request as ready for review April 3, 2025 02:40
@birschick-bq
Copy link
Contributor Author

Thanks for the quick turnaround! As per my other comment, I don't think this is the right approach to solving the problem.

@CurtHagenlocher - TLS enabled is now default for all HTTP-based drivers.

Copy link
Contributor

@CurtHagenlocher CurtHagenlocher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@CurtHagenlocher CurtHagenlocher merged commit 0a9d8c1 into apache:main Apr 3, 2025
9 checks passed
colin-rogers-dbt pushed a commit to dbt-labs/arrow-adbc that referenced this pull request Jun 10, 2025
…d drivers, by default (apache#2667)

This change fixes a problem when connecting to a Databricks data source
using the `adbc.hive.host` and `adbc.hive.path` instead of `uri`.

Databricks connections always require an encrypted connection, so the
default of `false` for option `adbc.http_options.tls.enabled` does not
correctly apply to a Databricks connection. This change makes the
default `True` for Databricks and all HTTP-based drivers. Callers can
provide a `uri` with `http:` scheme or set the
`adbc.http_options.tls.enabled` to `False` to disable TLS communication.

Further, there are improvements in the test environment to support all
options in the `http_options.tls.*` namespace
@birschick-bq birschick-bq deleted the dev/birschick-bq/fix-https-databricks branch June 11, 2025 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants