Skip to content

Proxy Settings for catalog REST API client #12059

@sfc-gh-dtriana

Description

@sfc-gh-dtriana

Feature Request / Improvement

Despite configuring a proxy for the driver or executor

spark.driver.extraJavaOptions="-Dhttp.proxyHost= -Dhttp.proxyPort= -Dhttps.proxyHost= -Dhttps.proxyPort="
spark.executor.extraJavaOptions="-Dhttp.proxyHost= -Dhttp.proxyPort= -Dhttps.proxyHost= -Dhttps.proxyPort="

Calls made by the Spark engine using Iceberg REST API do not follow the above proxy configurations,

This is a feature request for the ability to configure proxy settings when connecting to an Iceberg Rest API Catalog via Spark. This is especially critical for organizations where network access to catalog services is restricted and must go through a corporate proxy.

Query engine

Spark

Willingness to contribute

  • I can contribute this improvement/feature independently
  • I would be willing to contribute this improvement/feature with guidance from the Iceberg community
  • I cannot contribute this improvement/feature at this time

Metadata

Metadata

Assignees

No one assigned

    Labels

    improvementPR that improves existing functionality

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions