Core, OpenApi: Add X-Iceberg-Client-Capabilities header by singhpk234 · Pull Request #16394 · apache/iceberg

singhpk234 · 2026-05-18T18:07:10Z

Summary

Introduces a new X-Iceberg-Client-Capabilities HTTP header that lets REST clients
advertise the capabilities they support to the catalog server. The header is
informational, sent on every catalog request, and is intended as a forward-compatibility
mechanism — not a security mechanism.

This was brought up at the Read Restrictions community sync on May 12, 2026
(recording); a dev list thread is being raised
in parallel for broader discussion.

Motivation

Iceberg's REST protocol keeps evolving — server-side scan planning, vended
credentials, remote signing, and the in-flight fine-grained access control
proposal (#13879) are all features that catalogs may return but older clients
can't handle. Today there's no standard way for a client to declare "I
understand X" to the server. This PR introduces a single, generic mechanism
for the client to declare its capability set up front.

The header pattern mirrors X-Client-Version and X-Client-Git-Commit-Short
(already added on every request via HTTPClient.baseHeaders) — it is a stable
SDK-attribute hint, not a per-request preference.

Design notes

Forward-compat hint, not security. The header is informational — clients
can trivially spoof its value, so servers MUST NOT base trust or
authorization decisions on it. Trust establishment (mTLS, OAuth, etc.) is
out of scope. The parameter description in the spec calls this out
explicitly so it can't be missed.
enum: constraint on the schema. Mirrors the existing
X-Iceberg-Access-Delegation parameter. Adding a new capability is a
one-line spec edit. Generated clients can validate values; servers get a
machine-readable closed list of recognized tokens.
Independent of X-Iceberg-Access-Delegation. Despite token overlap
(vended-credentials, remote-signing appear in both), the headers carry
different semantics: data-access is a per-request preference ("I want
this for this call"); client-capabilities is a static SDK attribute
("I am built to handle this").
SDK overrides user config. The Java SDK's static value overrides any
user-configured header.X-Iceberg-Client-Capabilities property — same
pattern as the existing X-Client-Version / X-Client-Git-Commit-Short
headers. Pinned by an explicit test.

Future work

Add read-restrictions to the enum once Spec: Add finer grained read restrictions as part of loadTable #13879 (FGAC) lands. The framing
in that PR's review (catalogs MUST NOT return read-restrictions to
clients that don't advertise the capability) depends on this header being
available.
Other client SDKs (PyIceberg, Rust, Go) can mirror this enum to advertise
the same capability set.

Changes from the prior draft:

Added the May 12 sync reference + recording link in the Summary
Replaced the "free-form, no enum" design note with the "enum: constraint" rationale
Added the spoof-disclaimer note explicitly
Added a new bullet about SDK-overrides-user-config (now pinned by test)

Introduces a new HTTP header that lets REST clients advertise their supported capabilities to the catalog server on every request. * New ClientCapability enum (vended-credentials, remote-signing, scan-planning) as the single source of truth for supported capabilities, with javadoc describing the protocol each capability implies. * HTTPClient sets X-Iceberg-Client-Capabilities statically in build() alongside X-Client-Version and X-Client-Git-Commit-Short, so the header is included on every request and overrides any user-supplied header.* property. * OpenAPI client-capabilities parameter component documents the defined values without an enum constraint, allowing new capabilities to be added without schema changes. The header is optional; servers must not fail if it is absent.

steveloughran

commented as someone who has read far too many RFCs.

sungwy · 2026-05-19T13:52:06Z

+        Defined capability values:
+
+        - `vended-credentials`: The client supports receiving and using
+          storage credentials vended by the catalog server.
+
+        - `remote-signing`: The client supports delegating request signing
+          to a remote signing service provided by the catalog server.
+
+        - `scan-planning`: The client supports server-side scan planning.


I wonder if these capability values should include a major compatibility version, since values like read-restrictions may evolve in ways that older clients cannot safely consume. Would read-restrictions.v1 make more sense, so that incompatible future semantics can be represented as read-restrictions.v2?

This would allow us to introduce a new column mask subtype in read-restrictions without worrying about breaking the existing capability header contract, by simply introducing a new capability version.

From spec pov of the read restrictions we just need "if you don't understand you fail" baked into it so if new masks get added it would lead client to fail, presently there is no way for the older clients to bake this in.

Versioning is good call out, my understanding is this stems from fact that rest spec is itself not versioned, hence we always introduce changes in a backward compatible way, so if one uses v1 client (in terminology) and send v2 read restrictions (new mask) ... expectation is it should fail since the spec says if you dont understand you fail.

Please let me know what do you think considering above

+1 on the versioning. Especially if we extend the expression language, clients will not be able to express how much of the expression language they understand.

+1 from me as well to versioned capabilities.

we discussed a bit about it 05/25, ReadRestrictions sync, please have a pass and let us know what you think in context of that, while versioning would be ideal, but for now considering where we are how we have been maintaining spec (in a backward compatible way) we can park that discussion, specially for read restrictions since we just need a signal that, hey i understand what "ReadRestrictions are" and not "I understand everything within the read restrictions"

- should → SHOULD, must → MUST, must not fail → SHALL NOT reject - can → MAY

adutra · 2026-05-28T10:05:30Z

+        Defined capability values:
+
+        - `vended-credentials`: The client supports receiving and using
+          storage credentials vended by the catalog server.
+
+        - `remote-signing`: The client supports delegating request signing
+          to a remote signing service provided by the catalog server.
+
+        - `scan-planning`: The client supports server-side scan planning.


+1 from me as well to versioned capabilities.

The schema was defined as a single enum constant rather than a comma-separated list of enum constants. Following the same fix as apache#16594 for data-access, declare it as an array of enums so the example value (a comma-separated string) actually matches the wire format produced by style: simple, explode: false.

stevenzwu · 2026-06-02T04:23:47Z

+   * Comma-separated list of every capability's {@link #headerValue()}, suitable for use as the
+   * {@code X-Iceberg-Client-Capabilities} request header value.
+   */
+  public static final String HEADER_VALUE =


does current iceberg-core rest client have all 3 capabilities?

vended credentials: yes (via SupportsStorageCredentials)

remote signing: is it only available in aws module?

scan planning: DTOs and parsers are there. but does it need engine integration?

remote signing: is it only available in aws module?

This endpoint got promoted to a table level which now works for all storage provider

scan planning: DTOs and parsers are there. but does it need engine integration?

Spark already supports its so does the pure iceberg java client if we were to use iceberg generics to read this

All three supported ATM

Spark already supports its so does the pure iceberg java client if we were to use iceberg generics to read this

We hardcode these capabilities in iceberg-core. Any engine/client uses the new Java will have the capability header set. but what if the engine doesn't support remote scan planning?

Move the purpose paragraph before the security caveat so the header description leads with what the header is for.

danielcweeks · 2026-06-03T14:27:47Z

      explode: false
      example: "vended-credentials,remote-signing"

+    client-capabilities:


I'm not sure this is the right way to go about this. The more I think about what we're doing here, the more concerned I get that what we're really doing is creating a clever way to workaround the right way to do this, which is using versioned endpoints.

Capabilities like read restrictions means you're changing the semantics of the endpoint but rather than forcing clients through a new path, we're triggering different behavior based on sidecar information (in this case headers).

Why not introduce a loadTable v2? We can handle a number of issues that have come up in the past:

credentials in both config and the storage creds structure

how to handle tables with scan plan but we don't want to expose snapshots

read restrictions must be understood

I just feel like the right way to do this would be just spec out a v2 endpoint and address many of these concerns at once.

github-actions Bot added core OPENAPI labels May 18, 2026

singhpk234 changed the title ~~Core, REST: Add X-Iceberg-Client-Capabilities header~~ Core, OpenApi: Add X-Iceberg-Client-Capabilities header May 18, 2026

sfc-gh-prsingh force-pushed the x-iceberg-client-capabilities-header branch from eb2cb6f to b453c26 Compare May 18, 2026 18:40

singhpk234 requested review from RussellSpitzer, amogh-jahagirdar, danielcweeks, nastra, rdblue and stevenzwu May 18, 2026 19:04

singhpk234 marked this pull request as ready for review May 18, 2026 20:58

steveloughran reviewed May 19, 2026

View reviewed changes

Comment thread open-api/rest-catalog-open-api.yaml Resolved

Comment thread open-api/rest-catalog-open-api.yaml Resolved

Comment thread open-api/rest-catalog-open-api.yaml Resolved

sungwy reviewed May 19, 2026

View reviewed changes

Address review: use RFC 2119 keyword capitalization

b06e27e

- should → SHOULD, must → MUST, must not fail → SHALL NOT reject - can → MAY

adutra reviewed May 28, 2026

View reviewed changes

singhpk234 requested review from adutra and sungwy May 30, 2026 18:42

stevenzwu reviewed Jun 2, 2026

View reviewed changes

OpenAPI: Reorder X-Iceberg-Client-Capabilities description

54c8915

Move the purpose paragraph before the security caveat so the header description leads with what the header is for.

singhpk234 requested a review from stevenzwu June 2, 2026 16:48

danielcweeks reviewed Jun 3, 2026

View reviewed changes

danielcweeks mentioned this pull request Jun 3, 2026

Spec: Add finer grained read restrictions as part of loadTable #13879

Merged

singhpk234 closed this Jun 23, 2026

rahulsmahadev mentioned this pull request Aug 19, 2026

OpenAPI: Recommend a standard User-Agent format for REST clients #17727

Merged

Conversation

singhpk234 commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation

Design notes

Future work

Uh oh!

steveloughran left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

singhpk234 commented May 18, 2026 •

edited

Loading