Skip to content

docs(observe): OTLP/HTTP gzip and the X-Project-Name header [TH-8743] [TH-8744] - #1028

Open
nik13 wants to merge 9 commits into
mainfrom
docs/TH-8743-TH-8744-otlp-gzip-project-header
Open

nik13 wants to merge 9 commits into
mainfrom
docs/TH-8743-TH-8744-otlp-gzip-project-header

Conversation

@nik13

@nik13 nik13 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Documents future-agi/future-agi#3590 on the Observe > Reference > Export and endpoints page.

  • TH-8743: gzip. OTLP/HTTP accepts gzip (gzip / x-gzip, one layer, protobuf or JSON). The 16 MiB limit applies to the body as sent and, for gzip, again after decompression, as the OpenTelemetry Collector applies max_request_body_size to both. Corrupt gzip is 400, over either limit is 413, and an unknown coding or more than one gzip layer is 415. gRPC stays uncompressed-only (UNIMPLEMENTED). Earlier collectors ignored Content-Encoding, so a mislabelled plain body used to be accepted; the page now says so.
  • Busy collector. Each collector limits how much gzip it decompresses at once. A request takes its share only after it has been received in full, waits up to 2 seconds in arrival order, then gets 503 + Retry-After: 1, which exporters retry. The self-hosted note covers FI_GZIP_INFLIGHT_MIB (default 128 MiB, or GOMEMLIMIT/8 when lower, never below 16 MiB), the 16 MiB reservation while a body's decompressed size is counted, and that the budget counts decompressed body bytes only (size the memory limit as for uncompressed requests).
  • Nesting. More than 10,000 protobuf message levels or JSON object/array levels (roughly 5,000 nested protobuf arrays) is rejected: 400 over HTTP, INTERNAL over gRPC.
  • TH-8744: project header.
    • Per-ResourceSpans precedence; FI-Project-Name; auth-on and auth-disabled behaviour.
    • 255-character and UTF-8 checks.
    • Repeated or comma-joined values are rejected (RFC 9110 section 5.3).
    • Rejected characters: control characters, line/paragraph separators, Cf format characters, and other default-ignorable characters. A value with no visible character (only combining marks, or symbols that display as blank space such as U+2800) is rejected too.

The page describes shipped behaviour only. It cites no internal tickets and no planned work.

Merge after future-agi/future-agi#3590 is deployed. Until then Cloud's collector does not implement these changes.

Pages

Page Change
observe/reference/export-formats.mdx Endpoint protocol/compression/project/error rows; Compression section (four bullets: turning gzip on, encoding must match the body, request size, busy collector); Naming the project section and header table; request limits and nesting; self-hosted FI_GZIP_INFLIGHT_MIB note; auth-disabled compatibility note.

Vendor recipe pages

Not changed here. Open recipe PRs can update their own instructions after #3590 ships. HTTP recipes may use the project header; gRPC exporters must stay uncompressed. Existing recipe PRs that still transform or redact spans continue to need their collector.

Build

On 66464f90: npm run build passed (Astro built 1,181 pages; Pagefind indexed 716) and npm run audit-links found 0 broken nav or content links (29 pre-existing orphan pages). The built page contains the updated text, including the blank-name row and the reworded self-hosted note. No internal ticket ids or local paths are on the page.

GitHub CI on 66464f90: Link audit & redirect enforcement passes.

Squash-merge message

Paste this into GitHub's "Squash and merge" dialog, replacing the default text (which would copy every commit message). Title:

docs(observe): OTLP gzip, request limits and the X-Project-Name header (#1028)

Body:

Observe > Reference > Export and endpoints:
- Compression: OTLP/HTTP accepts gzip (one layer, protobuf or JSON) and
  gRPC stays uncompressed. How to turn gzip on in OpenTelemetry SDKs and
  the Collector, the 400, 413 and 415 cases, and the 503 + Retry-After a
  busy collector returns, which exporters retry.
- Request limits: 16 MiB as sent and, for gzip, again after
  decompression. Nesting deeper than 10,000 levels is 400 over OTLP/HTTP
  and INTERNAL over gRPC.
- Naming the project: the X-Project-Name header (alias FI-Project-Name),
  its precedence after the project_name resource attribute, an example
  exporter configuration, and the header values that are rejected.
- Self-hosted note: FI_GZIP_INFLIGHT_MIB, its defaults and what it
  counts.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

🤖 Generated with Claude Code

nik13 and others added 2 commits October 21, 2026 15:50
…743] [TH-8744]

Endpoint contract on the export and endpoints reference:

- Compression: gzip on OTLP/HTTP (Content-Encoding) and gRPC
  (grpc-encoding); 415 for other encodings, 400 for a corrupt body.
- Naming the project: project_name resource attribute first, then the
  X-Project-Name header (gRPC metadata x-project-name), then 400. Same
  lookup and auto-create as the attribute; FI-Project-Name alias;
  conflicting values and invalid UTF-8.
- Request limits: 16 MiB applies after decompression, and to the
  compressed body.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…TH-8743] [TH-8744]

- gRPC stays uncompressed: keep compression: none on the Collector otlp exporter.
- Content-Encoding must match the body; other codings are 415.
- A busy collector answers gzip requests 503 with Retry-After: 1.
- The header example sets OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf.
- Header values over 255 characters or with control characters are 400.
- The lazy-check note covers every header check; the header is trimmed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@nik13 nik13 changed the title docs(observe): OTLP gzip and the X-Project-Name header [TH-8743] [TH-8744] docs(observe): OTLP/HTTP gzip and the X-Project-Name header [TH-8743] [TH-8744] Oct 22, 2026
nik13 and others added 7 commits October 21, 2026 20:25
…LIGHT_MIB [TH-8743]

Matches future-agi/future-agi#3590 round 3: the collector's gzip budget now
counts what decoding a request will take, so a gzip batch too big to decode
within it gets 413, and busy gets 503. Says gzip is optional (uncompressed
requests are not budgeted), that x-gzip is gzip and only one gzip layer is
accepted, that OTLP/HTTP refuses nesting past 10,000 levels, and names the
self-hosted FI_GZIP_INFLIGHT_MIB setting.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…43] [TH-8744]

Document the 16 MiB OTLP/HTTP limit after decompression (no separate gzip
wire-size cap), the gzip reservation lifecycle and exact budget scope, the
10,000-level message/JSON guard, and the TH-8745 decode-amplification follow-up.
Document per-resource project precedence, auth-disabled compatibility, RFC
9110 rejection of repeated/comma-joined header values, and Unicode Cf rejection.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…r characters [TH-8743] [TH-8744]

- The 16 MiB limit applies to the body as sent and, for gzip, again
  after decompression, as the OpenTelemetry Collector applies
  max_request_body_size. Drop "the compressed wire stream has no
  separate size cap".
- A gzip request takes its decompressed size from the budget only after
  it has been received in full; say what that means for concurrency.
- The nesting limit covers OTLP/gRPC too (INTERNAL).
- Line/paragraph separators and default-ignorable characters in
  X-Project-Name are a 400.
- Note that earlier collectors ignored Content-Encoding.
- Remove internal ticket references from the public page.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…[TH-8743]

The gzip request is read in full before taking a budget share, but the
count/inflate pass must be budgeted too. Document the 16 MiB initial
reservation after upload, its shrink to the exact decompressed size, and
the fact that both inflate passes—but not the network upload—are under
the lease. Explain that small bodies release unused budget after counting.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…cket [TH-8743]

Clarify that the 16 MiB body limit and gzip decompressed-byte budget do
not limit allocations caused by decoding element-heavy requests, and that
hardening for that is planned separately. Keep internal issue IDs off the
public page.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The busy-collector bullet now says what an exporter sees: a gzip request
takes its share only after it is received in full, waits up to 2 s, then
gets 503 + Retry-After. The self-hosted note keeps the FI_GZIP_INFLIGHT_MIB
sizing in plain terms. The sentence about planned decode-cost work is
removed; the page describes shipped behaviour only.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The project-header table now lists a value with no visible character
(only combining marks, or symbols that display as blank space such as
U+2800) as rejected with 400 / INVALID_ARGUMENT. The self-hosted note says
the gzip budget counts decompressed body bytes only and to size the
collector's memory limit as for uncompressed requests.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant