fix(openai): close privacy and SDK lifecycle release blockers - #571

Current Python modality correction (2026-11-05)

Final correction head 4aebaaad2a823c2404d3d51ea4a5ee073a332346, tree 919f4247d185fccdc0bd78f83221f328009f1366. The normal push follows this metadata update so the resulting head receives one uninterrupted CI cycle. The existing branch now includes a normal merge of current dev, followed by the scoped Python core/Fireworks correction. Exact-head CI is being observed; no merge or release was performed. The older qualification details below are historical, not the current gate.

Privacy contract

• Core's narrow ChatInput representation (imported from fi_instrumentation.instrumentation.config) is accepted only at the exact input keys. Core validates and copies passive built-in data, applies modality policy, then serializes. It never accepts a boolean/trusted-string bypass, parses opaque JSON, or invokes custom mappings, serializers, iterators or callable values.
• Opaque/lazy raw input remains fail-closed and unevaluated. Malformed/forged structures fail closed; unknown fields are omitted, and unrecognized/opaque modality content is redacted under granular policies.
• Fireworks passes original typed request data through the supported FI span setter. Image hiding preserves permitted text; text hiding preserves typed image references. Input/message hiding, base64 limits, invocation/transport exclusions, model/usage and output directionality remain enforced. No TraceConfig swap or underlying OTel-span write is used. Creation-time sampler-returned opaque strings still fail closed.
• The published Fireworks core floor remains 1.2.2 with its existing path. The new path is used only when the core API exists; malformed values never fall back to the old serializer. Ordinary capture and caller request/response objects are unchanged.

Fresh cloud evidence

All installation/build/test execution used four serial disposable E2B sandboxes, synthetic provider HTTP and actual protobuf OTLP. All were killed and verified absent; no local tests or real-provider certification.

• Exact unchanged 24 original Fireworks failures reproduced on current dev+PR source at OpenAI 1.0.0 / 1.69.0 / 3.45.0; before/after third-party dependencies matched. All 24 now pass at every bound, with all 68 inherited core/Fireworks test files byte-identical.
• Final full Fireworks: 328 passed / 14 upstream-unavailable API skips at 1.0.0; 342 passed at each newer bound. This includes 11 additive regressions; no original assertion was edited.
• Final core privacy: 99 passed per bound. Complete non-framework core suite with its gRPC extra: 862 passed. New negatives cover malformed, custom, callable and forged values, unknown fields, hidden serializers, alias attempts, PII and actual OTLP output.
• Retained independent socket/OTLP probe: 32 passed per bound, including sync/async normal, stream, raw and raw-stream, unchanged provider requests, response parsing, credentials and model/usage. Its unchanged permitted-modality assertions now pass.
• OpenAI: 798 passed / 4 intentional legacy-path skips at 1.69.0 and 3.45.0; floor Fireworks-route 16 passed / 87 unavailable/shared-package-floor skips. vLLM: 117 passed at every bound. DeepSeek: 104 passed; representative Tavily privacy/span suites: 65 passed.
• Separate installed published-core-1.2.2 control: 326 passed / 16 skips (14 native API gaps plus 2 new-core-only API tests). This is a compatibility control, never a substitute for candidate-core qualification.
• Final installed core/Fireworks runtime hashes match the exact tested Git tree at all three bounds. The first run's 10 supplemental gRPC-test failures were due to the harness omitting the optional gRPC exporter; the unchanged suite passed after installing that extra. Original failed logs are retained.

The first pushed head exposed a new mixed-package import regression: pytest collecting the repository's python package alongside installed older core tried to eagerly import the new module from that older core. The final follow-up removes that eager top-level export (the API remains in core config), adds a collection regression, and repeats the full candidate matrix. The exact import failure was reproduced with published core 1.1.0; the corrected import and all 111 unchanged Lepton published-recipe tests pass with OpenAI 3.24.0 / traceAI-openai 0.1.10. One external recipe replay initially omitted its explicit httpx harness dependency; that collection error and the corrected pass are both retained. No test assertions or dependency floors were weakened.

The earlier PR body edit automatically restarted its Python workflow because this repository listens for pull_request.edited; that cancelled old run is not a product failure or a green gate. This final metadata update occurs before the final push to avoid repeating that interruption on the resulting head.

All six original and four follow-up repairs remain intact. OpenAI/vLLM/TypeScript implementation trees were not changed by this correction. PR #562 response-accumulator work was neither edited nor merged here; combined-tree qualification remains separate.

Qualification status

Do not merge on the older green checks. The repaired branch tip is d00a71f2786dd3c2b27ebabe582546f5f20a8c61, but GitHub currently reports PR #571 / refs/pull/571/head as fb04aede7c8cb7e4caea668e8d7f540fc78c8dc8. No workflow runs or check runs are registered for d00a71f2. The PR head must synchronize and CI must validate that exact head before merge. Nothing was merged or released.

The existing branch and PR were reused. All six assigned findings were independently reproduced against the original implementations, and the existing repairs were requalified in fresh E2B sandboxes. No additional implementation change was needed in this pass.

What was wrong and the fixes

1. Legacy vLLM leaked transport credentials. input.value included arbitrary request kwargs, including Authorization headers. Telemetry now allowlists semantic request fields; headers, extra query/body fields and other transport options never enter it. The SDK receives the original request unchanged.
2. Granular Python text privacy missed duplicate raw content. Core now masks opaque input/output/raw payloads before lazy serialization when directional text/message privacy applies, while preserving typed structured-message processors, safe model/usage metadata and typed image behavior.
3. Legacy streaming broke native context management. Sync/async ObjectProxy streams retain iteration, context managers, response access and close behavior. Span finalization is idempotent across exhaustion, early/unused exit, explicit close, errors and cancellation, including the response-level cleanup needed by OpenAI 1.0.
4. Cancellation leaked spans. Request and stream paths finalize on BaseException and re-raise the original cancellation. Error details still pass through FITracer privacy.
5. Mixed TypeScript CJS/ESM clients missed the second SDK. Patches are owned by concrete SDK class identity, with ownership shared across the instrumentor's public CJS/ESM builds. Disable removes every owned class, not another instrumentor's patches; immutable namespaces and duplicate aliases are supported.
6. OpenAI 1.0 streamed usage was omitted. Mapping-shaped and typed usage share extraction and validation. Reported non-negative integer token counts are exported; absent/invalid counts are not invented as zero. Sync/async real-SDK SSE regressions cover floor and current SDKs.

Required release ordering

This includes the fi-instrumentation-otel 1.2.3 candidate. Both Python integrations require >=1.2.3 and Python >=3.10. The repaired core must be published before the integrations in the later authorized release process. Dev-only local core dependencies are excluded from installed wheel requirements. This qualification does not publish packages, change release workflows or merge dev to main.

Fresh E2B evidence for d00a71f

Every test ran in disposable cloud sandboxes with synthetic credentials, mock providers/loopback servers and in-memory/loopback exporters. No real provider certification is claimed.

• rel-openai-fix-r1-recheck-baseline-1125: swapped the three original implementation files into the candidate test environment. Those implementations are unchanged on current dev. Original HTTP probes fail on both OpenAI 1.69.0 and 3.45.0: 5 failed / 2 passed, showing credential leakage, raw-content leakage, context-manager failure and a missing cancelled span. The committed legacy regressions on 1.0.0 fail 16 tests, including both streamed-usage tests. Packed mixed-import controls fail 2 tests / 2 pass at npm SDK 4.0.0 and 7.50.0; the ESM-only control passes.
• rel-openai-fix-r1-recheck-final-1125:
  • Python OpenAI: 798 passed / 4 skipped at 1.69.0 and 3.45.0.
  • Legacy vLLM: 117 passed at 1.69.0 and 3.45.0.
  • Independent HTTP probes: 7 passed at each version and 7 passed in an installed-wheel consumer.
  • Clean installed core/vLLM wheels with OpenAI 1.0.0: 117 package tests + 6 HTTP probes passed, including a real localhost SSE response whose dict-shaped usage exports 2 input / 3 output / 5 total tokens.
  • Wheel metadata requires core >=1.2.3, contains no local path dependencies, and imports from site-packages rather than source. Both installed consumers pass pip check.
  • TypeScript OpenAI Jest: 185 passed / 2 skipped; fi-core Jest: 164 passed.
  • Packed mixed-import/ownership consumers: 8 passed each at 4.0.0 and 7.50.0. Existing real-SDK contracts: 94 passed / 5 unavailable-API skips at floor; 162 passed at current.
• rel-openai-fix-r1-recheck-core-py310-1125: isolated core with OTel 1.45.2 and normal pytest collection: 828 passed / 6 skipped. Python 3.10 / OpenAI 1.0.0: 117 legacy package tests + 6 HTTP probes passed, plus pip check.

The prior private-OTel-_timeout assertions were replaced with requests send-boundary assertions in the existing repair commit; the current core suite passes. The fresh evidence also retains harness failures rather than hiding them: recheck-current-1125 omitted the tests' httpx dependency; recheck-current-r2-1125 omitted the cross-package Parallel README fixture. The final combined run passed every integration/consumer suite but failed core collection under forced --import-mode=importlib; isolated core with its normal collection passed all 828 tests. These were corrected in the external test staging, without modifying repository code.

All five fresh E2B sandboxes were destroyed and their deletion verified. Current-head GitHub CI remains an external qualification gate; older-head successes are not counted for d00a71f2.

Competitor comparison

Reviewed public documentation/source, not live competitor benchmarks:

• Arize OpenInference tracks separate CJS/ESM SDK classes. This repair closes that demonstrated identity/lifecycle gap and verifies per-instrumentor ownership across both public builds.
• Braintrust OpenAI integration documents maintained wrappers/import hooks, streaming token metrics when reported, Responses and broader image/audio/Decisions/attachment handling. The floor usage repair matches the documented reported-usage expectation; this PR does not claim parity with that broader API surface.
• Langfuse OpenAI integration documents client wrapping, streaming, Responses, usage, errors and flushing. These tests establish privacy, cleanup, accounting and mixed-client correctness on the scoped TraceAI paths, not blanket superiority or equivalent performance.

🤖 Generated with Claude Code