… OpenAI-compatible endpoint (#27508)

Squash-merged by litellm-agent from yimao's PR.

* fix(opentelemetry): JSON-serialize dict metadata fields for OTEL span attributes (#27451) (#27455)

Squash-merged by litellm-agent from Anai-Guo's PR.

* feat(dashscope): add embeddings and reranks(qwen3-rerank) support via OpenAI-compatible endpoint (#27508)

Squash-merged by litellm-agent from yimao's PR.

* fix(vertex_ai/gemini): raise BadRequestError when image_url or url fi… (#24550)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix(vertex_ai): raise error on mid-stream 429/error chunks instead of silently swallowing (#23711)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix: raise BadRequestError for file content blocks missing 'file' sub… (#24503)

Squash-merged by litellm-agent from krisxia0506's PR.

* Fix Gemini MIME detection for extensionless GCS URIs (#27278)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix(vertex_ai/partner_models): drop unused vertexai SDK gate from count_tokens (closes #28084) (#28107)

Squash-merged by litellm-agent from voidborne-d's PR.

* feat(chart): add support for autoscaling behavior in HPA (#27990)

Squash-merged by litellm-agent from FabrizioCafolla's PR.

* feat(proxy): add blocked flag to models for pause/resume from the UI (#27927)

Squash-merged by litellm-agent from Cyberfilo's PR.

* fix: pass socket timeouts to Redis cluster clients (#27920)

Squash-merged by litellm-agent from tomdee's PR.

* Fix/cache token (#28009)

Squash-merged by litellm-agent from escon1004's PR.

* fix(deepseek): forward reasoning_content in multi-turn thinking mode conversations (#28080)

Squash-merged by litellm-agent from Divyansh8321's PR.

* fix(guardrails): return HTTP 400 instead of 500 for blocked requests (#27617)

* fix: reset org and tag budgets (#27326)

* reset org budgets

* reset tag budgets

---------

Co-authored-by: Michael Riad Zaky <[email protected]>

* fix(ui): omit allowed_routes from key edit save when unchanged (#27553)

* fix(ui): omit allowed_routes from key edit save when unchanged

When a team admin opens Edit Settings on a key with key_type=AI APIs and
saves without changing anything, the UI re-sends the existing allowed_routes
value, which the backend's _check_allowed_routes_caller_permission gate
rejects for non-proxy-admins (LIT-2681).

Strip allowed_routes from the patch in handleSubmit when it deep-equals the
original keyData.allowed_routes. The backend treats absence as "leave alone,"
so no-op saves now succeed for non-admins. Admins explicitly editing the
field still send the new value.

* fix(ui): order-insensitive allowed_routes diff + cover null-original case

Address Greptile review:

- Switch the "is allowed_routes unchanged" check to a Set-based comparison so
  a server-side reorder of the array doesn't register as a user edit and
  re-trigger LIT-2681.
- Add two regression tests: (1) keyData.allowed_routes is null and the form
  is untouched — patch should strip the field; (2) server returned routes in
  a different order than the user originally entered — patch should still
  recognize the value as unchanged.

* chore(ui): strip ticket refs and tighten comments in key edit fix

- Remove internal-tracker references from in-code comments
- Tighten the WHY comment in handleSubmit to two lines
- Drop redundant test-block comments — test names already describe the case

* fix(ui): annotate Set<string> generic in allowed_routes diff to fix tsc

* fix(guardrails): return HTTP 400 instead of 500 for guardrail-blocked requests

GuardrailRaisedException and BlockedPiiEntityError both lacked a
status_code attribute.  When these exceptions reached the proxy
exception handler (getattr(e, 'status_code', 500)), the fallback
defaulted to HTTP 500 — making intentional guardrail blocks
indistinguishable from server errors and causing unnecessary client
retries.

Changes:
- Add status_code=400 (keyword-only) to GuardrailRaisedException
- Add status_code=400 (keyword-only) to BlockedPiiEntityError
- Update _is_guardrail_intervention() to recognize both exceptions
  so downstream loggers record 'guardrail_intervened' instead of
  'guardrail_failed_to_respond'
- Add 6 unit tests for default/custom status codes and getattr pattern
- Strengthen existing blocked-action test with status_code assertion

Fixes #24348

---------

Co-authored-by: Michael-RZ-Berri <[email protected]>
Co-authored-by: Michael Riad Zaky <[email protected]>
Co-authored-by: ryan-crabbe-berri <[email protected]>
Co-authored-by: Krrish Dholakia <[email protected]>

* fix(router/proxy): address Greptile P1+P2 review comments on PR #28161

- router: raise ServiceUnavailableError (503) instead of RouterRateLimitErrorBasic (429)
  when a specifically-addressed deployment is administratively blocked; 429 misleads
  retry-enabled clients into spinning forever against a paused model
- proxy_server: compute get_fully_blocked_model_names() once before both branches in
  model_list() instead of duplicating the call in each branch
- deepseek: upgrade silent debug log to warning when injecting placeholder
  reasoning_content so callers are clearly notified of degraded multi-turn quality
- tests: update two blocked-deployment assertions to expect ServiceUnavailableError

Co-authored-by: Cursor <[email protected]>

* fix: address bug detection findings (cache token order, mutable defaults)

Co-authored-by: Yassin Kortam <[email protected]>

* fix: address bugs in async pass-through, anthropic cache token detection, rerank tests

- async_get_available_deployment_for_pass_through: enforce blocked check on specific deployments
- cost_calculator: detect anthropic-style usage by attribute presence (not truthiness) to avoid mixing OpenAI cached_tokens into anthropic normalization when read=0
- dashscope rerank tests: pass request to httpx.Response constructions for consistency

Co-authored-by: Yassin Kortam <[email protected]>

* fix code qa

* fix(vertex_ai/gemini): strip MIME parameters from GCS contentType

GCS object metadata's contentType field can include parameters such as
'text/html; charset=utf-8'. Strip them in _apply_gemini_mime_type_aliases
so downstream get_file_extension_from_mime_type sees a bare MIME type.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(vertex_ai/gemini): clarify mime-type error message string concatenation

Co-authored-by: Yassin Kortam <[email protected]>

---------

Co-authored-by: Tai An <[email protected]>
Co-authored-by: Vincent <[email protected]>
Co-authored-by: Kris Xia <[email protected]>
Co-authored-by: d 🔹 <[email protected]>
Co-authored-by: Fabrizio Cafolla <[email protected]>
Co-authored-by: Filippo Menghi <[email protected]>
Co-authored-by: Tom Denham <[email protected]>
Co-authored-by: escon1004 <[email protected]>
Co-authored-by: Divyansh Singhal <[email protected]>
Co-authored-by: robin-fiddler <[email protected]>
Co-authored-by: Michael-RZ-Berri <[email protected]>
Co-authored-by: Michael Riad Zaky <[email protected]>
Co-authored-by: ryan-crabbe-berri <[email protected]>
Co-authored-by: Krrish Dholakia <[email protected]>
Co-authored-by: Cursor <[email protected]>
Co-authored-by: Yassin Kortam <[email protected]>

… model catalog) (#28223)

* fix(opentelemetry): JSON-serialize dict metadata fields for OTEL span attributes (#27451) (#27455)

Squash-merged by litellm-agent from Anai-Guo's PR.

* feat(dashscope): add embeddings and reranks(qwen3-rerank) support via OpenAI-compatible endpoint (#27508)

Squash-merged by litellm-agent from yimao's PR.

* fix(vertex_ai/gemini): raise BadRequestError when image_url or url fi… (#24550)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix(vertex_ai): raise error on mid-stream 429/error chunks instead of silently swallowing (#23711)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix: raise BadRequestError for file content blocks missing 'file' sub… (#24503)

Squash-merged by litellm-agent from krisxia0506's PR.

* Fix Gemini MIME detection for extensionless GCS URIs (#27278)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix(vertex_ai/partner_models): drop unused vertexai SDK gate from count_tokens (closes #28084) (#28107)

Squash-merged by litellm-agent from voidborne-d's PR.

* feat(chart): add support for autoscaling behavior in HPA (#27990)

Squash-merged by litellm-agent from FabrizioCafolla's PR.

* feat(proxy): add blocked flag to models for pause/resume from the UI (#27927)

Squash-merged by litellm-agent from Cyberfilo's PR.

* fix: pass socket timeouts to Redis cluster clients (#27920)

Squash-merged by litellm-agent from tomdee's PR.

* Fix/cache token (#28009)

Squash-merged by litellm-agent from escon1004's PR.

* fix(deepseek): forward reasoning_content in multi-turn thinking mode conversations (#28080)

Squash-merged by litellm-agent from Divyansh8321's PR.

* fix(guardrails): return HTTP 400 instead of 500 for blocked requests (#27617)

* fix: reset org and tag budgets (#27326)

* reset org budgets

* reset tag budgets

---------

Co-authored-by: Michael Riad Zaky <[email protected]>

* fix(ui): omit allowed_routes from key edit save when unchanged (#27553)

* fix(ui): omit allowed_routes from key edit save when unchanged

When a team admin opens Edit Settings on a key with key_type=AI APIs and
saves without changing anything, the UI re-sends the existing allowed_routes
value, which the backend's _check_allowed_routes_caller_permission gate
rejects for non-proxy-admins (LIT-2681).

Strip allowed_routes from the patch in handleSubmit when it deep-equals the
original keyData.allowed_routes. The backend treats absence as "leave alone,"
so no-op saves now succeed for non-admins. Admins explicitly editing the
field still send the new value.

* fix(ui): order-insensitive allowed_routes diff + cover null-original case

Address Greptile review:

- Switch the "is allowed_routes unchanged" check to a Set-based comparison so
  a server-side reorder of the array doesn't register as a user edit and
  re-trigger LIT-2681.
- Add two regression tests: (1) keyData.allowed_routes is null and the form
  is untouched — patch should strip the field; (2) server returned routes in
  a different order than the user originally entered — patch should still
  recognize the value as unchanged.

* chore(ui): strip ticket refs and tighten comments in key edit fix

- Remove internal-tracker references from in-code comments
- Tighten the WHY comment in handleSubmit to two lines
- Drop redundant test-block comments — test names already describe the case

* fix(ui): annotate Set<string> generic in allowed_routes diff to fix tsc

* fix(guardrails): return HTTP 400 instead of 500 for guardrail-blocked requests

GuardrailRaisedException and BlockedPiiEntityError both lacked a
status_code attribute.  When these exceptions reached the proxy
exception handler (getattr(e, 'status_code', 500)), the fallback
defaulted to HTTP 500 — making intentional guardrail blocks
indistinguishable from server errors and causing unnecessary client
retries.

Changes:
- Add status_code=400 (keyword-only) to GuardrailRaisedException
- Add status_code=400 (keyword-only) to BlockedPiiEntityError
- Update _is_guardrail_intervention() to recognize both exceptions
  so downstream loggers record 'guardrail_intervened' instead of
  'guardrail_failed_to_respond'
- Add 6 unit tests for default/custom status codes and getattr pattern
- Strengthen existing blocked-action test with status_code assertion

Fixes #24348

---------

Co-authored-by: Michael-RZ-Berri <[email protected]>
Co-authored-by: Michael Riad Zaky <[email protected]>
Co-authored-by: ryan-crabbe-berri <[email protected]>
Co-authored-by: Krrish Dholakia <[email protected]>

* fix(router/proxy): address Greptile P1+P2 review comments on PR #28161

- router: raise ServiceUnavailableError (503) instead of RouterRateLimitErrorBasic (429)
  when a specifically-addressed deployment is administratively blocked; 429 misleads
  retry-enabled clients into spinning forever against a paused model
- proxy_server: compute get_fully_blocked_model_names() once before both branches in
  model_list() instead of duplicating the call in each branch
- deepseek: upgrade silent debug log to warning when injecting placeholder
  reasoning_content so callers are clearly notified of degraded multi-turn quality
- tests: update two blocked-deployment assertions to expect ServiceUnavailableError

Co-authored-by: Cursor <[email protected]>

* fix: address bug detection findings (cache token order, mutable defaults)

Co-authored-by: Yassin Kortam <[email protected]>

* fix: address bugs in async pass-through, anthropic cache token detection, rerank tests

- async_get_available_deployment_for_pass_through: enforce blocked check on specific deployments
- cost_calculator: detect anthropic-style usage by attribute presence (not truthiness) to avoid mixing OpenAI cached_tokens into anthropic normalization when read=0
- dashscope rerank tests: pass request to httpx.Response constructions for consistency

Co-authored-by: Yassin Kortam <[email protected]>

* fix code qa

* fix(vertex_ai/gemini): strip MIME parameters from GCS contentType

GCS object metadata's contentType field can include parameters such as
'text/html; charset=utf-8'. Strip them in _apply_gemini_mime_type_aliases
so downstream get_file_extension_from_mime_type sees a bare MIME type.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(vertex_ai/gemini): clarify mime-type error message string concatenation

Co-authored-by: Yassin Kortam <[email protected]>

* feat(oci): add embeddings, fix streaming/reasoning, expand model catalog

- Add OCIEmbedConfig with full Cohere embed support (7 models, batch up to 96)
- Fix sync streaming: split SSE events on \n\n before JSON parsing
- Fix reasoning models (Gemini 2.5, xAI Grok): make completionTokens and message
  optional in OCIResponseChoice to handle max_tokens exhausted on reasoning
- Fix compartment_id resolution in chat transform to use resolve_oci_credentials
- Fix tool call id: make OCIToolCall.id optional, generate UUID fallback for
  providers (Google via OCI) that omit it
- Add OCI_KEY env var support for inline PEM keys
- Fix datetime.utcnow() deprecation in request signing
- Expand model catalog: 29 OCI models including Llama 4, Gemini 2.5, xAI Grok,
  Cohere Command A, and all Cohere embed variants
- Add 37 live integration tests: sync/async completions for Meta/Google/xAI/Cohere,
  sync/async embeddings, tool use across all vendors, streaming, env var auth
- Add 23 embed unit tests covering all transform and validation paths

* fix(oci): remove dead OCI elif branch in utils.py, align async split_chunks with sync version

* test(oci): add unit tests for split_chunks fix and no-duplicate-OCI-branch guard

* fix(oci): address remaining bugs from issue #25082 — streaming signed body, Cohere stop sequences, hardcoded defaults

- Bug 1: sync and async streaming paths now use signed_json_body when provided
  instead of re-serializing data with json.dumps() — the OCI RSA-SHA256 signature
  covers the exact request body bytes, so re-serializing produces an invalid sig
- Bug 3: Cohere stop sequences now map to 'stopSequences' (was incorrectly 'stop')
- Bug 4: removed hardcoded Cohere defaults (maxTokens=600, temperature=1, topK=0,
  topP=0.75, frequencyPenalty=0) that silently overrode user intent on every call
- Added 6 unit tests covering all three fixes

* fix(oci): comprehensive code quality pass — bugs, tests, schema accuracy

- Fix Cohere tool call IDs (was always call_0; now UUID per call)
- Fix TOOL_CALL finish reason mapping in both sync and streaming paths
- Fix Cohere stop parameter mapping (stop → stopSequences)
- Remove hardcoded Cohere defaults (maxTokens/topK/topP/frequencyPenalty)
- Fix content[0] safety guard against empty content arrays
- Fix streaming signed body used consistently (not re-serialized)
- Raise OCIError (not bare Exception/ValueError) throughout
- Centralize OCI_API_VERSION constant; import uuid at module level
- Fix embed get_complete_url to strip trailing slashes from api_base
- Fix OCIEmbedResponse schema: add inputTextTokenCounts (actual OCI field)
- Fix embed usage computed from inputTextTokenCounts (sum of per-input counts)
- Fix Cohere toolCallId included in tool result messages
- Add OCIToolCall.id as Optional (absent in Google/xAI streaming chunks)
- Update tests to reflect correct behavior (no hardcoded defaults, UUID ids,
  deferred credential validation, OCIError vs ValueError, real response schema)

* test(oci): move integration tests to tests/llm_translation/

Addresses greptile P1: tests/test_litellm/ is for mock-only unit tests
(make test-unit target). Real-network OCI tests now live in the correct
location alongside other provider integration tests.

* fix(oci): align types and transformation with official OCI SDK

- Remove OCIVendors.GEMINI — apiFormat="GEMINI" is invalid; all non-Cohere
  models use apiFormat="GENERIC"
- Add toolChoice, logitBias, logProbs to OCIChatRequestPayload so params
  present in the mapping are no longer silently dropped by Pydantic
- Exclude n→numGenerations from Cohere param map (not a Cohere API field)
- Fix CohereToolResult: change callId/result to call/outputs matching
  the OCI SDK's CohereToolResult structure
- Fix CohereToolMessage: replace non-existent toolCallId with toolResults
  list; update adapt_messages_to_cohere_standard to build proper tool-result
  history entries by resolving tool call name+params from preceding assistant
  messages
- Map generic-model stream finish reasons to OpenAI convention
  (COMPLETE→stop, MAX_TOKENS→length, TOOL_CALLS→tool_calls), consistent
  with the existing Cohere streaming path
- Add optional id field to OCIEmbedResponse so valid API responses
  carrying an id are not rejected by the Pydantic model

* fix(oci): use 'output' key in Cohere tool result outputs (matches reference impl)

* fix(oci): port schema/type utilities from langchain-oracle reference impl

- Add resolve_oci_schema_refs: inline $ref/$defs — OCI rejects JSON Schema refs
- Add resolve_oci_schema_anyof: flatten Optional[T] anyOf (Pydantic v2 emits these)
- Add sanitize_oci_schema: strip title, normalise null types, ensure array items
- Add OCI_JSON_TO_PYTHON_TYPES: Cohere expects Python type names (str/int/float),
  not JSON Schema names (string/integer/number)
- Add enrich_cohere_param_description: embed enum/format/range/pattern constraints
  into description since CohereParameterDefinition has no dedicated fields
- Apply all of the above in adapt_tool_definitions_to_cohere_standard and
  adapt_tool_definition_to_oci_standard
- Fix toolChoice conversion: map OpenAI string ('auto','none','required') to OCI
  dict form ({"type":"AUTO"} etc.) — the API rejects plain strings
- Update unit test expectations to match correct Python type names and enriched
  descriptions

* refactor(oci): split transformation.py into cohere.py and generic.py

transformation.py was 1 243 lines doing too many jobs. Split along the
same boundaries as the langchain-oracle reference (providers/cohere.py,
providers/generic.py):

  chat/cohere.py   — Cohere message/tool building, response + stream parsing
  chat/generic.py  — Generic message/tool building, response + stream parsing
  transformation.py — thin OCIChatConfig orchestrator + OCIStreamWrapper

Public symbols (OCIChatConfig, OCIStreamWrapper, adapt_messages_to_*,
OCIRequestWrapper, version, …) remain importable from transformation.py
for backward compatibility. OCIStreamWrapper gains delegating shims for
_handle_cohere_stream_chunk and _handle_generic_stream_chunk so existing
test call sites keep working unchanged.

transformation.py: 1 243 → 620 lines

* refactor(oci): principal-level code quality pass

- Remove _extract_text_content duplication — single definition in cohere.py,
  imported where needed; instance method on OCIChatConfig eliminated
- Move cryptography imports to module level with _CRYPTOGRAPHY_AVAILABLE flag
  and _require_cryptography() guard; no more re-import on every signing call
- Move litellm version import to module level via litellm._version; remove
  inline import inside validate_oci_environment
- sign_with_manual_credentials now returns Tuple[dict, bytes] matching
  sign_with_oci_signer — asymmetry eliminated, Optional[bytes] guards removed
  throughout stream wrappers (signed_json_body: bytes = b"")
- Rename _openai_to_oci_cohere_param_map → openai_to_oci_cohere_param_map
  for consistency with openai_to_oci_generic_param_map
- Remove double-key bug in map_openai_params where responseFormat was stored
  under both OCI and OpenAI key names simultaneously
- Remove delegating shims (adapt_messages_to_cohere_standard,
  adapt_tool_definitions_to_cohere_standard, _handle_generic_stream_chunk)
  from OCIChatConfig/OCIStreamWrapper; tests now import directly from
  cohere.py and generic.py where symbols live
- Trim __all__ to 7 genuine public symbols; remove the 13-symbol list that
  existed only to support test imports
- Collapse per-model integration test classes into pytest.mark.parametrize;
  CHAT_MODELS list is the single source of truth for model-specific config
- Black + Ruff clean across all OCI files

* fix(oci): address PR review findings

- types/llms/oci.py: add "TOOL_CALL" to CohereChatResponse.finishReason
  Literal so Pydantic does not raise ValidationError on non-streaming
  Cohere tool-use calls (Greptile P1)
- test_oci_cohere_tool_calls.py: add test covering TOOL_CALL finish reason
- model_prices_and_context_window.json: remove 6 duplicate oci/cohere.embed-*
  keys that were silently overridden by the more complete entries already
  present in the file (Greptile P1)
- common_utils.py: move OCI_API_VERSION here from chat/transformation.py
  so embed/transformation.py does not need to import chat/transformation;
  change Protocol stub body from ... to pass (CodeQL "statement no effect");
  add comment to sha256_base64 clarifying it implements OCI HTTP signing
  spec, not password hashing (CodeQL false positive)
- chat/transformation.py: import CustomStreamWrapper from
  litellm_core_utils.streaming_handler instead of litellm.utils to reduce
  import cycle depth (CodeQL cyclic import)
- chat/cohere.py, chat/generic.py: import Usage and
  ChatCompletionMessageToolCall from litellm.types.utils instead of
  litellm.utils for the same reason
- embed/transformation.py: import OCI_API_VERSION from common_utils
  instead of chat/transformation (removes the embed→chat import edge)

* test(oci): add unit tests to improve patch coverage

- test_oci_common_utils.py (new): covers sha256_base64, build_signature_string,
  OCIRequestWrapper.path_url, resolve_oci_credentials, get_oci_base_url,
  validate_oci_environment, sign_with_oci_signer error paths, sign_oci_request
  routing, load_private_key_from_file error paths, resolve_oci_schema_refs
  (including circular ref and external $ref), resolve_oci_schema_anyof,
  sanitize_oci_schema (all branches), enrich_cohere_param_description
- test_oci_generic_chat.py (new): covers content-message error paths (non-dict
  item, unsupported type, non-string text, invalid image_url), tool-call
  validation error paths, adapt_messages_to_generic_oci_standard error paths,
  handle_generic_response (None message, text content, tool calls),
  handle_generic_stream_chunk (finish reasons, streaming tool calls),
  OCIStreamWrapper non-string chunk error
- test_oci_chat_transformation.py: add error paths for validate_environment
  (empty messages), transform_request (missing compartment_id, Cohere without
  user messages), transform_response (error key), map_openai_params
  (unsupported param with and without drop_params), tool_choice string mapping
- test_oci_cohere_tool_calls.py: add edge cases for stream chunk finish
  reasons (TOOL_CALL, MAX_TOKENS, unknown), _extract_text_content with
  non-dict list items and non-string input,
  adapt_messages_to_cohere_standard with malformed JSON tool arguments

* fix(oci): rename supports_streaming to supports_native_streaming in model prices

The JSON schema for model_prices_and_context_window.json uses
`supports_native_streaming` (not `supports_streaming`) and has
`additionalProperties: false`. Rename the field across all OCI
entries to pass the schema validation test.

* test(oci): add 67 tests targeting uncovered happy paths for coverage

Boost patch coverage on the four lowest-coverage OCI files:
- common_utils.py: sign_with_manual_credentials (oci_key / oci_key_file
  paths), sign_oci_request routing, _require_cryptography
- generic.py: adapt_messages_to_generic_oci_standard (all roles),
  adapt_tool_definition_to_oci_standard, adapt_tools_to_openai_standard,
  handle_generic_stream_chunk text/finish-reason paths
- cohere.py: _extract_text_content, adapt_messages_to_cohere_standard
  (all roles including tool results), handle_cohere_response /
  handle_cohere_stream_chunk all finish-reason branches
- transformation.py: get_vendor_from_model, OCIChatConfig._get_optional_params
  (toolChoice string→dict, responseFormat, tools for both vendors),
  transform_request for GENERIC model, get_sync/async_custom_stream_wrapper
  with mocked HTTP, OCIStreamWrapper.chunk_creator happy paths

* fix(oci): suppress CodeQL false positive on sha256_base64 (OCI HTTP signing, not password hashing)

* fix(oci): remove 6 duplicate model price entries and reconcile conflicting values

Six OCI chat model keys appeared twice in model_prices_and_context_window.json
with conflicting pricing/context data (JSON parsers silently discard the first).
Remove the first-occurrence entries and update the surviving entries:
- meta.llama-4-maverick / llama-4-scout: keep updated entries (free preview
  pricing, larger context windows, vision support)
- meta.llama-3.1-70b: keep original pricing, restore supports_native_streaming
- google.gemini-2.5-{flash,pro,flash-lite}: keep OCI pricing page values,
  restore supports_native_streaming

* fix(oci): route GPT-5 family to maxCompletionTokens

GPT-5 / GPT-5-mini / GPT-5-nano / GPT-5.5 on OCI reject "maxTokens"
with HTTP 400:

  Invalid 'maxTokens': Unsupported parameter: 'maxTokens' is not
  supported with this model. Use 'maxCompletionTokens' instead.

(Same convention as OpenAI's reasoning-API contract.)

Add a model-aware rename in OCIChatConfig._get_optional_params so the
request payload uses maxCompletionTokens when the model id starts with
openai.gpt-5. Regular Llama / Cohere / Gemini / GPT-4.x continue to use
maxTokens unchanged.

Also widen OCIChatRequestPayload to carry the new optional field so it
survives Pydantic serialization.

Verified live against OCI us-chicago-1:
- openai.gpt-5, gpt-5-mini, gpt-5-nano, gpt-5.5 all return 200
- Full feature sweep on gpt-5.5 (basic, system, multi-turn, streaming,
  tools, usage) all green
- meta.llama-3.3-70b-instruct still uses maxTokens (no regression)

4 new unit tests cover the helper, the routing in both pre- and
post-translation states, and Pydantic serialization.

* ci(oci): fix CI failures — black formatting + recursive_detector ignore

- Run black on litellm/llms/oci/common_utils.py + 3 OCI test files
  that drifted out of black-compliance during the rebase.
- Add the three bounded recursive functions in oci/common_utils.py
  (`_resolve`, `resolve_oci_schema_anyof`, `sanitize_oci_schema`) to
  the recursive_detector IGNORE_FUNCTIONS list. All three are bounded:
  `_resolve` uses a `resolving_stack` cycle guard; the other two are
  bounded by JSON-schema tree depth (no cycles in well-formed input),
  matching the pattern of the existing OCI/Vertex schema walkers
  already on the list.

* fix(oci): silence MyPy errors in cohere.py — typed-dict access

Two errors flagged by `lint` CI:

  llms/oci/chat/cohere.py:73:  "object" has no attribute "__iter__"
  llms/oci/chat/cohere.py:119: No overload variant of "get" of "dict"
                               matches argument types "object", "CohereToolCall"

Both stem from `msg.get("tool_calls")` / `msg.get("tool_call_id")`
returning `object` per the AllMessageValues TypedDict union. Bind to
`Any` locally for the iteration and coerce the lookup key with `str()`,
removing the now-unused `# type: ignore` on those lines.

No behaviour change — pure type-narrowing for the type checker.

* fix(oci): silence CodeQL py/weak-sensitive-data-hashing on sha256_base64

CodeQL's taint analysis traces request bodies back to environment-loaded
secrets and flags `hashlib.sha256(body).digest()` as
`py/weak-sensitive-data-hashing` — even though SHA-256 is the algorithm
mandated by the OCI HTTP request signing spec for the
`x-content-sha256` header (not a password/secret hash).

The previous suppression used legacy `# lgtm[...]` syntax which the
modern CodeQL action ignores. Switch to Python's standard
`hashlib.sha256(..., usedforsecurity=False)` (Python 3.9+) which CodeQL
honours as a non-security declaration. Behaviour unchanged.

* feat(oci): add reasoning_effort passthrough — only true missing primitive

OCI's GenericChatRequest exposes a reasoningEffort field
(NONE/MINIMAL/LOW/MEDIUM/HIGH) that's the single biggest cost knob for
reasoning-capable models on the service:

  - GPT-5 family
  - Gemini 2.5
  - Grok reasoning variants (3-mini, 4-fast, 4.20)
  - Cohere Command-A-Reasoning

Setting reasoning_effort=LOW typically cuts reasoning-token spend 5-10×
vs the default. Without exposing this, litellm users had no way to tune
cost-vs-quality on these models.

The other GenericChatRequest fields (verbosity, parallel_tool_calls,
logit_bias, n, metadata, web_search_options, prediction) are not
exposed because they are not missing primitives — they either duplicate
prompt-engineering, framework-level controls, or are too niche to
justify the maintenance surface. We only ship what users genuinely
can't accomplish another way.

Excluded from the Cohere v1 param map: CohereChatRequest has no
reasoningEffort field, and Cohere reasoning models
(cohere.command-a-reasoning) use COHEREV2 which is a separate request
type not covered by this PR.

Verified live: GPT-5.5 + reasoning_effort="HIGH" sends
{"reasoningEffort": "HIGH"} on the wire and OCI accepts the request.

* feat(oci): reasoning_effort + reasoning_tokens for OCI GenAI

Three small additions for OCI reasoning models, requested by users
testing the PR in production fork builds:

1. **reasoning_effort param mapping (GENERIC vendors).** OCI expects
   uppercase levels ("LOW"/"MEDIUM"/"HIGH"/"NONE") on `reasoningEffort`,
   but OpenAI-compatible clients send lowercase. Mapped + uppercased in
   `_get_optional_params`. Marked unsupported on Cohere V1/V2 since OCI
   Cohere has no reasoning models (avoids Pydantic validation failure
   on CohereChatRequest).

2. **"disable" → "NONE" mapping.** OpenAI uses "disable" to turn off
   reasoning; OCI uses "NONE". Without this, callers get a 400.

3. **reasoning_tokens propagated to Usage.** OCI returns
   `completionTokensDetails.reasoningTokens` but it wasn't being passed
   to LiteLLM's Usage object. Now flows through to
   `Usage.completion_tokens_details.reasoning_tokens` so callers can
   track reasoning token consumption for cost/observability.

Tests: 7 new unit tests in TestOCIReasoningEffort covering upper/lower
case, "disable"→"NONE", Cohere drop/raise paths, and reasoning_tokens
extraction (with and without completionTokensDetails). 5 new live
integration tests against xai.grok-3-mini in us-chicago-1 verifying the
full request/response loop end-to-end. Existing
test_transform_response_simple_text assertion that
completion_tokens_details was None has been updated to assert
reasoning_tokens flows through.

Verified live on xai.grok-3-mini: reasoning_effort=low → OCI accepts
"LOW", returns reasoningTokens=316 in usage. reasoning_effort=disable
→ OCI accepts "NONE". Full suite: 370/370 unit + 51/51 integration.

* fix(codeql): re-scope py/weak-sensitive-data-hashing exclusion to OCI signing file

CodeQL's taint analysis re-fires the `py/weak-sensitive-data-hashing`
alert at `litellm/llms/oci/common_utils.py:103` whenever upstream code
paths into the OCI signing module change (touching `transformation.py`
opens new flow paths that CodeQL re-evaluates from scratch). The
`hashlib.sha256(..., usedforsecurity=False)` declaration silences the
direct-call form of the query but not the taint-flow form.

SHA-256 here is mandated by the OCI HTTP signing specification for the
x-content-sha256 content-integrity header — not for password storage:
https://docs.oracle.com/en-us/iaas/Content/API/Concepts/signingrequests.htm

CodeQL has no per-query path filter and GitHub Code Scanning ignores
inline lgtm/codeql comments, so path-ignoring this single ~560-line
signing utility file is the narrowest available suppression. All other
files retain full coverage of py/weak-sensitive-data-hashing — including
litellm/proxy/utils.py where the rule legitimately applies.

This restores the NEUTRAL CodeQL state the PR had on prior commits
(see `2111c98af7` for the same approach on the previous branch
evolution that the cherry-pick was rebased onto a different baseline).

* fix(oci): drop duplicate text on Cohere streaming terminal chunk

OCI Cohere's terminal SSE event re-sends the full assembled response in
`text` alongside a populated `chatHistory`. Emitting that text as another
delta concatenates the entire response onto the already-streamed output
(e.g. "How can I help?How can I help?").

Use `chatHistory is not None` as the discriminator for the consolidated
terminal event — `finishReason` is a weaker signal that could in principle
appear on a non-consolidated chunk. The two coincide today; this preserves
correctness if OCI ever ships finishReason on an incremental chunk.

Adds a live-OCI integration regression test that compares streamed vs
non-streamed length and asserts the response prefix appears only once.
Verified to fail under the previous code with the exact reported
reproduction: 'Hello! How can I help you today?Hello! How can I help you today?'.

Reported by @gotsysdba on PR #25177.

* fix(oci): buffer SSE stream across HTTP read boundaries

The old split_chunks helper split each individual HTTP read on "\n\n",
which assumed SSE event boundaries always aligned with read boundaries.
In practice the OCI streaming endpoint delivers events that may:

  - straddle two reads (chunk_creator gets a truncated JSON and crashes)
  - arrive separated by a single "\n" instead of "\n\n"
  - share a read with multiple complete events

Replace the inline split with module-level helpers _iter_sse_events
(sync) / _aiter_sse_events (async) that maintain a buffer across reads,
split on any newline, and yield only complete "data:" lines.

Add 25 regression tests covering event-split-across-reads, tiny-chunk
reads, single-newline separators, keepalive/comment lines, trailing
partial events flushed at EOF, "\r\n" line endings, and an end-to-end
smoke test that feeds an awkwardly-chopped payload through the splitter
into OCIStreamWrapper.chunk_creator.

Reported by John Lathouwers.

* test(oci): repoint TestOCIKeyNormalization to sign_with_manual_credentials

The signing helper moved from OCIChatConfig._sign_with_manual_credentials
to a module-level sign_with_manual_credentials in common_utils.py. Four
tests in TestOCIKeyNormalization still called the old method:

  - 2 failed outright with AttributeError
  - 2 passed by accident because they used pytest.raises(Exception),
    which happily caught the AttributeError instead of exercising the
    intended OCIError path

Repoint all four to the new module-level function so they exercise the
actual oci_key type-validation branch.

* fix(oci): validate oci_region before URL interpolation to prevent SSRF

Anchor oci_region to ^[a-z][a-z0-9-]{0,30}[a-z0-9]$ inside get_oci_base_url
so user-supplied regions that would redirect the signed request to an
attacker-controlled host (e.g. 'evil.com/#') fail with HTTP 400 before
the URL or signature is built. Empty string still falls back to the
us-ashburn-1 default, so existing callers are unaffected.

* test(audio): skip when gpt-4o-audio-preview is unavailable upstream

OpenAI retired `gpt-4o-audio-preview` (404 model_not_found in CI as of
2026-05-19), and the existing try/except in these tests only re-raised
on 'openai-internal' errors. Other exceptions were silently swallowed,
so the next line ran with an unbound `response`/`completion` and
failed with an unrelated UnboundLocalError that masked the real cause.

Extend the skip condition to also cover model_not_found / 'does not exist'
so the suite reports the upstream outage cleanly, matching the pattern
used in ce87c41 for the realtime and nvidia_nim rerank tests.
Re-raise unknown exceptions instead of falling through.

* fix(oci/router): catalog-driven maxCompletionTokens; generic blocked-deployment message

- Drive OCI maxCompletionTokens via supports_reasoning from the model
  catalog instead of a hardcoded openai.gpt-5 prefix. Add OCI GPT-5 family
  entries (gpt-5, gpt-5-mini, gpt-5-nano) with supports_reasoning: true.
  Gate the override to non-Cohere vendor so Cohere reasoning models keep
  maxTokens (Cohere endpoint does not accept maxCompletionTokens).
- Replace proxy-specific 'Contact your proxy admin' phrasing in the four
  Router blocked-deployment ServiceUnavailableError messages with neutral
  SDK-appropriate text.

* fix(oci/cohere): guard handle_cohere_response against missing usage

* fix(oci): address bug review findings in chat transformation

- Cohere param map: keep tool_choice/n as False (not omitted) so unsupported
  params are dropped or rejected rather than silently passed through.
- get_complete_url: when an explicit api_base/litellm.api_base is provided,
  use it as-is instead of unconditionally appending /20231130/actions/chat
  (mirrors the embed config behavior).
- Cohere stream: require both chatHistory and finishReason to be present to
  identify a terminal consolidation chunk, avoiding silent text suppression
  if chatHistory ever appears on a non-terminal chunk.
- Generic usage: use 'is not None' for reasoningTokens so a legitimate value
  of 0 is preserved instead of being treated as absent.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/cohere): emit tool calls in streaming and null content when text empty

handle_cohere_response now sets message.content to None when the Cohere
response text is empty, matching the OpenAI convention for tool-call-only
responses.

handle_cohere_stream_chunk now extracts toolCalls — both directly from
the chunk and from the terminal chunk's chatHistory CHATBOT message —
and emits them in the delta. Previously, CohereStreamChunk lacked a
toolCalls field, so any tool calls in the stream were silently dropped.

* fix(oci): preserve tool results, embed URL path, and generic finish reason

- Use SerializeAsAny on CohereChatRequest.chatHistory so subclass-specific
  fields like CohereToolMessage.toolResults are not dropped during Pydantic
  v2 serialization.
- Make OCIEmbedConfig.get_complete_url append the /20231130/actions/embedText
  action path consistently with chat, so setting litellm.api_base to the
  region inference base URL no longer posts to the bare hostname.
- Map OCI finishReason (COMPLETE / MAX_TOKENS / TOOL_CALLS) to OpenAI
  finish_reason values in handle_generic_response, mirroring the streaming
  handler and the Cohere non-streaming handler.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/generic): silence mypy assignment error on dynamic finish_reason

* fix(oci/embed): always set usage on embedding response

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/chat): append /20231130/actions/chat to explicit api_base

Restore the embed-style behavior so OCIChatConfig.get_complete_url always
appends the OCI GenAI chat path. Routing through get_oci_base_url ensures the
optional explicit api_base has its trailing slash stripped before the suffix is
joined, matching the embed config and the test_respects_explicit_api_base
expectation.

* fix(oci/cohere): mark logprobs/logit_bias unsupported and normalize unknown stream finish reasons

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/cohere): preserve trailing tool result in chatHistory

When the last message in the OpenAI-format input is a tool result (the
standard agentic continuation pattern), the prior messages[:-1] slice
silently dropped that tool result from chatHistory and the model never
saw it. Excluding the last user message by index instead keeps tool
results that trail the last user turn intact.

* fix(main): remove dead OCI embedding elif block

The earlier elif at line 5119 already routes OCI embeddings through the
base HTTP handler with the headers None-guard, so the later identical
block was unreachable dead code.

* test(oci): move integration tests out of llm_translation mock-only folder

Greptile flags tests/llm_translation/ as mock-only via a project-specific
rule; relocate the live-network OCI integration suite to tests/integration/
and adjust the in-file sys.path / run instructions accordingly.

* fix(oci/cohere): suppress tool calls on stream terminal consolidation chunk

The terminal SSE event re-sends the full assembled response in both
`text` and `chatHistory`. The existing logic already suppresses
`text` to avoid double-emit, but tool calls extracted from the
terminal chunk (via `typed_chunk.toolCalls` or the `chatHistory`
CHATBOT fallback) would still be re-emitted with fresh uuid4 IDs.
If OCI Cohere ever streams tool calls progressively in intermediate
chunks (now possible since CohereStreamChunk has a toolCalls field),
this would cause downstream agentic frameworks to execute each tool
call twice.

Suppress tool calls on the terminal consolidation chunk for the same
reason `text` is suppressed.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci,httpx): normalize finish_reason, preserve response_format, fix sync embed JSON content-type

- cohere.py / generic.py: normalize unknown OCI finishReason values (ERROR,
  ERROR_TOXIC, CONTENT_FILTERED, USER_CANCEL, ...) to 'stop' in non-streaming
  and streaming generic handlers, matching the streaming Cohere handler so
  downstream consumers switching on finish_reason aren't broken by raw OCI
  values.
- transformation.py: restore the dual-key alias so optional_params still
  carries the original 'response_format' key alongside the OCI-mapped
  'responseFormat'. Downstream litellm framework code (json_mode detection,
  logging) inspects 'response_format' after map_openai_params runs.
- llm_http_handler.py: make the sync embedding path mirror the async path —
  when sign_request returns no signed_body, send via json=data (which sets
  Content-Type: application/json) instead of data=json.dumps(data) which
  doesn't. Removes a sync/async behavioural asymmetry for non-OCI providers
  that adopt the sign_request pattern.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): clean up OCIChatConfig init, normalize generic stream finish reasons, correct embed sign_request return type

- Replace fragile setattr(self.__class__, ...) pattern in OCIChatConfig.__init__ with a @property for has_custom_stream_wrapper, matching the pattern used by other providers.
- Normalize unknown OCI finish reasons (e.g. ERROR, ERROR_TOXIC, USER_CANCEL) to 'stop' in handle_generic_stream_chunk, matching the existing Cohere stream handler behaviour.
- Tighten OCIEmbedConfig.sign_request return type from Tuple[dict, Optional[bytes]] to Tuple[dict, bytes] — sign_oci_request never returns None for the body, and this matches OCIChatConfig.sign_request.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): strip trailing action path in get_oci_base_url to avoid URL doubling

A fully-formed OCI endpoint URL (e.g. https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/20231130/actions/chat) passed via api_base previously had the action path appended a second time by get_complete_url in both chat and embed configs, yielding a 404. get_oci_base_url now strips a trailing /20231130/actions/<name> so callers can always append the action path safely.

* fix(httpx): preserve sync embed data= kwarg to avoid breaking mock-based tests

The earlier sync_httpx_client.post() call passed data=json.dumps(data),
which downstream embedding tests assert on (e.g. tests for hosted_vllm,
jina_ai, watsonx). Switching to json=data changed the kwarg name and broke
those tests. The OCI signed_body path keeps using data=signed_body and is
unaffected.

* fix(oci): stable tool-call ids across stream chunks; lenient Cohere finishReason

- Replace random uuid4 per chunk with a deterministic content-derived
  digest for synthetic tool-call ids in both Cohere and Generic OCI
  handlers. Previously, when OCI omitted 'id' (always for Cohere, often
  for Generic streaming deltas), every chunk for the same logical tool
  call received a new uuid, causing downstream stream-mergers (which key
  off id) to treat each fragment as a distinct call.

- Relax CohereChatResponse.finishReason from a strict Literal[...] to
  Optional[str], matching CohereStreamChunk.finishReason. The
  handle_cohere_response 'elif oci_finish_reason is not None' fallback
  was previously unreachable because Pydantic raised ValidationError on
  any unknown value before the fallback executed. Now non-streaming
  responses degrade unknown reasons to 'stop' just like the streaming
  path.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/embed): validate OCI credentials in validate_environment

Mirror OCIChatConfig.validate_environment so embedding requests fail
fast with a clear error when oci_user/oci_fingerprint/oci_tenancy/
oci_compartment_id or an oci_key/oci_key_file is missing, instead of
deferring the failure until sign_request.

Co-authored-by: Yassin Kortam <[email protected]>

* test(oci/embed): expect OCIError from validate_environment when credentials are missing

OCIEmbedConfig.validate_environment now raises eagerly (mirroring OCIChatConfig)
when oci_user/oci_fingerprint/oci_tenancy/oci_compartment_id or oci_key/oci_key_file
is missing. Update the test to match.

* fix(oci): polish stream chunk handling and signed body default

- cohere stream terminal consolidation now emits content=None instead of ""
- drop redundant index truthiness check (None is already replaced with 0)
- accept both "TOOL_CALL" and "TOOL_CALLS" finish reasons in cohere
- signed_json_body defaults to None and uses explicit None check, so an
  explicitly empty bytes body wouldn't be silently re-serialized

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/chat): catch pydantic ValidationError when parsing OCI responses

Pydantic v2 raises ValidationError (not TypeError) when field validation
fails, so malformed OCI completion responses or stream chunks would
propagate unhandled out of handle_generic_response,
handle_generic_stream_chunk, and handle_cohere_stream_chunk. Widen the
except clauses to also catch ValidationError so callers get a clean
OCIError.

* fix(oci/catalog): real prices for Llama 4, drop zero-cost OCI OpenAI entries

Zero-cost catalog entries (input_cost_per_token=0, output_cost_per_token=0)
make proxy spend tracking silently report $0 for these paid OCI models, so
any caller can drive them without decrementing a budget.

For Llama 4 Maverick and Scout, OCI charges the same character-based rate
as Llama 3.3 70B ($0.0018 per 10,000 characters), so use the same per-token
price as the existing oci/meta.llama-3.3-70b-instruct entry (7.2e-07 in/out).

For oci/openai.gpt-5, gpt-5-mini, gpt-5-nano, gpt-oss-120b, and gpt-oss-20b,
no public per-token pricing is available; drop the entries so operators must
register them with explicit custom pricing. The existing GPT-5 reasoning test
fixture already injects synthetic entries when the catalog omits them, so the
chat transformation's supports_reasoning lookup keeps working in tests.

* fix(oci/chat): wrap CohereChatResult construction in try/except

Match the handle_generic_response pattern: surface OCIError with the
upstream status code instead of letting a raw pydantic.ValidationError
propagate when the Cohere response payload is malformed.

* fix(oci): harden Cohere stream/finish-reason and dedupe maxTokens param mapping

- Cohere stream: track per-stream tool-call emission and only suppress the
  terminal consolidation chunk's tool calls once they've been seen earlier.
  Prevents silent drop if tool calls are delivered exclusively on the
  terminal chunk.
- Cohere stream: emit content=None (not "") on non-terminal text-free
  chunks (e.g. tool-call-only / keep-alive) so downstream consumers that
  distinguish missing vs explicitly-empty deltas behave correctly.
- Generic handlers: accept singular TOOL_CALL finish reason in addition to
  TOOL_CALLS, matching the Cohere handlers.
- _get_optional_params: when both max_tokens and max_completion_tokens are
  provided, explicitly prefer max_completion_tokens instead of relying on
  dict iteration order.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): emit content=None instead of empty string for text-free generic stream chunks

Co-authored-by: Yassin Kortam <[email protected]>

* test(oci): expect content=None for text-free generic stream chunks

handle_generic_stream_chunk now emits content=None instead of empty
string when a chunk carries no text parts. Update the corresponding
no-message test to match.

* codeql: narrow OCI sha256 suppression to query-filter, not whole file

paths-ignore was suppressing every CodeQL query on
litellm/llms/oci/common_utils.py, hiding all future findings in a
security-critical file (private key loading, credential resolution,
URL construction, RSA signing). Move the suppression for
py/weak-sensitive-data-hashing into query-filters so common_utils.py
remains fully analyzed by every other query.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): use locale-independent RFC 7231 date for manual signing

email.utils.formatdate(usegmt=True) emits canonical English weekday/
month abbreviations regardless of system locale, so signature
verification doesn't break on non-en_US deployments.

* fix(oci): strip 'oci/' prefix in get_vendor_from_model

Previously, get_vendor_from_model split on '.' without stripping the
optional 'oci/' provider prefix, so 'oci/cohere.command-a-03-2025' was
routed through the GENERIC pipeline instead of COHERE.

Co-authored-by: Yassin Kortam <[email protected]>

* codeql: scope OCI sha256 suppression to common_utils.py via filter-sarif

Replace the global query-filters exclude for py/weak-sensitive-data-hashing
with a SARIF post-filter that only drops the alert when it originates from
litellm/llms/oci/common_utils.py, keeping the rule active on every other
SHA-256 callsite in the repository.

* Fix OCI chat bugs: tool_calls None key, dead max_tokens dedup, single-event stream text suppression

- handle_cohere_response: omit tool_calls key from message dict when None,
  matching the generic handler's behaviour and avoiding tripping consumers
  that key off 'tool_calls' in message.
- _get_optional_params: remove dead prefer_max_completion branch. By the
  time this helper runs, map_openai_params has already collapsed
  max_tokens/max_completion_tokens onto the OCI alias, so the OpenAI-key
  membership check is unreachable.
- handle_cohere_stream_chunk: add prior_text_emitted parameter mirroring
  prior_tool_calls_emitted. The terminal consolidation chunk's text is
  only suppressed when prior deltas already emitted text — otherwise
  (degenerate single-event stream) the text passes through so the
  response content isn't silently lost. OCIStreamWrapper now tracks
  emitted text alongside emitted tool calls.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): preserve all text parts in generic response and emit SYSTEM role for Cohere

- handle_generic_response: iterate all content parts and concatenate text
  (matches the streaming handler) so non-leading text parts are not lost
  and a leading non-text part does not suppress trailing text.
- adapt_messages_to_cohere_standard: emit CohereSystemMessage for system
  messages so direct callers do not silently drop them. The Cohere
  request builder filters system messages before calling this helper to
  avoid duplicating preambleOverride content into chatHistory.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): normalise dict-format tool_choice to OCI flat uppercase shape

The OCI Generative AI API only accepts toolChoice values of the form
{"type": "AUTO"|"NONE"|"REQUIRED"} or {"type": "FUNCTION",
"name": "<fn>"}. The previous conversion only handled string
tool_choice values, so OpenAI's standard dict shape
{"type": "function", "function": {"name": "<fn>"}} passed
through unchanged and was rejected by OCI with a 400.

Normalise the dict shape by uppercasing the discriminator and hoisting
the function name to the top level. Also accept dict variants of the
non-function selectors (e.g. {"type": "auto"}).

* test(oci): exercise system-message filtering at transform_request boundary

adapt_messages_to_cohere_standard now emits SYSTEM-role entries by design
so direct callers don't silently drop system content. The Cohere request
builder filters system messages before calling the helper and routes them
into preambleOverride, so the user-visible 'no SYSTEM in chatHistory'
guarantee holds at the transform_request boundary, where the test should
live.

* fix(oci/chat): extract tool_choice/response_format helpers to satisfy PLR0915

_get_optional_params exceeded ruff's 50-statement cap. The toolChoice and
responseFormat normalisation blocks are self-contained mutations, so move
them to module-level helpers.

* fix(oci): normalize None finishReason in generic non-streaming handler; drop dead Cohere system-role branch

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/generic): silence mypy assignment error on cleared finish_reason

* fix(docker): install libatomic in builder for prisma nodeenv binary

The prebuilt node binary that prisma-python's nodeenv downloads links
against libatomic.so.1, which Wolfi does not pull in via gcc/nodejs.
Without this, fresh Docker builds (no GHA cache hit) fail at
`prisma generate` with:
  node: error while loading shared libraries: libatomic.so.1

* fix(oci): raise on invalid tool_choice instead of silently passing OpenAI shape

_normalize_tool_choice previously left an OpenAI-format dict in selected_params['toolChoice'] when the type was unrecognized or when 'FUNCTION' was given with a missing/empty name. OCI would then reject the request with a non-obvious error. Raise ValueError with a clear message in these cases.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): raise OCIError instead of ValueError in _normalize_tool_choice

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/generic): declare non-security intent on sha256 for synthetic tool-call id

* fix(oci): simplify _get_optional_params and reject invalid tool_choice types

- Collapse the two-loop _get_optional_params into a single pass with
  clear precedence (OpenAI key wins over OCI alias; first OpenAI key
  reaching a given OCI target wins). Removes the redundant maxTokens
  special-case in the second loop and makes the map_openai_params /
  transform_request handoff easier to reason about.
- Raise OCIError when _normalize_tool_choice sees an unexpected type
  (list, bool, int, ...) instead of silently letting it through to the
  OCI API where it would produce an opaque server-side error.

Co-authored-by: Yassin Kortam <[email protected]>

* Remove no-op data['stream'] deletion in OCI stream wrappers

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): always send Cohere isStream field explicitly

Match OCIChatRequestPayload by defaulting CohereChatRequest.isStream to
False instead of None so model_dump(exclude_none=True) does not silently
omit the field on non-streaming requests.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): revert Cohere isStream to Optional[bool]=None to preserve omission semantics

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/generic): raise OCIError on empty choices instead of IndexError

Pydantic accepts an empty choices list when validating OCICompletionResponse, so accessing chatResponse.choices[0] could raise an unhandled IndexError. Surface it as OCIError so the response error path is consistent with the existing (TypeError, ValidationError) guard.

* fix(oci/cohere): map top_k -> topK so Cohere topK param is settable

The Cohere param map (derived from the GENERIC map) had no entry for
topK. Since the simplified _get_optional_params only iterates over
param_map entries, callers had no way to pass topK to CohereChatRequest
(neither via an OpenAI-style key nor via the OCI alias).

Add 'top_k': 'topK' to the Cohere map only — OCIChatRequestPayload
(GENERIC) has no topK field. _get_optional_params accepts both the
OpenAI key (top_k) and the OCI alias (topK) in optional_params, so this
covers both calling conventions.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): tighten cohere stream dedup flags and forward stream args in embed signing

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/chat): reorder dict guard and wrap stream chunk json.loads

- Move isinstance(response_json, dict) check before .get("error") so
  the guard runs before the attribute access it is supposed to protect.
- Wrap json.loads in OCIStreamWrapper.chunk_creator with try/except so
  malformed SSE payloads surface as OCIError instead of a raw
  JSONDecodeError propagating out of the stream loop.

* fix(oci/cohere stream): only flag text emitted on non-empty content

An intermediate Cohere SSE chunk carrying text="" was flipping
_cohere_text_emitted via the "is not None" check, which then caused
the terminal consolidation chunk to drop its real text as a duplicate.
Use a truthy check so only actual content marks the stream as having
emitted text.

* test(oci): end-to-end proxy integration test against real OCI GenAI

Spins up the litellm proxy via the console-script entrypoint with a
minimal OCI-only config and drives real OpenAI-shaped HTTP requests
through it against OCI GenAI. Covers non-streaming chat, streaming
chat, embeddings, and /v1/models for Cohere, Llama, Gemini, and Grok.

Skips automatically when ~/.oci/config is absent or when the active
profile uses session-token auth (the OCI provider currently only
consumes OCI_* env vars; session tokens would need an in-process
signer). API-key profiles work out of the box.

* test(oci): move proxy integration test to tests/integration/

tests/llm_translation/ is mock-only; the OCI proxy integration test
spawns a real proxy subprocess and makes live HTTP calls, so move
it (and the companion config) to tests/integration/ alongside the
existing test_oci_integration.py.

* fix(oci): dedupe finish-reason mapping and batch Cohere tool results

- Extract _normalize_oci_finish_reason helper so the four chat handlers
  (Cohere/GENERIC, sync/stream) share one OCI->OpenAI mapping instead of
  four near-identical if/elif chains.
- Merge consecutive OpenAI tool-role messages into a single
  CohereToolMessage with multiple toolResults entries, matching the OCI
  Cohere API's expectation for parallel tool calls in one assistant turn.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): drop dead Cohere toolChoice field and emit GENERIC tool-call dicts inline

- Remove the unreachable toolChoice field from CohereChatRequest. The
  Cohere param map explicitly marks tool_choice as unsupported, so the
  field can never be populated through the normal optional_params flow
  and only confused the public model surface.
- Build GENERIC stream tool-call dicts inline (id/type/function shape)
  instead of round-tripping through ChatCompletionMessageToolCall and
  model_dump(). Matches handle_cohere_stream_chunk so downstream
  stream-mergers see the same minimal payload regardless of which
  vendor produced the chunk.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(docker): drop redundant libatomic from non_root builder

litellm_internal_staging already fixes the prisma `nodeenv` build
failure at the root cause by restoring `npm` to the builder (#28519):
with npm on PATH, prisma-python uses the system Node and never downloads
the nodeenv binary that links against libatomic.so.1. After merging
internal_staging the libatomic line is dead weight, so remove it.

https://claude.ai/code/session_01SwKzxRxgUhLFyyEf4UV812

* fix(oci/catalog): add openai.gpt-5{,-mini,-nano} entries with supports_reasoning

Without these catalog entries, supports_reasoning(model='openai.gpt-5*',
custom_llm_provider='oci') returned False, so _model_uses_max_completion_tokens
fell back to the default and OCI rejected the request with HTTP 400
('Use maxCompletionTokens instead.'). Add the three entries so the catalog-driven
maxCompletionTokens routing works against a stock LiteLLM install.

Also reword the test fixture docstring — the bundled backup now actually ships
these entries, so the fixture is only a fallback for environments that loaded
their cost map from a stale remote source.

---------

Co-authored-by: Tai An <[email protected]>
Co-authored-by: Vincent <[email protected]>
Co-authored-by: Kris Xia <[email protected]>
Co-authored-by: d 🔹 <[email protected]>
Co-authored-by: Fabrizio Cafolla <[email protected]>
Co-authored-by: Filippo Menghi <[email protected]>
Co-authored-by: Tom Denham <[email protected]>
Co-authored-by: escon1004 <[email protected]>
Co-authored-by: Divyansh Singhal <[email protected]>
Co-authored-by: robin-fiddler <[email protected]>
Co-authored-by: Michael-RZ-Berri <[email protected]>
Co-authored-by: Michael Riad Zaky <[email protected]>
Co-authored-by: ryan-crabbe-berri <[email protected]>
Co-authored-by: Krrish Dholakia <[email protected]>
Co-authored-by: Sameer Kankute <[email protected]>
Co-authored-by: Cursor <[email protected]>
Co-authored-by: Yassin Kortam <[email protected]>
Co-authored-by: Federico Kamelhar <[email protected]>
Co-authored-by: Claude <[email protected]>

* fix(opentelemetry): JSON-serialize dict metadata fields for OTEL span attributes (BerriAI#27451) (BerriAI#27455)

Squash-merged by litellm-agent from Anai-Guo's PR.

* feat(dashscope): add embeddings and reranks(qwen3-rerank) support via OpenAI-compatible endpoint (BerriAI#27508)

Squash-merged by litellm-agent from yimao's PR.

* fix(vertex_ai/gemini): raise BadRequestError when image_url or url fi… (BerriAI#24550)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix(vertex_ai): raise error on mid-stream 429/error chunks instead of silently swallowing (BerriAI#23711)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix: raise BadRequestError for file content blocks missing 'file' sub… (BerriAI#24503)

Squash-merged by litellm-agent from krisxia0506's PR.

* Fix Gemini MIME detection for extensionless GCS URIs (BerriAI#27278)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix(vertex_ai/partner_models): drop unused vertexai SDK gate from count_tokens (closes BerriAI#28084) (BerriAI#28107)

Squash-merged by litellm-agent from voidborne-d's PR.

* feat(chart): add support for autoscaling behavior in HPA (BerriAI#27990)

Squash-merged by litellm-agent from FabrizioCafolla's PR.

* feat(proxy): add blocked flag to models for pause/resume from the UI (BerriAI#27927)

Squash-merged by litellm-agent from Cyberfilo's PR.

* fix: pass socket timeouts to Redis cluster clients (BerriAI#27920)

Squash-merged by litellm-agent from tomdee's PR.

* Fix/cache token (BerriAI#28009)

Squash-merged by litellm-agent from escon1004's PR.

* fix(deepseek): forward reasoning_content in multi-turn thinking mode conversations (BerriAI#28080)

Squash-merged by litellm-agent from Divyansh8321's PR.

* fix(guardrails): return HTTP 400 instead of 500 for blocked requests (BerriAI#27617)

* fix: reset org and tag budgets (BerriAI#27326)

* reset org budgets

* reset tag budgets

---------

Co-authored-by: Michael Riad Zaky <[email protected]>

* fix(ui): omit allowed_routes from key edit save when unchanged (BerriAI#27553)

* fix(ui): omit allowed_routes from key edit save when unchanged

When a team admin opens Edit Settings on a key with key_type=AI APIs and
saves without changing anything, the UI re-sends the existing allowed_routes
value, which the backend's _check_allowed_routes_caller_permission gate
rejects for non-proxy-admins (LIT-2681).

Strip allowed_routes from the patch in handleSubmit when it deep-equals the
original keyData.allowed_routes. The backend treats absence as "leave alone,"
so no-op saves now succeed for non-admins. Admins explicitly editing the
field still send the new value.

* fix(ui): order-insensitive allowed_routes diff + cover null-original case

Address Greptile review:

- Switch the "is allowed_routes unchanged" check to a Set-based comparison so
  a server-side reorder of the array doesn't register as a user edit and
  re-trigger LIT-2681.
- Add two regression tests: (1) keyData.allowed_routes is null and the form
  is untouched — patch should strip the field; (2) server returned routes in
  a different order than the user originally entered — patch should still
  recognize the value as unchanged.

* chore(ui): strip ticket refs and tighten comments in key edit fix

- Remove internal-tracker references from in-code comments
- Tighten the WHY comment in handleSubmit to two lines
- Drop redundant test-block comments — test names already describe the case

* fix(ui): annotate Set<string> generic in allowed_routes diff to fix tsc

* fix(guardrails): return HTTP 400 instead of 500 for guardrail-blocked requests

GuardrailRaisedException and BlockedPiiEntityError both lacked a
status_code attribute.  When these exceptions reached the proxy
exception handler (getattr(e, 'status_code', 500)), the fallback
defaulted to HTTP 500 — making intentional guardrail blocks
indistinguishable from server errors and causing unnecessary client
retries.

Changes:
- Add status_code=400 (keyword-only) to GuardrailRaisedException
- Add status_code=400 (keyword-only) to BlockedPiiEntityError
- Update _is_guardrail_intervention() to recognize both exceptions
  so downstream loggers record 'guardrail_intervened' instead of
  'guardrail_failed_to_respond'
- Add 6 unit tests for default/custom status codes and getattr pattern
- Strengthen existing blocked-action test with status_code assertion

Fixes BerriAI#24348

---------

Co-authored-by: Michael-RZ-Berri <[email protected]>
Co-authored-by: Michael Riad Zaky <[email protected]>
Co-authored-by: ryan-crabbe-berri <[email protected]>
Co-authored-by: Krrish Dholakia <[email protected]>

* fix(router/proxy): address Greptile P1+P2 review comments on PR BerriAI#28161

- router: raise ServiceUnavailableError (503) instead of RouterRateLimitErrorBasic (429)
  when a specifically-addressed deployment is administratively blocked; 429 misleads
  retry-enabled clients into spinning forever against a paused model
- proxy_server: compute get_fully_blocked_model_names() once before both branches in
  model_list() instead of duplicating the call in each branch
- deepseek: upgrade silent debug log to warning when injecting placeholder
  reasoning_content so callers are clearly notified of degraded multi-turn quality
- tests: update two blocked-deployment assertions to expect ServiceUnavailableError

Co-authored-by: Cursor <[email protected]>

* fix: address bug detection findings (cache token order, mutable defaults)

Co-authored-by: Yassin Kortam <[email protected]>

* fix: address bugs in async pass-through, anthropic cache token detection, rerank tests

- async_get_available_deployment_for_pass_through: enforce blocked check on specific deployments
- cost_calculator: detect anthropic-style usage by attribute presence (not truthiness) to avoid mixing OpenAI cached_tokens into anthropic normalization when read=0
- dashscope rerank tests: pass request to httpx.Response constructions for consistency

Co-authored-by: Yassin Kortam <[email protected]>

* fix code qa

* fix(vertex_ai/gemini): strip MIME parameters from GCS contentType

GCS object metadata's contentType field can include parameters such as
'text/html; charset=utf-8'. Strip them in _apply_gemini_mime_type_aliases
so downstream get_file_extension_from_mime_type sees a bare MIME type.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(vertex_ai/gemini): clarify mime-type error message string concatenation

Co-authored-by: Yassin Kortam <[email protected]>

---------

Co-authored-by: Tai An <[email protected]>
Co-authored-by: Vincent <[email protected]>
Co-authored-by: Kris Xia <[email protected]>
Co-authored-by: d 🔹 <[email protected]>
Co-authored-by: Fabrizio Cafolla <[email protected]>
Co-authored-by: Filippo Menghi <[email protected]>
Co-authored-by: Tom Denham <[email protected]>
Co-authored-by: escon1004 <[email protected]>
Co-authored-by: Divyansh Singhal <[email protected]>
Co-authored-by: robin-fiddler <[email protected]>
Co-authored-by: Michael-RZ-Berri <[email protected]>
Co-authored-by: Michael Riad Zaky <[email protected]>
Co-authored-by: ryan-crabbe-berri <[email protected]>
Co-authored-by: Krrish Dholakia <[email protected]>
Co-authored-by: Cursor <[email protected]>
Co-authored-by: Yassin Kortam <[email protected]>

… fixes, model catalog) (BerriAI#28223)

* fix(opentelemetry): JSON-serialize dict metadata fields for OTEL span attributes (#27451) (#27455)

Squash-merged by litellm-agent from Anai-Guo's PR.

* feat(dashscope): add embeddings and reranks(qwen3-rerank) support via OpenAI-compatible endpoint (#27508)

Squash-merged by litellm-agent from yimao's PR.

* fix(vertex_ai/gemini): raise BadRequestError when image_url or url fi… (#24550)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix(vertex_ai): raise error on mid-stream 429/error chunks instead of silently swallowing (#23711)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix: raise BadRequestError for file content blocks missing 'file' sub… (#24503)

Squash-merged by litellm-agent from krisxia0506's PR.

* Fix Gemini MIME detection for extensionless GCS URIs (#27278)

Squash-merged by litellm-agent from krisxia0506's PR.

* fix(vertex_ai/partner_models): drop unused vertexai SDK gate from count_tokens (closes #28084) (#28107)

Squash-merged by litellm-agent from voidborne-d's PR.

* feat(chart): add support for autoscaling behavior in HPA (#27990)

Squash-merged by litellm-agent from FabrizioCafolla's PR.

* feat(proxy): add blocked flag to models for pause/resume from the UI (#27927)

Squash-merged by litellm-agent from Cyberfilo's PR.

* fix: pass socket timeouts to Redis cluster clients (#27920)

Squash-merged by litellm-agent from tomdee's PR.

* Fix/cache token (#28009)

Squash-merged by litellm-agent from escon1004's PR.

* fix(deepseek): forward reasoning_content in multi-turn thinking mode conversations (#28080)

Squash-merged by litellm-agent from Divyansh8321's PR.

* fix(guardrails): return HTTP 400 instead of 500 for blocked requests (#27617)

* fix: reset org and tag budgets (#27326)

* reset org budgets

* reset tag budgets

---------

Co-authored-by: Michael Riad Zaky <[email protected]>

* fix(ui): omit allowed_routes from key edit save when unchanged (#27553)

* fix(ui): omit allowed_routes from key edit save when unchanged

When a team admin opens Edit Settings on a key with key_type=AI APIs and
saves without changing anything, the UI re-sends the existing allowed_routes
value, which the backend's _check_allowed_routes_caller_permission gate
rejects for non-proxy-admins (LIT-2681).

Strip allowed_routes from the patch in handleSubmit when it deep-equals the
original keyData.allowed_routes. The backend treats absence as "leave alone,"
so no-op saves now succeed for non-admins. Admins explicitly editing the
field still send the new value.

* fix(ui): order-insensitive allowed_routes diff + cover null-original case

Address Greptile review:

- Switch the "is allowed_routes unchanged" check to a Set-based comparison so
  a server-side reorder of the array doesn't register as a user edit and
  re-trigger LIT-2681.
- Add two regression tests: (1) keyData.allowed_routes is null and the form
  is untouched — patch should strip the field; (2) server returned routes in
  a different order than the user originally entered — patch should still
  recognize the value as unchanged.

* chore(ui): strip ticket refs and tighten comments in key edit fix

- Remove internal-tracker references from in-code comments
- Tighten the WHY comment in handleSubmit to two lines
- Drop redundant test-block comments — test names already describe the case

* fix(ui): annotate Set<string> generic in allowed_routes diff to fix tsc

* fix(guardrails): return HTTP 400 instead of 500 for guardrail-blocked requests

GuardrailRaisedException and BlockedPiiEntityError both lacked a
status_code attribute.  When these exceptions reached the proxy
exception handler (getattr(e, 'status_code', 500)), the fallback
defaulted to HTTP 500 — making intentional guardrail blocks
indistinguishable from server errors and causing unnecessary client
retries.

Changes:
- Add status_code=400 (keyword-only) to GuardrailRaisedException
- Add status_code=400 (keyword-only) to BlockedPiiEntityError
- Update _is_guardrail_intervention() to recognize both exceptions
  so downstream loggers record 'guardrail_intervened' instead of
  'guardrail_failed_to_respond'
- Add 6 unit tests for default/custom status codes and getattr pattern
- Strengthen existing blocked-action test with status_code assertion

Fixes #24348

---------

Co-authored-by: Michael-RZ-Berri <[email protected]>
Co-authored-by: Michael Riad Zaky <[email protected]>
Co-authored-by: ryan-crabbe-berri <[email protected]>
Co-authored-by: Krrish Dholakia <[email protected]>

* fix(router/proxy): address Greptile P1+P2 review comments on PR #28161

- router: raise ServiceUnavailableError (503) instead of RouterRateLimitErrorBasic (429)
  when a specifically-addressed deployment is administratively blocked; 429 misleads
  retry-enabled clients into spinning forever against a paused model
- proxy_server: compute get_fully_blocked_model_names() once before both branches in
  model_list() instead of duplicating the call in each branch
- deepseek: upgrade silent debug log to warning when injecting placeholder
  reasoning_content so callers are clearly notified of degraded multi-turn quality
- tests: update two blocked-deployment assertions to expect ServiceUnavailableError

Co-authored-by: Cursor <[email protected]>

* fix: address bug detection findings (cache token order, mutable defaults)

Co-authored-by: Yassin Kortam <[email protected]>

* fix: address bugs in async pass-through, anthropic cache token detection, rerank tests

- async_get_available_deployment_for_pass_through: enforce blocked check on specific deployments
- cost_calculator: detect anthropic-style usage by attribute presence (not truthiness) to avoid mixing OpenAI cached_tokens into anthropic normalization when read=0
- dashscope rerank tests: pass request to httpx.Response constructions for consistency

Co-authored-by: Yassin Kortam <[email protected]>

* fix code qa

* fix(vertex_ai/gemini): strip MIME parameters from GCS contentType

GCS object metadata's contentType field can include parameters such as
'text/html; charset=utf-8'. Strip them in _apply_gemini_mime_type_aliases
so downstream get_file_extension_from_mime_type sees a bare MIME type.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(vertex_ai/gemini): clarify mime-type error message string concatenation

Co-authored-by: Yassin Kortam <[email protected]>

* feat(oci): add embeddings, fix streaming/reasoning, expand model catalog

- Add OCIEmbedConfig with full Cohere embed support (7 models, batch up to 96)
- Fix sync streaming: split SSE events on \n\n before JSON parsing
- Fix reasoning models (Gemini 2.5, xAI Grok): make completionTokens and message
  optional in OCIResponseChoice to handle max_tokens exhausted on reasoning
- Fix compartment_id resolution in chat transform to use resolve_oci_credentials
- Fix tool call id: make OCIToolCall.id optional, generate UUID fallback for
  providers (Google via OCI) that omit it
- Add OCI_KEY env var support for inline PEM keys
- Fix datetime.utcnow() deprecation in request signing
- Expand model catalog: 29 OCI models including Llama 4, Gemini 2.5, xAI Grok,
  Cohere Command A, and all Cohere embed variants
- Add 37 live integration tests: sync/async completions for Meta/Google/xAI/Cohere,
  sync/async embeddings, tool use across all vendors, streaming, env var auth
- Add 23 embed unit tests covering all transform and validation paths

* fix(oci): remove dead OCI elif branch in utils.py, align async split_chunks with sync version

* test(oci): add unit tests for split_chunks fix and no-duplicate-OCI-branch guard

* fix(oci): address remaining bugs from issue #25082 — streaming signed body, Cohere stop sequences, hardcoded defaults

- Bug 1: sync and async streaming paths now use signed_json_body when provided
  instead of re-serializing data with json.dumps() — the OCI RSA-SHA256 signature
  covers the exact request body bytes, so re-serializing produces an invalid sig
- Bug 3: Cohere stop sequences now map to 'stopSequences' (was incorrectly 'stop')
- Bug 4: removed hardcoded Cohere defaults (maxTokens=600, temperature=1, topK=0,
  topP=0.75, frequencyPenalty=0) that silently overrode user intent on every call
- Added 6 unit tests covering all three fixes

* fix(oci): comprehensive code quality pass — bugs, tests, schema accuracy

- Fix Cohere tool call IDs (was always call_0; now UUID per call)
- Fix TOOL_CALL finish reason mapping in both sync and streaming paths
- Fix Cohere stop parameter mapping (stop → stopSequences)
- Remove hardcoded Cohere defaults (maxTokens/topK/topP/frequencyPenalty)
- Fix content[0] safety guard against empty content arrays
- Fix streaming signed body used consistently (not re-serialized)
- Raise OCIError (not bare Exception/ValueError) throughout
- Centralize OCI_API_VERSION constant; import uuid at module level
- Fix embed get_complete_url to strip trailing slashes from api_base
- Fix OCIEmbedResponse schema: add inputTextTokenCounts (actual OCI field)
- Fix embed usage computed from inputTextTokenCounts (sum of per-input counts)
- Fix Cohere toolCallId included in tool result messages
- Add OCIToolCall.id as Optional (absent in Google/xAI streaming chunks)
- Update tests to reflect correct behavior (no hardcoded defaults, UUID ids,
  deferred credential validation, OCIError vs ValueError, real response schema)

* test(oci): move integration tests to tests/llm_translation/

Addresses greptile P1: tests/test_litellm/ is for mock-only unit tests
(make test-unit target). Real-network OCI tests now live in the correct
location alongside other provider integration tests.

* fix(oci): align types and transformation with official OCI SDK

- Remove OCIVendors.GEMINI — apiFormat="GEMINI" is invalid; all non-Cohere
  models use apiFormat="GENERIC"
- Add toolChoice, logitBias, logProbs to OCIChatRequestPayload so params
  present in the mapping are no longer silently dropped by Pydantic
- Exclude n→numGenerations from Cohere param map (not a Cohere API field)
- Fix CohereToolResult: change callId/result to call/outputs matching
  the OCI SDK's CohereToolResult structure
- Fix CohereToolMessage: replace non-existent toolCallId with toolResults
  list; update adapt_messages_to_cohere_standard to build proper tool-result
  history entries by resolving tool call name+params from preceding assistant
  messages
- Map generic-model stream finish reasons to OpenAI convention
  (COMPLETE→stop, MAX_TOKENS→length, TOOL_CALLS→tool_calls), consistent
  with the existing Cohere streaming path
- Add optional id field to OCIEmbedResponse so valid API responses
  carrying an id are not rejected by the Pydantic model

* fix(oci): use 'output' key in Cohere tool result outputs (matches reference impl)

* fix(oci): port schema/type utilities from langchain-oracle reference impl

- Add resolve_oci_schema_refs: inline $ref/$defs — OCI rejects JSON Schema refs
- Add resolve_oci_schema_anyof: flatten Optional[T] anyOf (Pydantic v2 emits these)
- Add sanitize_oci_schema: strip title, normalise null types, ensure array items
- Add OCI_JSON_TO_PYTHON_TYPES: Cohere expects Python type names (str/int/float),
  not JSON Schema names (string/integer/number)
- Add enrich_cohere_param_description: embed enum/format/range/pattern constraints
  into description since CohereParameterDefinition has no dedicated fields
- Apply all of the above in adapt_tool_definitions_to_cohere_standard and
  adapt_tool_definition_to_oci_standard
- Fix toolChoice conversion: map OpenAI string ('auto','none','required') to OCI
  dict form ({"type":"AUTO"} etc.) — the API rejects plain strings
- Update unit test expectations to match correct Python type names and enriched
  descriptions

* refactor(oci): split transformation.py into cohere.py and generic.py

transformation.py was 1 243 lines doing too many jobs. Split along the
same boundaries as the langchain-oracle reference (providers/cohere.py,
providers/generic.py):

  chat/cohere.py   — Cohere message/tool building, response + stream parsing
  chat/generic.py  — Generic message/tool building, response + stream parsing
  transformation.py — thin OCIChatConfig orchestrator + OCIStreamWrapper

Public symbols (OCIChatConfig, OCIStreamWrapper, adapt_messages_to_*,
OCIRequestWrapper, version, …) remain importable from transformation.py
for backward compatibility. OCIStreamWrapper gains delegating shims for
_handle_cohere_stream_chunk and _handle_generic_stream_chunk so existing
test call sites keep working unchanged.

transformation.py: 1 243 → 620 lines

* refactor(oci): principal-level code quality pass

- Remove _extract_text_content duplication — single definition in cohere.py,
  imported where needed; instance method on OCIChatConfig eliminated
- Move cryptography imports to module level with _CRYPTOGRAPHY_AVAILABLE flag
  and _require_cryptography() guard; no more re-import on every signing call
- Move litellm version import to module level via litellm._version; remove
  inline import inside validate_oci_environment
- sign_with_manual_credentials now returns Tuple[dict, bytes] matching
  sign_with_oci_signer — asymmetry eliminated, Optional[bytes] guards removed
  throughout stream wrappers (signed_json_body: bytes = b"")
- Rename _openai_to_oci_cohere_param_map → openai_to_oci_cohere_param_map
  for consistency with openai_to_oci_generic_param_map
- Remove double-key bug in map_openai_params where responseFormat was stored
  under both OCI and OpenAI key names simultaneously
- Remove delegating shims (adapt_messages_to_cohere_standard,
  adapt_tool_definitions_to_cohere_standard, _handle_generic_stream_chunk)
  from OCIChatConfig/OCIStreamWrapper; tests now import directly from
  cohere.py and generic.py where symbols live
- Trim __all__ to 7 genuine public symbols; remove the 13-symbol list that
  existed only to support test imports
- Collapse per-model integration test classes into pytest.mark.parametrize;
  CHAT_MODELS list is the single source of truth for model-specific config
- Black + Ruff clean across all OCI files

* fix(oci): address PR review findings

- types/llms/oci.py: add "TOOL_CALL" to CohereChatResponse.finishReason
  Literal so Pydantic does not raise ValidationError on non-streaming
  Cohere tool-use calls (Greptile P1)
- test_oci_cohere_tool_calls.py: add test covering TOOL_CALL finish reason
- model_prices_and_context_window.json: remove 6 duplicate oci/cohere.embed-*
  keys that were silently overridden by the more complete entries already
  present in the file (Greptile P1)
- common_utils.py: move OCI_API_VERSION here from chat/transformation.py
  so embed/transformation.py does not need to import chat/transformation;
  change Protocol stub body from ... to pass (CodeQL "statement no effect");
  add comment to sha256_base64 clarifying it implements OCI HTTP signing
  spec, not password hashing (CodeQL false positive)
- chat/transformation.py: import CustomStreamWrapper from
  litellm_core_utils.streaming_handler instead of litellm.utils to reduce
  import cycle depth (CodeQL cyclic import)
- chat/cohere.py, chat/generic.py: import Usage and
  ChatCompletionMessageToolCall from litellm.types.utils instead of
  litellm.utils for the same reason
- embed/transformation.py: import OCI_API_VERSION from common_utils
  instead of chat/transformation (removes the embed→chat import edge)

* test(oci): add unit tests to improve patch coverage

- test_oci_common_utils.py (new): covers sha256_base64, build_signature_string,
  OCIRequestWrapper.path_url, resolve_oci_credentials, get_oci_base_url,
  validate_oci_environment, sign_with_oci_signer error paths, sign_oci_request
  routing, load_private_key_from_file error paths, resolve_oci_schema_refs
  (including circular ref and external $ref), resolve_oci_schema_anyof,
  sanitize_oci_schema (all branches), enrich_cohere_param_description
- test_oci_generic_chat.py (new): covers content-message error paths (non-dict
  item, unsupported type, non-string text, invalid image_url), tool-call
  validation error paths, adapt_messages_to_generic_oci_standard error paths,
  handle_generic_response (None message, text content, tool calls),
  handle_generic_stream_chunk (finish reasons, streaming tool calls),
  OCIStreamWrapper non-string chunk error
- test_oci_chat_transformation.py: add error paths for validate_environment
  (empty messages), transform_request (missing compartment_id, Cohere without
  user messages), transform_response (error key), map_openai_params
  (unsupported param with and without drop_params), tool_choice string mapping
- test_oci_cohere_tool_calls.py: add edge cases for stream chunk finish
  reasons (TOOL_CALL, MAX_TOKENS, unknown), _extract_text_content with
  non-dict list items and non-string input,
  adapt_messages_to_cohere_standard with malformed JSON tool arguments

* fix(oci): rename supports_streaming to supports_native_streaming in model prices

The JSON schema for model_prices_and_context_window.json uses
`supports_native_streaming` (not `supports_streaming`) and has
`additionalProperties: false`. Rename the field across all OCI
entries to pass the schema validation test.

* test(oci): add 67 tests targeting uncovered happy paths for coverage

Boost patch coverage on the four lowest-coverage OCI files:
- common_utils.py: sign_with_manual_credentials (oci_key / oci_key_file
  paths), sign_oci_request routing, _require_cryptography
- generic.py: adapt_messages_to_generic_oci_standard (all roles),
  adapt_tool_definition_to_oci_standard, adapt_tools_to_openai_standard,
  handle_generic_stream_chunk text/finish-reason paths
- cohere.py: _extract_text_content, adapt_messages_to_cohere_standard
  (all roles including tool results), handle_cohere_response /
  handle_cohere_stream_chunk all finish-reason branches
- transformation.py: get_vendor_from_model, OCIChatConfig._get_optional_params
  (toolChoice string→dict, responseFormat, tools for both vendors),
  transform_request for GENERIC model, get_sync/async_custom_stream_wrapper
  with mocked HTTP, OCIStreamWrapper.chunk_creator happy paths

* fix(oci): suppress CodeQL false positive on sha256_base64 (OCI HTTP signing, not password hashing)

* fix(oci): remove 6 duplicate model price entries and reconcile conflicting values

Six OCI chat model keys appeared twice in model_prices_and_context_window.json
with conflicting pricing/context data (JSON parsers silently discard the first).
Remove the first-occurrence entries and update the surviving entries:
- meta.llama-4-maverick / llama-4-scout: keep updated entries (free preview
  pricing, larger context windows, vision support)
- meta.llama-3.1-70b: keep original pricing, restore supports_native_streaming
- google.gemini-2.5-{flash,pro,flash-lite}: keep OCI pricing page values,
  restore supports_native_streaming

* fix(oci): route GPT-5 family to maxCompletionTokens

GPT-5 / GPT-5-mini / GPT-5-nano / GPT-5.5 on OCI reject "maxTokens"
with HTTP 400:

  Invalid 'maxTokens': Unsupported parameter: 'maxTokens' is not
  supported with this model. Use 'maxCompletionTokens' instead.

(Same convention as OpenAI's reasoning-API contract.)

Add a model-aware rename in OCIChatConfig._get_optional_params so the
request payload uses maxCompletionTokens when the model id starts with
openai.gpt-5. Regular Llama / Cohere / Gemini / GPT-4.x continue to use
maxTokens unchanged.

Also widen OCIChatRequestPayload to carry the new optional field so it
survives Pydantic serialization.

Verified live against OCI us-chicago-1:
- openai.gpt-5, gpt-5-mini, gpt-5-nano, gpt-5.5 all return 200
- Full feature sweep on gpt-5.5 (basic, system, multi-turn, streaming,
  tools, usage) all green
- meta.llama-3.3-70b-instruct still uses maxTokens (no regression)

4 new unit tests cover the helper, the routing in both pre- and
post-translation states, and Pydantic serialization.

* ci(oci): fix CI failures — black formatting + recursive_detector ignore

- Run black on litellm/llms/oci/common_utils.py + 3 OCI test files
  that drifted out of black-compliance during the rebase.
- Add the three bounded recursive functions in oci/common_utils.py
  (`_resolve`, `resolve_oci_schema_anyof`, `sanitize_oci_schema`) to
  the recursive_detector IGNORE_FUNCTIONS list. All three are bounded:
  `_resolve` uses a `resolving_stack` cycle guard; the other two are
  bounded by JSON-schema tree depth (no cycles in well-formed input),
  matching the pattern of the existing OCI/Vertex schema walkers
  already on the list.

* fix(oci): silence MyPy errors in cohere.py — typed-dict access

Two errors flagged by `lint` CI:

  llms/oci/chat/cohere.py:73:  "object" has no attribute "__iter__"
  llms/oci/chat/cohere.py:119: No overload variant of "get" of "dict"
                               matches argument types "object", "CohereToolCall"

Both stem from `msg.get("tool_calls")` / `msg.get("tool_call_id")`
returning `object` per the AllMessageValues TypedDict union. Bind to
`Any` locally for the iteration and coerce the lookup key with `str()`,
removing the now-unused `# type: ignore` on those lines.

No behaviour change — pure type-narrowing for the type checker.

* fix(oci): silence CodeQL py/weak-sensitive-data-hashing on sha256_base64

CodeQL's taint analysis traces request bodies back to environment-loaded
secrets and flags `hashlib.sha256(body).digest()` as
`py/weak-sensitive-data-hashing` — even though SHA-256 is the algorithm
mandated by the OCI HTTP request signing spec for the
`x-content-sha256` header (not a password/secret hash).

The previous suppression used legacy `# lgtm[...]` syntax which the
modern CodeQL action ignores. Switch to Python's standard
`hashlib.sha256(..., usedforsecurity=False)` (Python 3.9+) which CodeQL
honours as a non-security declaration. Behaviour unchanged.

* feat(oci): add reasoning_effort passthrough — only true missing primitive

OCI's GenericChatRequest exposes a reasoningEffort field
(NONE/MINIMAL/LOW/MEDIUM/HIGH) that's the single biggest cost knob for
reasoning-capable models on the service:

  - GPT-5 family
  - Gemini 2.5
  - Grok reasoning variants (3-mini, 4-fast, 4.20)
  - Cohere Command-A-Reasoning

Setting reasoning_effort=LOW typically cuts reasoning-token spend 5-10×
vs the default. Without exposing this, litellm users had no way to tune
cost-vs-quality on these models.

The other GenericChatRequest fields (verbosity, parallel_tool_calls,
logit_bias, n, metadata, web_search_options, prediction) are not
exposed because they are not missing primitives — they either duplicate
prompt-engineering, framework-level controls, or are too niche to
justify the maintenance surface. We only ship what users genuinely
can't accomplish another way.

Excluded from the Cohere v1 param map: CohereChatRequest has no
reasoningEffort field, and Cohere reasoning models
(cohere.command-a-reasoning) use COHEREV2 which is a separate request
type not covered by this PR.

Verified live: GPT-5.5 + reasoning_effort="HIGH" sends
{"reasoningEffort": "HIGH"} on the wire and OCI accepts the request.

* feat(oci): reasoning_effort + reasoning_tokens for OCI GenAI

Three small additions for OCI reasoning models, requested by users
testing the PR in production fork builds:

1. **reasoning_effort param mapping (GENERIC vendors).** OCI expects
   uppercase levels ("LOW"/"MEDIUM"/"HIGH"/"NONE") on `reasoningEffort`,
   but OpenAI-compatible clients send lowercase. Mapped + uppercased in
   `_get_optional_params`. Marked unsupported on Cohere V1/V2 since OCI
   Cohere has no reasoning models (avoids Pydantic validation failure
   on CohereChatRequest).

2. **"disable" → "NONE" mapping.** OpenAI uses "disable" to turn off
   reasoning; OCI uses "NONE". Without this, callers get a 400.

3. **reasoning_tokens propagated to Usage.** OCI returns
   `completionTokensDetails.reasoningTokens` but it wasn't being passed
   to LiteLLM's Usage object. Now flows through to
   `Usage.completion_tokens_details.reasoning_tokens` so callers can
   track reasoning token consumption for cost/observability.

Tests: 7 new unit tests in TestOCIReasoningEffort covering upper/lower
case, "disable"→"NONE", Cohere drop/raise paths, and reasoning_tokens
extraction (with and without completionTokensDetails). 5 new live
integration tests against xai.grok-3-mini in us-chicago-1 verifying the
full request/response loop end-to-end. Existing
test_transform_response_simple_text assertion that
completion_tokens_details was None has been updated to assert
reasoning_tokens flows through.

Verified live on xai.grok-3-mini: reasoning_effort=low → OCI accepts
"LOW", returns reasoningTokens=316 in usage. reasoning_effort=disable
→ OCI accepts "NONE". Full suite: 370/370 unit + 51/51 integration.

* fix(codeql): re-scope py/weak-sensitive-data-hashing exclusion to OCI signing file

CodeQL's taint analysis re-fires the `py/weak-sensitive-data-hashing`
alert at `litellm/llms/oci/common_utils.py:103` whenever upstream code
paths into the OCI signing module change (touching `transformation.py`
opens new flow paths that CodeQL re-evaluates from scratch). The
`hashlib.sha256(..., usedforsecurity=False)` declaration silences the
direct-call form of the query but not the taint-flow form.

SHA-256 here is mandated by the OCI HTTP signing specification for the
x-content-sha256 content-integrity header — not for password storage:
https://docs.oracle.com/en-us/iaas/Content/API/Concepts/signingrequests.htm

CodeQL has no per-query path filter and GitHub Code Scanning ignores
inline lgtm/codeql comments, so path-ignoring this single ~560-line
signing utility file is the narrowest available suppression. All other
files retain full coverage of py/weak-sensitive-data-hashing — including
litellm/proxy/utils.py where the rule legitimately applies.

This restores the NEUTRAL CodeQL state the PR had on prior commits
(see `2111c98af7` for the same approach on the previous branch
evolution that the cherry-pick was rebased onto a different baseline).

* fix(oci): drop duplicate text on Cohere streaming terminal chunk

OCI Cohere's terminal SSE event re-sends the full assembled response in
`text` alongside a populated `chatHistory`. Emitting that text as another
delta concatenates the entire response onto the already-streamed output
(e.g. "How can I help?How can I help?").

Use `chatHistory is not None` as the discriminator for the consolidated
terminal event — `finishReason` is a weaker signal that could in principle
appear on a non-consolidated chunk. The two coincide today; this preserves
correctness if OCI ever ships finishReason on an incremental chunk.

Adds a live-OCI integration regression test that compares streamed vs
non-streamed length and asserts the response prefix appears only once.
Verified to fail under the previous code with the exact reported
reproduction: 'Hello! How can I help you today?Hello! How can I help you today?'.

Reported by @gotsysdba on PR #25177.

* fix(oci): buffer SSE stream across HTTP read boundaries

The old split_chunks helper split each individual HTTP read on "\n\n",
which assumed SSE event boundaries always aligned with read boundaries.
In practice the OCI streaming endpoint delivers events that may:

  - straddle two reads (chunk_creator gets a truncated JSON and crashes)
  - arrive separated by a single "\n" instead of "\n\n"
  - share a read with multiple complete events

Replace the inline split with module-level helpers _iter_sse_events
(sync) / _aiter_sse_events (async) that maintain a buffer across reads,
split on any newline, and yield only complete "data:" lines.

Add 25 regression tests covering event-split-across-reads, tiny-chunk
reads, single-newline separators, keepalive/comment lines, trailing
partial events flushed at EOF, "\r\n" line endings, and an end-to-end
smoke test that feeds an awkwardly-chopped payload through the splitter
into OCIStreamWrapper.chunk_creator.

Reported by John Lathouwers.

* test(oci): repoint TestOCIKeyNormalization to sign_with_manual_credentials

The signing helper moved from OCIChatConfig._sign_with_manual_credentials
to a module-level sign_with_manual_credentials in common_utils.py. Four
tests in TestOCIKeyNormalization still called the old method:

  - 2 failed outright with AttributeError
  - 2 passed by accident because they used pytest.raises(Exception),
    which happily caught the AttributeError instead of exercising the
    intended OCIError path

Repoint all four to the new module-level function so they exercise the
actual oci_key type-validation branch.

* fix(oci): validate oci_region before URL interpolation to prevent SSRF

Anchor oci_region to ^[a-z][a-z0-9-]{0,30}[a-z0-9]$ inside get_oci_base_url
so user-supplied regions that would redirect the signed request to an
attacker-controlled host (e.g. 'evil.com/#') fail with HTTP 400 before
the URL or signature is built. Empty string still falls back to the
us-ashburn-1 default, so existing callers are unaffected.

* test(audio): skip when gpt-4o-audio-preview is unavailable upstream

OpenAI retired `gpt-4o-audio-preview` (404 model_not_found in CI as of
2026-05-19), and the existing try/except in these tests only re-raised
on 'openai-internal' errors. Other exceptions were silently swallowed,
so the next line ran with an unbound `response`/`completion` and
failed with an unrelated UnboundLocalError that masked the real cause.

Extend the skip condition to also cover model_not_found / 'does not exist'
so the suite reports the upstream outage cleanly, matching the pattern
used in ce87c41 for the realtime and nvidia_nim rerank tests.
Re-raise unknown exceptions instead of falling through.

* fix(oci/router): catalog-driven maxCompletionTokens; generic blocked-deployment message

- Drive OCI maxCompletionTokens via supports_reasoning from the model
  catalog instead of a hardcoded openai.gpt-5 prefix. Add OCI GPT-5 family
  entries (gpt-5, gpt-5-mini, gpt-5-nano) with supports_reasoning: true.
  Gate the override to non-Cohere vendor so Cohere reasoning models keep
  maxTokens (Cohere endpoint does not accept maxCompletionTokens).
- Replace proxy-specific 'Contact your proxy admin' phrasing in the four
  Router blocked-deployment ServiceUnavailableError messages with neutral
  SDK-appropriate text.

* fix(oci/cohere): guard handle_cohere_response against missing usage

* fix(oci): address bug review findings in chat transformation

- Cohere param map: keep tool_choice/n as False (not omitted) so unsupported
  params are dropped or rejected rather than silently passed through.
- get_complete_url: when an explicit api_base/litellm.api_base is provided,
  use it as-is instead of unconditionally appending /20231130/actions/chat
  (mirrors the embed config behavior).
- Cohere stream: require both chatHistory and finishReason to be present to
  identify a terminal consolidation chunk, avoiding silent text suppression
  if chatHistory ever appears on a non-terminal chunk.
- Generic usage: use 'is not None' for reasoningTokens so a legitimate value
  of 0 is preserved instead of being treated as absent.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/cohere): emit tool calls in streaming and null content when text empty

handle_cohere_response now sets message.content to None when the Cohere
response text is empty, matching the OpenAI convention for tool-call-only
responses.

handle_cohere_stream_chunk now extracts toolCalls — both directly from
the chunk and from the terminal chunk's chatHistory CHATBOT message —
and emits them in the delta. Previously, CohereStreamChunk lacked a
toolCalls field, so any tool calls in the stream were silently dropped.

* fix(oci): preserve tool results, embed URL path, and generic finish reason

- Use SerializeAsAny on CohereChatRequest.chatHistory so subclass-specific
  fields like CohereToolMessage.toolResults are not dropped during Pydantic
  v2 serialization.
- Make OCIEmbedConfig.get_complete_url append the /20231130/actions/embedText
  action path consistently with chat, so setting litellm.api_base to the
  region inference base URL no longer posts to the bare hostname.
- Map OCI finishReason (COMPLETE / MAX_TOKENS / TOOL_CALLS) to OpenAI
  finish_reason values in handle_generic_response, mirroring the streaming
  handler and the Cohere non-streaming handler.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/generic): silence mypy assignment error on dynamic finish_reason

* fix(oci/embed): always set usage on embedding response

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/chat): append /20231130/actions/chat to explicit api_base

Restore the embed-style behavior so OCIChatConfig.get_complete_url always
appends the OCI GenAI chat path. Routing through get_oci_base_url ensures the
optional explicit api_base has its trailing slash stripped before the suffix is
joined, matching the embed config and the test_respects_explicit_api_base
expectation.

* fix(oci/cohere): mark logprobs/logit_bias unsupported and normalize unknown stream finish reasons

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/cohere): preserve trailing tool result in chatHistory

When the last message in the OpenAI-format input is a tool result (the
standard agentic continuation pattern), the prior messages[:-1] slice
silently dropped that tool result from chatHistory and the model never
saw it. Excluding the last user message by index instead keeps tool
results that trail the last user turn intact.

* fix(main): remove dead OCI embedding elif block

The earlier elif at line 5119 already routes OCI embeddings through the
base HTTP handler with the headers None-guard, so the later identical
block was unreachable dead code.

* test(oci): move integration tests out of llm_translation mock-only folder

Greptile flags tests/llm_translation/ as mock-only via a project-specific
rule; relocate the live-network OCI integration suite to tests/integration/
and adjust the in-file sys.path / run instructions accordingly.

* fix(oci/cohere): suppress tool calls on stream terminal consolidation chunk

The terminal SSE event re-sends the full assembled response in both
`text` and `chatHistory`. The existing logic already suppresses
`text` to avoid double-emit, but tool calls extracted from the
terminal chunk (via `typed_chunk.toolCalls` or the `chatHistory`
CHATBOT fallback) would still be re-emitted with fresh uuid4 IDs.
If OCI Cohere ever streams tool calls progressively in intermediate
chunks (now possible since CohereStreamChunk has a toolCalls field),
this would cause downstream agentic frameworks to execute each tool
call twice.

Suppress tool calls on the terminal consolidation chunk for the same
reason `text` is suppressed.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci,httpx): normalize finish_reason, preserve response_format, fix sync embed JSON content-type

- cohere.py / generic.py: normalize unknown OCI finishReason values (ERROR,
  ERROR_TOXIC, CONTENT_FILTERED, USER_CANCEL, ...) to 'stop' in non-streaming
  and streaming generic handlers, matching the streaming Cohere handler so
  downstream consumers switching on finish_reason aren't broken by raw OCI
  values.
- transformation.py: restore the dual-key alias so optional_params still
  carries the original 'response_format' key alongside the OCI-mapped
  'responseFormat'. Downstream litellm framework code (json_mode detection,
  logging) inspects 'response_format' after map_openai_params runs.
- llm_http_handler.py: make the sync embedding path mirror the async path —
  when sign_request returns no signed_body, send via json=data (which sets
  Content-Type: application/json) instead of data=json.dumps(data) which
  doesn't. Removes a sync/async behavioural asymmetry for non-OCI providers
  that adopt the sign_request pattern.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): clean up OCIChatConfig init, normalize generic stream finish reasons, correct embed sign_request return type

- Replace fragile setattr(self.__class__, ...) pattern in OCIChatConfig.__init__ with a @property for has_custom_stream_wrapper, matching the pattern used by other providers.
- Normalize unknown OCI finish reasons (e.g. ERROR, ERROR_TOXIC, USER_CANCEL) to 'stop' in handle_generic_stream_chunk, matching the existing Cohere stream handler behaviour.
- Tighten OCIEmbedConfig.sign_request return type from Tuple[dict, Optional[bytes]] to Tuple[dict, bytes] — sign_oci_request never returns None for the body, and this matches OCIChatConfig.sign_request.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): strip trailing action path in get_oci_base_url to avoid URL doubling

A fully-formed OCI endpoint URL (e.g. https://inference.generativeai.us-chicago-1.oci.oraclecloud.com/20231130/actions/chat) passed via api_base previously had the action path appended a second time by get_complete_url in both chat and embed configs, yielding a 404. get_oci_base_url now strips a trailing /20231130/actions/<name> so callers can always append the action path safely.

* fix(httpx): preserve sync embed data= kwarg to avoid breaking mock-based tests

The earlier sync_httpx_client.post() call passed data=json.dumps(data),
which downstream embedding tests assert on (e.g. tests for hosted_vllm,
jina_ai, watsonx). Switching to json=data changed the kwarg name and broke
those tests. The OCI signed_body path keeps using data=signed_body and is
unaffected.

* fix(oci): stable tool-call ids across stream chunks; lenient Cohere finishReason

- Replace random uuid4 per chunk with a deterministic content-derived
  digest for synthetic tool-call ids in both Cohere and Generic OCI
  handlers. Previously, when OCI omitted 'id' (always for Cohere, often
  for Generic streaming deltas), every chunk for the same logical tool
  call received a new uuid, causing downstream stream-mergers (which key
  off id) to treat each fragment as a distinct call.

- Relax CohereChatResponse.finishReason from a strict Literal[...] to
  Optional[str], matching CohereStreamChunk.finishReason. The
  handle_cohere_response 'elif oci_finish_reason is not None' fallback
  was previously unreachable because Pydantic raised ValidationError on
  any unknown value before the fallback executed. Now non-streaming
  responses degrade unknown reasons to 'stop' just like the streaming
  path.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/embed): validate OCI credentials in validate_environment

Mirror OCIChatConfig.validate_environment so embedding requests fail
fast with a clear error when oci_user/oci_fingerprint/oci_tenancy/
oci_compartment_id or an oci_key/oci_key_file is missing, instead of
deferring the failure until sign_request.

Co-authored-by: Yassin Kortam <[email protected]>

* test(oci/embed): expect OCIError from validate_environment when credentials are missing

OCIEmbedConfig.validate_environment now raises eagerly (mirroring OCIChatConfig)
when oci_user/oci_fingerprint/oci_tenancy/oci_compartment_id or oci_key/oci_key_file
is missing. Update the test to match.

* fix(oci): polish stream chunk handling and signed body default

- cohere stream terminal consolidation now emits content=None instead of ""
- drop redundant index truthiness check (None is already replaced with 0)
- accept both "TOOL_CALL" and "TOOL_CALLS" finish reasons in cohere
- signed_json_body defaults to None and uses explicit None check, so an
  explicitly empty bytes body wouldn't be silently re-serialized

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/chat): catch pydantic ValidationError when parsing OCI responses

Pydantic v2 raises ValidationError (not TypeError) when field validation
fails, so malformed OCI completion responses or stream chunks would
propagate unhandled out of handle_generic_response,
handle_generic_stream_chunk, and handle_cohere_stream_chunk. Widen the
except clauses to also catch ValidationError so callers get a clean
OCIError.

* fix(oci/catalog): real prices for Llama 4, drop zero-cost OCI OpenAI entries

Zero-cost catalog entries (input_cost_per_token=0, output_cost_per_token=0)
make proxy spend tracking silently report $0 for these paid OCI models, so
any caller can drive them without decrementing a budget.

For Llama 4 Maverick and Scout, OCI charges the same character-based rate
as Llama 3.3 70B ($0.0018 per 10,000 characters), so use the same per-token
price as the existing oci/meta.llama-3.3-70b-instruct entry (7.2e-07 in/out).

For oci/openai.gpt-5, gpt-5-mini, gpt-5-nano, gpt-oss-120b, and gpt-oss-20b,
no public per-token pricing is available; drop the entries so operators must
register them with explicit custom pricing. The existing GPT-5 reasoning test
fixture already injects synthetic entries when the catalog omits them, so the
chat transformation's supports_reasoning lookup keeps working in tests.

* fix(oci/chat): wrap CohereChatResult construction in try/except

Match the handle_generic_response pattern: surface OCIError with the
upstream status code instead of letting a raw pydantic.ValidationError
propagate when the Cohere response payload is malformed.

* fix(oci): harden Cohere stream/finish-reason and dedupe maxTokens param mapping

- Cohere stream: track per-stream tool-call emission and only suppress the
  terminal consolidation chunk's tool calls once they've been seen earlier.
  Prevents silent drop if tool calls are delivered exclusively on the
  terminal chunk.
- Cohere stream: emit content=None (not "") on non-terminal text-free
  chunks (e.g. tool-call-only / keep-alive) so downstream consumers that
  distinguish missing vs explicitly-empty deltas behave correctly.
- Generic handlers: accept singular TOOL_CALL finish reason in addition to
  TOOL_CALLS, matching the Cohere handlers.
- _get_optional_params: when both max_tokens and max_completion_tokens are
  provided, explicitly prefer max_completion_tokens instead of relying on
  dict iteration order.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): emit content=None instead of empty string for text-free generic stream chunks

Co-authored-by: Yassin Kortam <[email protected]>

* test(oci): expect content=None for text-free generic stream chunks

handle_generic_stream_chunk now emits content=None instead of empty
string when a chunk carries no text parts. Update the corresponding
no-message test to match.

* codeql: narrow OCI sha256 suppression to query-filter, not whole file

paths-ignore was suppressing every CodeQL query on
litellm/llms/oci/common_utils.py, hiding all future findings in a
security-critical file (private key loading, credential resolution,
URL construction, RSA signing). Move the suppression for
py/weak-sensitive-data-hashing into query-filters so common_utils.py
remains fully analyzed by every other query.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): use locale-independent RFC 7231 date for manual signing

email.utils.formatdate(usegmt=True) emits canonical English weekday/
month abbreviations regardless of system locale, so signature
verification doesn't break on non-en_US deployments.

* fix(oci): strip 'oci/' prefix in get_vendor_from_model

Previously, get_vendor_from_model split on '.' without stripping the
optional 'oci/' provider prefix, so 'oci/cohere.command-a-03-2025' was
routed through the GENERIC pipeline instead of COHERE.

Co-authored-by: Yassin Kortam <[email protected]>

* codeql: scope OCI sha256 suppression to common_utils.py via filter-sarif

Replace the global query-filters exclude for py/weak-sensitive-data-hashing
with a SARIF post-filter that only drops the alert when it originates from
litellm/llms/oci/common_utils.py, keeping the rule active on every other
SHA-256 callsite in the repository.

* Fix OCI chat bugs: tool_calls None key, dead max_tokens dedup, single-event stream text suppression

- handle_cohere_response: omit tool_calls key from message dict when None,
  matching the generic handler's behaviour and avoiding tripping consumers
  that key off 'tool_calls' in message.
- _get_optional_params: remove dead prefer_max_completion branch. By the
  time this helper runs, map_openai_params has already collapsed
  max_tokens/max_completion_tokens onto the OCI alias, so the OpenAI-key
  membership check is unreachable.
- handle_cohere_stream_chunk: add prior_text_emitted parameter mirroring
  prior_tool_calls_emitted. The terminal consolidation chunk's text is
  only suppressed when prior deltas already emitted text — otherwise
  (degenerate single-event stream) the text passes through so the
  response content isn't silently lost. OCIStreamWrapper now tracks
  emitted text alongside emitted tool calls.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): preserve all text parts in generic response and emit SYSTEM role for Cohere

- handle_generic_response: iterate all content parts and concatenate text
  (matches the streaming handler) so non-leading text parts are not lost
  and a leading non-text part does not suppress trailing text.
- adapt_messages_to_cohere_standard: emit CohereSystemMessage for system
  messages so direct callers do not silently drop them. The Cohere
  request builder filters system messages before calling this helper to
  avoid duplicating preambleOverride content into chatHistory.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): normalise dict-format tool_choice to OCI flat uppercase shape

The OCI Generative AI API only accepts toolChoice values of the form
{"type": "AUTO"|"NONE"|"REQUIRED"} or {"type": "FUNCTION",
"name": "<fn>"}. The previous conversion only handled string
tool_choice values, so OpenAI's standard dict shape
{"type": "function", "function": {"name": "<fn>"}} passed
through unchanged and was rejected by OCI with a 400.

Normalise the dict shape by uppercasing the discriminator and hoisting
the function name to the top level. Also accept dict variants of the
non-function selectors (e.g. {"type": "auto"}).

* test(oci): exercise system-message filtering at transform_request boundary

adapt_messages_to_cohere_standard now emits SYSTEM-role entries by design
so direct callers don't silently drop system content. The Cohere request
builder filters system messages before calling the helper and routes them
into preambleOverride, so the user-visible 'no SYSTEM in chatHistory'
guarantee holds at the transform_request boundary, where the test should
live.

* fix(oci/chat): extract tool_choice/response_format helpers to satisfy PLR0915

_get_optional_params exceeded ruff's 50-statement cap. The toolChoice and
responseFormat normalisation blocks are self-contained mutations, so move
them to module-level helpers.

* fix(oci): normalize None finishReason in generic non-streaming handler; drop dead Cohere system-role branch

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/generic): silence mypy assignment error on cleared finish_reason

* fix(docker): install libatomic in builder for prisma nodeenv binary

The prebuilt node binary that prisma-python's nodeenv downloads links
against libatomic.so.1, which Wolfi does not pull in via gcc/nodejs.
Without this, fresh Docker builds (no GHA cache hit) fail at
`prisma generate` with:
  node: error while loading shared libraries: libatomic.so.1

* fix(oci): raise on invalid tool_choice instead of silently passing OpenAI shape

_normalize_tool_choice previously left an OpenAI-format dict in selected_params['toolChoice'] when the type was unrecognized or when 'FUNCTION' was given with a missing/empty name. OCI would then reject the request with a non-obvious error. Raise ValueError with a clear message in these cases.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): raise OCIError instead of ValueError in _normalize_tool_choice

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/generic): declare non-security intent on sha256 for synthetic tool-call id

* fix(oci): simplify _get_optional_params and reject invalid tool_choice types

- Collapse the two-loop _get_optional_params into a single pass with
  clear precedence (OpenAI key wins over OCI alias; first OpenAI key
  reaching a given OCI target wins). Removes the redundant maxTokens
  special-case in the second loop and makes the map_openai_params /
  transform_request handoff easier to reason about.
- Raise OCIError when _normalize_tool_choice sees an unexpected type
  (list, bool, int, ...) instead of silently letting it through to the
  OCI API where it would produce an opaque server-side error.

Co-authored-by: Yassin Kortam <[email protected]>

* Remove no-op data['stream'] deletion in OCI stream wrappers

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): always send Cohere isStream field explicitly

Match OCIChatRequestPayload by defaulting CohereChatRequest.isStream to
False instead of None so model_dump(exclude_none=True) does not silently
omit the field on non-streaming requests.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): revert Cohere isStream to Optional[bool]=None to preserve omission semantics

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/generic): raise OCIError on empty choices instead of IndexError

Pydantic accepts an empty choices list when validating OCICompletionResponse, so accessing chatResponse.choices[0] could raise an unhandled IndexError. Surface it as OCIError so the response error path is consistent with the existing (TypeError, ValidationError) guard.

* fix(oci/cohere): map top_k -> topK so Cohere topK param is settable

The Cohere param map (derived from the GENERIC map) had no entry for
topK. Since the simplified _get_optional_params only iterates over
param_map entries, callers had no way to pass topK to CohereChatRequest
(neither via an OpenAI-style key nor via the OCI alias).

Add 'top_k': 'topK' to the Cohere map only — OCIChatRequestPayload
(GENERIC) has no topK field. _get_optional_params accepts both the
OpenAI key (top_k) and the OCI alias (topK) in optional_params, so this
covers both calling conventions.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): tighten cohere stream dedup flags and forward stream args in embed signing

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci/chat): reorder dict guard and wrap stream chunk json.loads

- Move isinstance(response_json, dict) check before .get("error") so
  the guard runs before the attribute access it is supposed to protect.
- Wrap json.loads in OCIStreamWrapper.chunk_creator with try/except so
  malformed SSE payloads surface as OCIError instead of a raw
  JSONDecodeError propagating out of the stream loop.

* fix(oci/cohere stream): only flag text emitted on non-empty content

An intermediate Cohere SSE chunk carrying text="" was flipping
_cohere_text_emitted via the "is not None" check, which then caused
the terminal consolidation chunk to drop its real text as a duplicate.
Use a truthy check so only actual content marks the stream as having
emitted text.

* test(oci): end-to-end proxy integration test against real OCI GenAI

Spins up the litellm proxy via the console-script entrypoint with a
minimal OCI-only config and drives real OpenAI-shaped HTTP requests
through it against OCI GenAI. Covers non-streaming chat, streaming
chat, embeddings, and /v1/models for Cohere, Llama, Gemini, and Grok.

Skips automatically when ~/.oci/config is absent or when the active
profile uses session-token auth (the OCI provider currently only
consumes OCI_* env vars; session tokens would need an in-process
signer). API-key profiles work out of the box.

* test(oci): move proxy integration test to tests/integration/

tests/llm_translation/ is mock-only; the OCI proxy integration test
spawns a real proxy subprocess and makes live HTTP calls, so move
it (and the companion config) to tests/integration/ alongside the
existing test_oci_integration.py.

* fix(oci): dedupe finish-reason mapping and batch Cohere tool results

- Extract _normalize_oci_finish_reason helper so the four chat handlers
  (Cohere/GENERIC, sync/stream) share one OCI->OpenAI mapping instead of
  four near-identical if/elif chains.
- Merge consecutive OpenAI tool-role messages into a single
  CohereToolMessage with multiple toolResults entries, matching the OCI
  Cohere API's expectation for parallel tool calls in one assistant turn.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(oci): drop dead Cohere toolChoice field and emit GENERIC tool-call dicts inline

- Remove the unreachable toolChoice field from CohereChatRequest. The
  Cohere param map explicitly marks tool_choice as unsupported, so the
  field can never be populated through the normal optional_params flow
  and only confused the public model surface.
- Build GENERIC stream tool-call dicts inline (id/type/function shape)
  instead of round-tripping through ChatCompletionMessageToolCall and
  model_dump(). Matches handle_cohere_stream_chunk so downstream
  stream-mergers see the same minimal payload regardless of which
  vendor produced the chunk.

Co-authored-by: Yassin Kortam <[email protected]>

* fix(docker): drop redundant libatomic from non_root builder

litellm_internal_staging already fixes the prisma `nodeenv` build
failure at the root cause by restoring `npm` to the builder (#28519):
with npm on PATH, prisma-python uses the system Node and never downloads
the nodeenv binary that links against libatomic.so.1. After merging
internal_staging the libatomic line is dead weight, so remove it.

https://claude.ai/code/session_01SwKzxRxgUhLFyyEf4UV812

* fix(oci/catalog): add openai.gpt-5{,-mini,-nano} entries with supports_reasoning

Without these catalog entries, supports_reasoning(model='openai.gpt-5*',
custom_llm_provider='oci') returned False, so _model_uses_max_completion_tokens
fell back to the default and OCI rejected the request with HTTP 400
('Use maxCompletionTokens instead.'). Add the three entries so the catalog-driven
maxCompletionTokens routing works against a stock LiteLLM install.

Also reword the test fixture docstring — the bundled backup now actually ships
these entries, so the fixture is only a fallback for environments that loaded
their cost map from a stale remote source.

---------

Co-authored-by: Tai An <[email protected]>
Co-authored-by: Vincent <[email protected]>
Co-authored-by: Kris Xia <[email protected]>
Co-authored-by: d 🔹 <[email protected]>
Co-authored-by: Fabrizio Cafolla <[email protected]>
Co-authored-by: Filippo Menghi <[email protected]>
Co-authored-by: Tom Denham <[email protected]>
Co-authored-by: escon1004 <[email protected]>
Co-authored-by: Divyansh Singhal <[email protected]>
Co-authored-by: robin-fiddler <[email protected]>
Co-authored-by: Michael-RZ-Berri <[email protected]>
Co-authored-by: Michael Riad Zaky <[email protected]>
Co-authored-by: ryan-crabbe-berri <[email protected]>
Co-authored-by: Krrish Dholakia <[email protected]>
Co-authored-by: Sameer Kankute <[email protected]>
Co-authored-by: Cursor <[email protected]>
Co-authored-by: Yassin Kortam <[email protected]>
Co-authored-by: Federico Kamelhar <[email protected]>