Repository navigation
Description
Summary
Add a recipe documenting Oracle Cloud Infrastructure (OCI) Generative AI as a Switchyard backend. OCI exposes an OpenAI-compatible endpoint (https://inference.generativeai.<region>.oci.oraclecloud.com/openai/v1, Chat Completions and Responses) authenticated with a compartment-scoped Generative AI API key sent as a bearer token, so the released v0.3.0 server and Python bindings work with no code change. The repository currently has no OCI mention; this closes that documentation gap the same way the single-provider coding-agent recipe does for OpenAI and Anthropic logins.
What the recipe covers
- Endpoint, model ID form (
vendor.model, e.g.meta.llama-4-scout-17b-16e-instruct,xai.grok-4.7,openai.gpt-5.5), and how to list the regional catalog (the endpoint does not serveGET /models). - Credential setup in the order that works: IAM policy first, then the API key (a key created before its policy can stay unauthorized; OCI returns the same 401 for unknown and unauthorized keys).
- A
llm_classifiercapability route and astage_routerroute over OCI targets, with two clients:openai_chatfor Meta/xAI/Google models andopenai_responsesfor OpenAIgpt-5.x/gpt-6.x, which rejectmax_tokenson Chat Completions. - The Python host path with the released bindings.
- Known limits: 404 for catalog models not served on demand, Cohere embeddings unsupported, judge
timeout_ms, cost table needs an override for OCI IDs, dedicated clusters and signed OCI principals need a signing gateway in front.
Verification
switchyard-server --config oci-routing.toml --dry-runon v0.3.0 accepts the configuration (oci-task,oci-execution, plus two passthrough baselines).- Live runs against
us-chicago-1with the judge and answers served by OCI are being recorded now and will be attached to the PR before it leaves draft.
Proposed placement
docs/recipes/oci_generative_ai.md, linked in mkdocs.yml under Integrate Switchyard → Model Providers (a new group next to Gateways). Happy to move it if you prefer it under Local Demo with the other recipe.
Opening this first because the page is about 140 lines, per CONTRIBUTING's guidance for larger changes. A follow-up, separate from this one, would address #665 (serving judge calls in the LiteLLM plugin), which I have working against OCI.
Activity