Provider Relay

An OpenAI-compatible provider relay. It forwards chat requests, supports JSON and Server-Sent Events (SSE) streaming, stores complete interactions, and can use a second provider or a manually supplied response when the primary provider fails.

Features

• POST /v1/chat/completions with normal and streaming (SSE) responses, including tool calls
• POST /v1/responses modern OpenAI Responses API (non-streaming, converts to chat completions upstream)
• POST /v1/embeddings and POST /v1/completions (legacy)
• GET /v1/models with live upstream model discovery
• OpenAI-compatible Bearer authentication and error responses
• x-request-id on responses
• CORS enabled for browser clients
• Automatic primary and optional fallback provider attempts
• Provider client errors (4xx) forwarded immediately; server/network errors trigger fallback/manual response
• Manual response via dashboard or admin API; rejected once the client request has disconnected or timed out
• Complete interaction records in data/interactions/
• Raw request bytes in data/raw_requests/
• Application events in logs/application.jsonl
• Browser dashboard with separate “Messages received” and “Response returned” panels, including tool calls
• Swagger UI and ReDoc
• Docker deployment with persistent storage

Configuration

Copy .env.example to .env and set:

RELAY_API_KEY=the-key-your-agent-uses
PRIMARY_BASE_URL=https://api.hormouz.net/v1
PRIMARY_API_KEY=your-provider-key
DEFAULT_MODEL=openai/gpt-4o-mini
SUPPORTED_MODELS=openai/gpt-4o-mini

Optional fallback settings:

FALLBACK_BASE_URL=https://backup-provider.example/v1
FALLBACK_API_KEY=backup-provider-key

SUPPORTED_MODELS is a comma-separated fallback list. When the primary provider is available, /v1/models returns its live model catalog instead.

Logging

Application events and exceptions are written as structured JSON lines to LOG_DIR/application.jsonl and emitted to the container console. Each request includes its request ID, method, path, status, and duration. Provider, storage, streaming, authentication, validation, and unexpected exceptions include structured context; unexpected exceptions include tracebacks.

The file log rotates at LOG_MAX_BYTES (default 50 MiB) and keeps LOG_BACKUP_COUNT rotated files (default 5). Interaction and raw-request file operations use a bounded background queue configured by STORAGE_QUEUE_SIZE (default 256), so synchronous filesystem work does not block the async request loop.

Secrets such as API keys, authorization headers, cookies, and passwords are redacted automatically. Request and response content is not added to application logs by default.

Docker

copy .env.example .env
docker compose up --build -d

The container is named relay and exposes port 8001 on the host:

http://localhost:8001

Interaction data survives container restarts through the ./data volume. Logs survive through ./logs.

Stop the service:

docker compose down

View runtime logs:

docker compose logs -f relay

Browser URLs

Swagger UI:  http://localhost:8001/docs
OpenAPI:     http://localhost:8001/openapi.json
ReDoc:       http://localhost:8001/redoc
Dashboard:   http://localhost:8001/dashboard
Health:      http://localhost:8001/health

The dashboard refreshes automatically and lets you click the eye/view action for any interaction. The detail view shows the original messages, the exact request JSON, the assistant response, the exact response JSON, any provider error details, and a manual-response form for interactions that have not yet completed.

OpenAPI endpoints are grouped by tag (Chat, Models, Embeddings, Completions, Admin, Dashboard, Meta).

Chat Completions API

POST /v1/chat/completions is the main endpoint. It accepts every common OpenAI chat parameter and returns OpenAI-shaped responses.

Supported request fields

model
messages
stream
stream_options

temperature
top_p
n
max_tokens
max_completion_tokens
stop

presence_penalty
frequency_penalty
logit_bias
logprobs
top_logprobs

seed
user
metadata

response_format

service_tier
reasoning_effort
modalities
prediction
store
audio
web_search_options

tools
tool_choice
parallel_tool_calls

Any extra field is forwarded as-is (extra="allow"), so provider-specific parameters pass through.

Response types

The relay returns exactly what the upstream provider returns:

normal JSON completion
multiple choices when n > 1
tool_calls[] in the assistant message
streaming SSE (text and tool-call deltas)
final usage chunk when stream_options.include_usage is true
OpenAI-compatible error objects

curl examples

Set your relay key and base URL:

RELAY_KEY="the-key-your-agent-uses"
BASE_URL="http://localhost:8001/v1"

Basic completion:

curl -s "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $RELAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello"}]
  }' | jq .

Streaming:

curl -s "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $RELAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "Count to 3"}],
    "stream": true
  }'

Multiple choices (n > 1):

curl -s "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $RELAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "Say hi"}],
    "n": 2
  }' | jq '.choices | length'

Tool calling:

curl -s "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $RELAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "What is the weather in Paris?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get weather",
        "parameters": {
          "type": "object",
          "properties": {"location": {"type": "string"}},
          "required": ["location"]
        }
      }
    }],
    "tool_choice": "auto",
    "parallel_tool_calls": true
  }' | jq '.choices[0].message.tool_calls'

Structured output (JSON mode):

curl -s "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $RELAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "Return JSON with answer equal to hello."}],
    "response_format": {"type": "json_object"}
  }' | jq '.choices[0].message.content'

Error example (invalid model):

curl -s -w "\nHTTP %{http_code}\n" "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $RELAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "this-model-does-not-exist",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Responses API

POST /v1/responses exposes OpenAI's modern Responses API. The relay transforms the request into a chat completion call upstream and converts the response back into a Responses API object, so it works with providers that only implement /v1/chat/completions.

Supported fields:

model
input (string or array of message/function_call/function_call_output items)
instructions
tools
tool_choice
parallel_tool_calls
max_output_tokens
reasoning
temperature
top_p
stop
n
seed
user
store
metadata

Streaming is not yet supported for the Responses API.

curl example

curl -s "$BASE_URL/responses" \
  -H "Authorization: Bearer $RELAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "input": "Summarize latest developments in artificial intelligence."
  }' | jq .

Expected response shape:

{
  "id": "resp_...",
  "object": "response",
  "status": "completed",
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [{"type": "output_text", "text": "..."}]
    }
  ],
  "usage": {...}
}

When the model returns tool calls, the output contains function_call items with call_id, name, and arguments.

Agent Usage

Python OpenAI client:

from openai import OpenAI

client = OpenAI(
    api_key="the-key-your-agent-uses",
    base_url="http://localhost:8001/v1",
)

models = client.models.list()
print([model.id for model in models.data])

answer = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)
print(answer.choices[0].message.content)

Streaming:

stream = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Tell me a short joke"}],
    stream=True,
)

for chunk in stream:
    text = chunk.choices[0].delta.content
    if text:
        print(text, end="", flush=True)

End-to-End Test

The test script loads .env, prints every model returned by /v1/models, and sends a real request through the relay:

python scripts/test_api.py --model openai/gpt-4o-mini

Test streaming:

python scripts/test_api.py --model openai/gpt-4o-mini --stream

The script prints each SSE line for streaming tests and prints the complete JSON response for non-streaming tests.

Run the five-case compatibility exercise:

python scripts/exercise_api.py

This sends content-part, tool-history, structured-output, sampling/metadata, and streaming-with-usage requests. Every request is stored as a complete interaction.

Manual Fallback

List captured interactions:

curl http://localhost:8001/admin/interactions

Submit assistant text for an interaction waiting for manual fallback:

curl -X POST -H "Content-Type: application/json" -d '{"content":"Manual answer"}' http://localhost:8001/admin/interactions/INTERACTION_ID/response

For non-streaming requests, the original request waits for MANUAL_RESPONSE_TIMEOUT_SECONDS. For streaming requests, manual fallback is possible only before any provider chunk has reached the client. A partially sent stream cannot be replaced with a different complete answer.

You can also submit a manual response directly in the dashboard. The form is hidden once an interaction is completed, and the API returns 409 Conflict if you try to overwrite a completed response.

Deployment

For a public server, place HTTPS and your chosen domain or subdomain in front of port 8001 using Caddy, Nginx, Traefik, or a hosting provider load balancer. Set the agent’s base_url to the resulting /v1 URL.

The /v1 API requires Authorization: Bearer <RELAY_API_KEY>. Dashboard and /admin/* endpoints are open by default. To protect them when exposing the service publicly, set DASHBOARD_PASSWORD in your .env and log in with username admin and that password (HTTP Basic Auth).