Skip to content

feat(proxy): show real upstream provider on dashboard for OpenAI-compatible endpoints - #1594

Merged
JerrettDavis merged 2 commits into
headroomlabs-ai:mainfrom
tenderdeve:feat/1533-upstream-provider-classify
Jul 15, 2026
Merged

JerrettDavis merged 2 commits into
headroomlabs-ai:mainfrom
tenderdeve:feat/1533-upstream-provider-classify

Conversation

@tenderdeve

@tenderdeve tenderdeve commented Jun 30, 2026 •

Copy link
Copy Markdown
Contributor

Description

When the proxy runs against a custom OpenAI-compatible endpoint via --openai-api-url (OpenRouter, Groq, Together, Azure OpenAI, …), the dashboard always showed the provider as OpenAI, because the OpenAI handler records every request with provider="openai".

This detects well-known upstreams from the --openai-api-url host and adds a --provider-name override that takes precedence (the issue's option 3). The label is resolved only where the dashboard/stats payload is built — the internal provider key stays openai, so pricing and request formatting are unaffected.

Upstream URL Provider shown
https://api.openai.com/v1 OpenAI
https://openrouter.ai/api/v1 OpenRouter
https://api.groq.com/openai/v1 Groq
https://api.together.xyz/v1 Together AI
https://<resource>.openai.azure.com/ Azure OpenAI

Unknown hosts keep the openai label unless --provider-name is set.

Closes #1533

Type of Change

  • New feature (non-breaking change that adds functionality)

Changes Made

  • helpers.py: classify_openai_upstream() (host → display name) + resolve_display_provider() (precedence: --provider-name > host detection > raw provider; only relabels openai).
  • models.py: ProxyConfig.provider_name.
  • cli/proxy.py: --provider-name flag, threaded into ProxyConfig.
  • server.py: relabel at the four dashboard/stats display sites (recent requests, transformations feed, requests.by_provider, agent-usage breakdown) via the resolver / _remap_provider_counts. Stored logs and metrics keys are untouched.
  • docs/content/docs/proxy.mdx: document --provider-name.

Testing

  • Unit tests pass (pytest)
  • Linting passes (ruff check)
  • New tests added

Test Output

$ pytest tests/test_provider_display_classification.py tests/test_dashboard_agent_usage.py -q
16 passed
13 passed

$ ruff check headroom/proxy/helpers.py headroom/proxy/server.py headroom/proxy/models.py headroom/cli/proxy.py
All checks passed!

Real Behavior Proof

  • Environment: repo branch feat/1533-upstream-provider-classify @ HEAD, local .venv (Python 3)
  • Exact command / steps: ran the helpers directly from the venv — python -c "from headroom.proxy.helpers import classify_openai_upstream, resolve_display_provider; print(classify_openai_upstream('https://openrouter.ai/api/v1')); print(resolve_display_provider('openai', openai_api_url='https://openrouter.ai/api/v1')); print(resolve_display_provider('openai', openai_api_url='https://openrouter.ai/api/v1', provider_name='Groq')); print(resolve_display_provider('anthropic'))"
  • Observed result: host detection relabels openai → OpenRouter, --provider-name overrides detection (Groq), and the anthropic label (plus the openai pricing key) is unchanged. Full output below:
    classify openrouter           -> OpenRouter
    resolve openai+openrouter url -> OpenRouter
    override provider-name        -> Groq
    anthropic untouched           -> anthropic
    
  • Not tested: live dashboard render against a real OpenRouter key (the payload-builder logic is covered by the unit tests above).

Review Readiness

  • I have performed a self-review
  • This PR is ready for human review

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective

…atible endpoints

When the proxy runs against a custom OpenAI-compatible endpoint via
--openai-api-url (OpenRouter, Groq, Together, Azure OpenAI, …), the dashboard
always labelled traffic as 'openai' because the OpenAI handler records every
request with provider='openai'.

Detect well-known upstreams from the configured --openai-api-url host and add
a --provider-name override that takes precedence. The label is resolved only
where the dashboard/stats payload is built; the internal provider key stays
'openai' so pricing and request formatting are unchanged.

Closes headroomlabs-ai#1533
@github-actions

github-actions Bot commented Jun 30, 2026 •

Copy link
Copy Markdown
Contributor

PR governance

This PR follows the template and is marked ready for human review.

@github-actions github-actions Bot added status: needs author action Pull request body or readiness checklist still needs author updates status: ready for review Pull request body is complete and the author marked it ready for human review and removed status: needs author action Pull request body or readiness checklist still needs author updates labels Jun 30, 2026

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provider-label change is scoped to display/stat payloads: internal provider keys remain openai, while dashboard/recent-request/count presentation can resolve --provider-name or known OpenAI-compatible hosts such as OpenRouter/Groq/Together/Azure. That is the right boundary for preserving pricing and request formatting behavior.

I attempted python -m pytest tests/test_provider_display_classification.py -q, but this review worktree cannot collect full proxy imports because headroom._core is not built locally. From direct diff review, I do not see a remaining code-review blocker.

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved current head cfcdf43 after merging current headroomlabs/main and resolving the /stats recent-request payload conflict by preserving current-main token-accounting fields while applying the PR's display-provider remapping.

Validated locally:

  • uv run --extra dev python -m pytest tests/test_provider_display_classification.py tests/test_dashboard_agent_usage.py tests/test_proxy_stats_recent_requests.py -q
  • uvx [email protected] check headroom/proxy/helpers.py headroom/proxy/server.py headroom/proxy/models.py headroom/cli/proxy.py tests/test_provider_display_classification.py tests/test_dashboard_agent_usage.py tests/test_proxy_stats_recent_requests.py
  • uvx [email protected] format --check headroom/proxy/helpers.py headroom/proxy/server.py headroom/proxy/models.py headroom/cli/proxy.py tests/test_provider_display_classification.py tests/test_dashboard_agent_usage.py tests/test_proxy_stats_recent_requests.py
  • uvx [email protected] headroom/proxy/helpers.py headroom/proxy/server.py headroom/proxy/models.py headroom/cli/proxy.py --ignore-missing-imports
  • git diff --check

@JerrettDavis
JerrettDavis merged commit 996c117 into headroomlabs-ai:main Jul 15, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: ready for review Pull request body is complete and the author marked it ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Support custom provider classification when using --openai-api-url

2 participants