Check for existing issues
What happened?
When using Claude models on Vertex AI with a non-us-central1 location (e.g., europe-west1), the count_tokens endpoint always routes to us-central1 regardless of
vertex_location or vertex_count_tokens_location config. This causes 500 errors if the project doesn't have Claude count-tokens available in us-central1.
A workaround was added in common_utils.py (line 1064) to support vertex_count_tokens_location, but it is silently nullified by handler.py further down the call chain.
Root Cause
The fix in common_utils.py correctly resolves vertex_count_tokens_location:
# common_utils.py:1064
vertex_location = (
count_tokens_params_request.get("vertex_count_tokens_location")
or vertex_location
)
# → correctly resolves to "europe-west1" ✅
This value is passed to VertexAIPartnerModels.count_tokens() as vertex_location="europe-west1", which copies it into _litellm_params["vertex_location"] and calls
handle_count_tokens_request().
However, inside handler.py, the location is re-read from litellm_params and then unconditionally overridden because "claude" is always present in the model name:
# handler.py:108-113 — BROKEN
vertex_location = self.get_vertex_ai_location(litellm_params)
# → reads "europe-west1"
if not vertex_location or "claude" in model.lower():
vertex_location = "us-central1"
# → "claude" is ALWAYS in Claude model names → always overrides to "us-central1" ❌
# → vertex_count_tokens_location is never consulted here
---
Expected Behavior
vertex_count_tokens_location should take precedence over the us-central1 default. The default override should only apply when no explicit count-tokens location is configured.
---
Proposed Fix
In litellm/llms/vertex_ai/vertex_ai_partner_models/count_tokens/handler.py, change lines 108–113:
# Current (broken):
vertex_location = self.get_vertex_ai_location(litellm_params)
if not vertex_location or "claude" in model.lower():
vertex_location = "us-central1"
# Fixed:
vertex_location = (
litellm_params.get("vertex_count_tokens_location")
or self.get_vertex_ai_location(litellm_params)
)
if not vertex_location or "claude" in model.lower():
vertex_location = "us-central1"
This ensures vertex_count_tokens_location is checked first. If set, the us-central1 override is bypassed. If not set and the model is Claude, it falls back to us-central1 as before.
---
Steps to Reproduce
1. Configure a Claude model on Vertex AI with vertex_location: europe-west1 and vertex_count_tokens_location: europe-west1
2. Make a request that triggers token counting
3. Observe: request routes to us-central1 instead of europe-west1 → 500 error
LiteLLM config:
model_list:
- model_name: claude-sonnet-4-6
litellm_params:
model: vertex_ai/claude-sonnet-4-6
vertex_project: my-gcp-project
vertex_location: europe-west1
vertex_count_tokens_location: europe-west1 # ignored due to bug
common_utils.py:1064 → resolves vertex_count_tokens_location = "europe-west1" ✅
main.py:323-327 → copies into _litellm_params["vertex_location"] = "europe-west1" ✅
handler.py:108 → re-reads vertex_location from litellm_params = "europe-west1"
handler.py:112 → "claude" in model → overrides to "us-central1" ❌
handler.py:123-127 → builds URL with us-central1 → 500
### Relevant log output
```shell
litellm.proxy.anthropic_endpoints.count_tokens(): Exception occurred -
Server error '500 Internal Server Error' for url
'https://us-central1-aiplatform.googleapis.com/v1/projects/getmoss-ai/locations/us-central1/publishers/anthropic/models/count-tokens:rawPredict'
What part of LiteLLM is this about?
Proxy
What LiteLLM version are you on ?
1.81.14
Twitter / LinkedIn details
https://www.linkedin.com/in/akanksha-singh-2989aa164/
Check for existing issues
What happened?
When using Claude models on Vertex AI with a non-
us-central1location (e.g.,europe-west1), thecount_tokensendpoint always routes tous-central1regardless ofvertex_locationorvertex_count_tokens_locationconfig. This causes 500 errors if the project doesn't have Claude count-tokens available inus-central1.A workaround was added in
common_utils.py(line 1064) to supportvertex_count_tokens_location, but it is silently nullified byhandler.pyfurther down the call chain.Root Cause
The fix in
common_utils.pycorrectly resolvesvertex_count_tokens_location:model_list:
- model_name: claude-sonnet-4-6
litellm_params:
model: vertex_ai/claude-sonnet-4-6
vertex_project: my-gcp-project
vertex_location: europe-west1
vertex_count_tokens_location: europe-west1 # ignored due to bug
common_utils.py:1064 → resolves vertex_count_tokens_location = "europe-west1" ✅
main.py:323-327 → copies into _litellm_params["vertex_location"] = "europe-west1" ✅
handler.py:108 → re-reads vertex_location from litellm_params = "europe-west1"
handler.py:112 → "claude" in model → overrides to "us-central1" ❌
handler.py:123-127 → builds URL with us-central1 → 500
What part of LiteLLM is this about?
Proxy
What LiteLLM version are you on ?
1.81.14
Twitter / LinkedIn details
https://www.linkedin.com/in/akanksha-singh-2989aa164/