Skip to content

[Fix] Forward client kwargs only to the openai.OpenAI constructor in experimental adapters - #1544

Open
BlueX888 wants to merge 1 commit into
guidance-ai:mainfrom
BlueX888:fix/prep-exp-adapters-double-kwargs-typeerror
Open

BlueX888 wants to merge 1 commit into
guidance-ai:mainfrom
BlueX888:fix/prep-exp-adapters-double-kwargs-typeerror

Conversation

@BlueX888

Copy link
Copy Markdown

Summary

VLLMModel and SglangModel raised TypeError at construction whenever any standard openai.OpenAI constructor argument (e.g. timeout, max_retries, organization) was passed, because **kwargs was forwarded to both openai.OpenAI(...) and BaseOpenAIInterpreter.__init__(...). The interpreter base class only accepts model, client and reasoning_effort, so every client kwarg crashed with TypeError: BaseOpenAIInterpreter.__init__() got an unexpected keyword argument 'timeout'. Conversely, reasoning_effort, which the interpreter does accept, was rejected by openai.OpenAI() instead.

Why

The unpatched constructors in guidance/models/experimental/_vllm.py:27 and guidance/models/experimental/_sglang.py:27 did:

client = openai.OpenAI(base_url=base_url, api_key=api_key, **kwargs)
super().__init__(model=model, client=OpenAIClientWrapper(client), **kwargs)

BaseOpenAIInterpreter.__init__ (guidance/models/_openai_base.py:258) has signature (self, model, client, *, reasoning_effort=None) and accepts no other keyword arguments. The sibling OpenAIInterpreter already handles this correctly (guidance/models/_openai.py:30-31): it forwards **kwargs only to openai.OpenAI(...) and passes reasoning_effort explicitly to super().__init__, and its docstring documents that "All extra keyword arguments are passed directly to the openai.OpenAI constructor" (guidance/models/_openai.py:56-58).

Changes

  • guidance/models/experimental/_vllm.py, guidance/models/experimental/_sglang.py: add an explicit reasoning_effort: str | None = None parameter, forward **kwargs only to openai.OpenAI(...), and pass only reasoning_effort to super().__init__, mirroring OpenAIInterpreter.
  • tests/unit/test_experimental_models.py: new regression tests asserting that client kwargs (timeout, max_retries) reach the openai.OpenAI client and that reasoning_effort reaches the interpreter, for both models.

Testing

python -m pytest tests/unit/test_experimental_models.py -v
# unpatched: 4 failed
#   TypeError: BaseOpenAIInterpreter.__init__() got an unexpected keyword argument 'timeout'
#   TypeError: OpenAI.__init__() got an unexpected keyword argument 'reasoning_effort'
# patched: 4 passed

python -m pytest tests/unit/
# 1641 passed, 1 skipped, 220 xfailed in 15.36s

ruff format --check            # 151 files already formatted
ruff check --select I,RUF022   # All checks passed!
ruff check                     # All checks passed!
mypy guidance                  # 125 pre-existing errors before and after (no new errors introduced)

…experimental adapters

VLLMInterpreter.__init__ and SglangInterpreter.__init__ forwarded **kwargs
to both openai.OpenAI(...) and BaseOpenAIInterpreter.__init__(...), which
does not accept arbitrary keyword arguments. Any standard openai.OpenAI
constructor argument (e.g. timeout, max_retries, organization) therefore
raised 'TypeError: BaseOpenAIInterpreter.__init__() got an unexpected
keyword argument' at construction, while reasoning_effort, which the
interpreter does accept, was rejected by openai.OpenAI() instead.

Forward **kwargs only to openai.OpenAI and pass reasoning_effort
explicitly to super(), matching OpenAIInterpreter.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant