fix: skip max_completion_tokens when maxTokens is None#3284
fix: skip max_completion_tokens when maxTokens is None#3284jlowin merged 1 commit intoPrefectHQ:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes a bug in the OpenAI sampling handler where sending an explicit null value for max_completion_tokens causes the API to return empty responses. The fix ensures that when maxTokens is not specified (None), the parameter is omitted from the API request entirely, which is the expected behavior.
Changes:
- Modified the OpenAI sampling handler to conditionally include
max_completion_tokensonly whenparams.maxTokens is not None, making it consistent with howtemperatureandstopSequencesare handled.
Test Failure AnalysisSummary: One test failed on Python 3.13 in the "Run unit tests" step: Root Cause: The test is timing-sensitive and incorrectly calibrated for Python 3.13. It configures Suggested Solution: Fix
This failure is unrelated to the Detailed AnalysisFailing test ( Log excerpt:
Token bucket logic ( With Only fails on: Python 3.13 / ubuntu-latest Related Files
|
Description
When
maxTokensis not specified by the caller, the OpenAI sampling handler was sending"max_completion_tokens": nullexplicitly in the API request. OpenAI treats an explicitnulldifferently from an omitted key, it returns an empty response withfinish_reason: "stop"and no content, which causes the handler to raiseValueError("No content in response from completion").Fix: guard
max_completion_tokenswithif params.maxTokens is not None, consistent with howtemperatureandstopSequencesare already handled in the same block.This is a follow-up to the rename fix in #3252.
Contributors Checklist
Review Checklist