Skip to content

Raise on model responses without text - #534

Merged
aksg87 merged 1 commit into
mainfrom
aksg87/raise-on-empty-model-responses
Sep 7, 2026
Merged

aksg87 merged 1 commit into
mainfrom
aksg87/raise-on-empty-model-responses

Conversation

@aksg87

@aksg87 aksg87 commented Aug 27, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Fixes #491. Fixes #508.

Gemini and OpenAI realtime inference now raise InferenceRuntimeError when blocking, refusal, or an exhausted output budget produces no text. Provider-local adapters use typed SDK responses and preserve diagnostics through retries and parallel inference.

Empty-string completions that stop normally retain existing resolver handling. Batch changes are out of scope; Gemini batch is tracked by #527.

Builds on #496, #507 and #516; thanks to @malai001, @mittalpk and @uuzzrm.

Tests

  • tox -p auto -e py310,py311,py313,format,lint-src,lint-tests: 828 unit tests pass per Python version; formatting and lint pass.
  • pytest tests/gemini_retry_test.py tests/provider_schema_test.py tests/test_kwargs_passthrough.py tests/inference_test.py -q: 214 tests pass with minimum SDK versions.
  • Live API and integration checks run in PR CI.

@github-actions github-actions Bot added the size/M Pull request with 150-600 lines changed label Aug 27, 2026
@aksg87
aksg87 force-pushed the aksg87/raise-on-empty-model-responses branch from 9242bc8 to 404163e Compare September 6, 2026 05:24
@aksg87 aksg87 changed the title Raise on model responses with no text content instead of empty success Raise on model responses without text Sep 6, 2026
@aksg87
aksg87 force-pushed the aksg87/raise-on-empty-model-responses branch 3 times, most recently from 6ef6caf to 412bb47 Compare September 7, 2026 04:59
@github-actions github-actions Bot added size/L Pull request with 600-1000 lines changed and removed size/M Pull request with 150-600 lines changed labels Sep 7, 2026
@aksg87
aksg87 force-pushed the aksg87/raise-on-empty-model-responses branch from 412bb47 to 1704fe4 Compare September 7, 2026 05:21
@aksg87
aksg87 force-pushed the aksg87/raise-on-empty-model-responses branch from 1704fe4 to 642c50d Compare September 7, 2026 05:32
@aksg87
aksg87 merged commit bd5d1eb into main Sep 7, 2026
20 of 21 checks passed
ojassharma7 added a commit to ojassharma7/langextract that referenced this pull request Sep 11, 2026
…ogle#534

Rebasing onto main surfaced a behavioural conflict that merged cleanly as
text. google#534 added an `except InferenceRuntimeError: raise` pass-through to the
parallel result loop, which returns the provider's own diagnostic unwrapped.
That pass-through runs before the draining logic here, so once
_process_single_prompt started wrapping failures as InferenceRuntimeError,
every parallel failure short-circuited and the completed chunks were discarded
again, which is the exact behaviour this branch set out to fix.

Failures are now collected uniformly and the pool is allowed to drain. Which
error is raised then depends on whether there is anything to preserve:

- Nothing completed: re-raise the provider's own InferenceRuntimeError
  unchanged, so a blocked prompt or an exhausted token budget still reports its
  own reason rather than being buried a level down, with the drained
  bookkeeping attached to it.
- Some chunks completed: raise the aggregate error carrying partial_results and
  failed_indices, with the first underlying error kept as `original` and quoted
  in the message.

Both test suites pass together: the no-text tests added by google#534 and the
partial-result tests on this branch.
@aksg87 aksg87 mentioned this pull request Sep 13, 2026
ojassharma7 added a commit to ojassharma7/langextract that referenced this pull request Sep 16, 2026
…ogle#534

Rebasing onto main surfaced a behavioural conflict that merged cleanly as
text. google#534 added an `except InferenceRuntimeError: raise` pass-through to the
parallel result loop, which returns the provider's own diagnostic unwrapped.
That pass-through runs before the draining logic here, so once
_process_single_prompt started wrapping failures as InferenceRuntimeError,
every parallel failure short-circuited and the completed chunks were discarded
again, which is the exact behaviour this branch set out to fix.

Failures are now collected uniformly and the pool is allowed to drain. Which
error is raised then depends on whether there is anything to preserve:

- Nothing completed: re-raise the provider's own InferenceRuntimeError
  unchanged, so a blocked prompt or an exhausted token budget still reports its
  own reason rather than being buried a level down, with the drained
  bookkeeping attached to it.
- Some chunks completed: raise the aggregate error carrying partial_results and
  failed_indices, with the first underlying error kept as `original` and quoted
  in the message.

Both test suites pass together: the no-text tests added by google#534 and the
partial-result tests on this branch.
@aksg87
aksg87 deleted the aksg87/raise-on-empty-model-responses branch October 11, 2026 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L Pull request with 600-1000 lines changed

Projects

None yet

1 participant