Skip to content

chore(release): consolidate fixes and prepare Instructor 1.17.0 - #2597

Merged
jxnl merged 23 commits into
mainfrom
codex/1.17-release
Sep 9, 2026
Merged

jxnl merged 23 commits into
mainfrom
codex/1.17-release

Conversation

@jxnl

@jxnl jxnl commented Sep 5, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Prepare Instructor 1.17.0 on top of current main, including the unpublished 1.16.1 fixes and security hardening. This is a draft, not approval to publish or bypass review.

  • Preserve reusable GenAI configuration and normalize cached-content requests after configuration merging.
  • Correct GenAI retry roles and Anthropic local PDF handling.
  • Isolate response caches by client namespace, provider, and prepared generation settings; preserve validation context and strictness on cache hits.
  • Support OpenAI SDK 3 construction without taking ownership of SDK transports, retaining SDK 2/Python 3.9 compatibility.
  • Await nested async validators with explicit ordering, inheritance, cache, and unsupported-workflow contracts.
  • Consolidate documentation corrections and repair remaining malformed examples. Reconcile release metadata and upgrade notes against the last published version, 1.16.0.

Consolidated PRs And Issues

Consolidates #2557, #2577, #2578, #2579, #2581, #2582, #2583, #2584, #2586, #2587, #2588, #2591, #2593, and #2596. Original commits retain author attribution where applicable; overlapping cache/cached-content proposals are consolidated with additional regressions.

Addresses #2528, #2553, #2580, and #2585. Closing comments will link to this PR after verification. No unrelated feature request should be closed by this batch.

Validation So Far

  • Documentation syntax/lint: 610 passed, 89 existing skips. This does not claim all historical provider examples were executed.
  • Cache/v2/offline coverage/batch suites: 2,529 passed, 91 skipped. Fixed order-dependent deprecation-warning state without removing its assertion.
  • Focused live OpenAI cache tests passed earlier: sync/async namespace isolation and explicit sharing; cached async context rejection.
  • Runtime Ruff and typing passed; release metadata extraction and offline lock consistency passed.
  • SDK construction probes: OpenAI 3.0.0; OpenAI 2.48.0 on Python 3.9.
  • Initial broader run included 22 live-provider network failures under restricted network access; live matrix rerun is outstanding.

Remaining Release Gates

  • Required CI, full-repository coverage thresholds, live-provider validation, wheel/sdist and clean-install checks.
  • Complete security scan of the pinned consolidation revision and review subsequent deltas. No clean security result is claimed yet.
  • Independent review and passing exact-head checks before merge. Earlier admin exceptions for other PRs do not apply here.
  • Publish only through the release workflow after merge and verify GitHub/PyPI artifacts and installation.

Deferred Items

Model Refresh Follow-up (63660d0)

  • Updated 16 primary provider guides, including Luna, Sonnet 5, Gemini 3.8 Flash, Groq, Fireworks, Writer, Mistral, Perplexity, DeepSeek, OpenRouter, Together, Bedrock, and Vertex IDs. Added a linked provider-reference audit in docs/integrations/model-selection.md.
  • Corrected Luna tool parameters and audio handling, Sonnet adaptive thinking, Google prefix normalization and default selection, and a stale retry-count assertion. Runtime provider behavior is unchanged.
  • Added real smoke tests to CI; locally OpenAI, Anthropic, Google, Cohere, xAI, and OpenRouter passed. Groq, Mistral, Fireworks, Cerebras, Writer, Perplexity, DeepSeek, and Together skipped for missing credentials. Bedrock and Vertex IDs are documentation-verified, not live-verified.
  • Validation: 3,101 offline tests passed (199 skipped); 716 docs checks passed (89 skipped); six provider smoke tests plus Sonnet adaptive thinking passed; 12 Gemini schema tests and one Gemini retry test passed. Code/examples/tests Ruff lint and formatting, tests typing, and diff whitespace checks passed. Repository-wide Ruff also reports 36 pre-existing errors in untouched notebooks/scripts; these were not changed.
  • Old models remain in historical outputs and targeted compatibility tests. Deployment-specific Databricks, Anyscale, Ollama, and LiteLLM configuration is not certified by this update.
  • Exact-head CI must rerun. Independent review and authorized security sign-off remain outstanding; prior green checks and the interrupted security scan do not certify this head.

No release, tag, PyPI publish, merge, or tweet has been performed by this PR preparation.


Note

Low Risk
This diff is documentation, changelog, and optional live API smoke tests in CI; it does not modify core library code paths.

Overview
Prepares Instructor 1.17.0 release messaging and refreshes public docs so examples match current provider APIs.

Release notes in CHANGELOG.md add 1.17.0 (skipping unpublished 1.16.1): upgrade notes for cache namespaces, async-validator policy, and remote media fetching, plus consolidated fix/security entries and link updates.

Provider documentation updates 16 integration guides to current model IDs (e.g. gpt-5.6-luna, claude-sonnet-5, gemini-3.8-flash) and adds docs/integrations/model-selection.md as the canonical ID table with provider-specific request caveats (Luna reasoning_effort, Sonnet adaptive thinking, Fireworks full model paths). The OpenAI guide gains OpenAI-compatible base_url examples and revises audio guidance to transcribe with gpt-4o-transcribe before Luna extraction.

Concept/agent docs document 1.17 cache isolation (cache_namespace, make_request_cache_key) and fail-closed rejection of response models with async Pydantic validators; AGENT.md notes GenAI generation_config cloning and points to regression tests.

CI runs tests/llm/test_current_doc_models.py with provider API keys to smoke-test documented from_provider strings.

Blog, examples, and concepts receive broad example repairs (malformed snippets, from_provider usage, typos, hook signatures, formatting)—runtime library behavior is not changed in this diff.

Reviewed by Cursor Bugbot for commit ad6d613. Configure here.

voyagersha and others added 17 commits August 28, 2026 19:48
`pdf_to_anthropic` encoded any PDF with unset `data` by calling
`requests.get(str(pdf.source))`. That branch is only reachable for
non-HTTP sources, because remote `http(s)` URLs return earlier as a URL
document block. Every source that could reach it (a local path, `gs://`,
`s3://`) is therefore one `requests` cannot resolve, so it raised
`MissingSchema`/`InvalidSchema` rather than encoding the document.

`PDF.source` is documented as "URL, file path, or base64 data of the
PDF", and constructing `PDF(source=Path(...))` directly leaves `data`
unset. Building a PDF from a local path and sending it to Anthropic thus
always failed, even though the file was readable on disk.

Read local files from disk instead, and raise the same clear `ValueError`
that `pdf_to_openai` raises when no usable source is present. This also
removes the last raw `requests` call left in a provider multimodal
module after remote media fetching was centralized in #2569.

The existing coverage test asserted the broken behaviour by mocking
`requests.get`, and additionally failed on Windows because it compared a
`Path` against a hardcoded POSIX string. It now writes a real file to
`tmp_path`, and two regression tests cover the local-path and
missing-source cases.
Remove the runtime dependency on legacy HTTPX during automatic client
construction. Permit OpenAI 3 while preserving OpenAI 2 support on
Python 3.9, and cover both synchronous and asynchronous paths.

Refs: #2553
@async_field_validator and @async_model_validator only attached metadata
via setattr; nothing ever read it, so declared async validators were
silently never invoked and validation errors never triggered a reask.

After a non-streaming parse succeeds, retry_async_v2 now recursively runs
declared async validators (including on nested models) and raises the
existing AsyncValidationError into the reask path on failure. A sync
client given a response_model that declares async validators now raises
ConfigurationError immediately instead of silently skipping validation,
since a sync client can never await them.

Fixes #2528.
Fixes the 12 formatting-lint failures the scheduled Weekly Tests job reports
on main (black/ruff per-block via pytest_examples):

- validation.md: drop unused Field import, prefer double quotes
- distillation.md: trailing comma in RAW-format dict literal
- logging.md: remove extra blank line black collapses
- reask_validation.md: typing_extensions.Annotated -> typing.Annotated,
  extra blank line
- batch.md: drop unused BatchProcessor/extract_results imports, quote style
- hooks.md: drop unused *args, underline unused lambda args, black line
  wrapping

All 141 tests in tests/docs/test_concepts{,_advanced}.py pass locally.
The Gemini multimodal post showed its sample output as `Recomendations(`,
but the model class is defined and passed as `Recommendations`, so the
snippet did not match the code directly above it. The Gemini integration
page linked to that post as "Using Geminin To Extract Travel Video
Recomendations".

Also corrects spelling in prose and example prompts elsewhere: ambigious,
convinient, useable, recieves, targetted, succint, contrain (constrain),
Lanuage, Declaritive, occuring, probabilites, genearted, indentifying,
numberical, Janurary, Februray, concatinate, procide, duplcates, runnign,
serveral, roomate, isntance, neeed and possibily, plus two doubled words
("a a world class", "of the the signed date").
`tests/docs/test_posts.py` lints every code block under `docs/blog/posts` with
black via pytest-examples. 34 of them fail on `main`, which is why `weekly-tests`
and the `release` job's test step are both red.

This fixes 16 of the 34. Two were genuinely broken Python -- a reader copying
them gets a SyntaxError:

  bad-schemas-could-break-llms.md   " lass AnswerWithIntermediateCalculations"
                                    -> "class ..."  (missing the c)

  jinja-proposal.md                 "topic": "...7890"     <- no comma
                                    "banned_words": [...]
                                    two adjacent string literals, so the dict
                                    key after them is a syntax error

Once those parse, black can format them, so the rest of the change is what
pytest-examples' own formatter produces: joined lines that now fit, and blank
lines black adds or removes between definitions.

In jinja-proposal.md the formatter also restored `#> ` on an output comment that
had drifted to `# >`. `#> ` (with the trailing space) is pytest-examples'
`comment_prefix`; `# >` is not matched as an output marker.

Deliberately not touched: introducing-structured-outputs.md. Its block contains
a bare `#> ` marker standing for an empty printed line, and that form cannot
survive formatting -- black strips the trailing space, the result no longer
matches `comment_prefix`, and black then rewrites it to `# >`, which both breaks
the marker and leaves it inconsistent with the `#> ...` lines under it. Writing
`# >` into the docs to satisfy the linter seemed worse than leaving it, so it is
in the group below.

The remaining 18 need a call I did not want to make unilaterally, since they are
published posts:

  - empty ```python blocks where install instructions appear to have been lost
    (multimodal-gemini.md, generating-pdf-citations.md) -- the prose introduces
    a snippet that is not there
  - pseudo-code fenced as python, e.g.
    `client.create(..., response_model=type[T]) -> T` in version-1.md, which is
    a type signature rather than a statement
  - fragments that were never standalone modules, e.g. a bare `@field_validator`
    in chain-of-density.md

For those the options are to complete the snippet, re-fence as ```text (only
fences starting with `py`/`{.py` are collected), or exclude older posts from the
lint. Happy to follow up with whichever you prefer.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 5, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
instructor ad6d613 Commit Preview URL

Branch Preview URL
Sep 06 2026, 11:34 PM

@jxnl

jxnl commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator Author

Validation update at 329ddd2: local full credential-free offline suite passes 3,101 tests (199 skipped); final lazy-export regression brings local statement coverage to zero missing lines. Final-head CI is running. Prior-head release preparation, wheel smoke Python 3.9/3.13, typing, Ruff, offline matrix and provider CI passed. Security review is NOT complete: the scan worker stopped with a service cybersecurity-access restriction. An authorized security review and independent GitHub review remain required; do not merge or publish based on green tests alone.

@jxnl

jxnl commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator Author

Final validation handoff at 329ddd2: Test run 33943363939 completed successfully, including zero-missing-statement full coverage. Release readiness, Python 3.9/3.13 wheel smoke, Ruff and typing pass. Coverage clarification: Google/Gemini/GenAI, Writer and Vertex CI jobs skipped credential-dependent tests; local real GenAI checks passed 7 tests. Independent review and authorized security sign-off remain unresolved. This PR remains draft; no merge, tag or publication is authorized by these passing checks alone.

@jxnl

jxnl commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator Author

Resolved conflicts with main at 5ea35cd in merge commit 9a06499. Preserved the complete prepared-request cache identity and public-network/PDF security protections, plus local PDF support. Main’s fail-closed async-validator policy takes precedence: automatic async-validator execution is deferred; related tests, validation docs, caching docs, and changelog were updated accordingly. Validation: 3,144 offline tests passed (199 skipped); 724 documentation/release checks passed (89 skipped); three cache integration checks passed; code/examples/tests lint and formatting, runtime/tests typing, and whitespace checks passed. This is conflict resolution, not a new security audit or merge approval. Exact-head CI and independent review remain required.

@jxnl

jxnl commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator Author

Fixed the full-coverage failure in ad6d613 without changing the gate. Added six focused regression tests for schema-valued cache keys, bounded JSON nesting, socket cleanup on source-bind failure, the uppercase PDF fallback, and explicit async-validator helper error/return paths. Removed a redundant empty-address check: the existing exhausted-address failure already rejects an empty result. Local CI-equivalent run: 3,150 passed, 199 skipped; zero uncovered statements; 99.9536% branch-inclusive coverage. Tests typing, lint, and formatting pass. Fresh exact-head CI is required before merge.

@jxnl
jxnl marked this pull request as ready for review September 9, 2026 02:20
@jxnl
jxnl merged commit f539bf2 into main Sep 9, 2026
26 checks passed
@jxnl
jxnl deleted the codex/1.17-release branch September 9, 2026 02:20
@jxnl

jxnl commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator Author

Merged as f539bf2 following explicit maintainer approval to admin-merge the exact passing head ad6d613 and waive independent/security review. All checks passed before merge. The security review was waived, not completed; this comment does not claim a clean audit. Package publication and post-release verification remain separate steps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants