Comparing changes

…m elements (#4259) ## Summary Adds `create_file_from_elements()` in `unstructured.staging.base` so users can re-build a document file from a list of elements (reverse of partition). Supports the workflow: partition -> modify elements (e.g. replace Image with NarrativeText using alt text) -> write back to file. Closes #3994. ## Changes - **`unstructured/staging/base.py`**: New `create_file_from_elements(elements, format="markdown"|"html"|"text", filename=None, ...)` that delegates to `elements_to_md`, `elements_to_html`, or `elements_to_text` and optionally writes to a file. - **`test_unstructured/staging/test_base.py`**: Tests for markdown, text, and HTML output and for unsupported format raising `ValueError`.

> [!NOTE] > **Medium Risk** > Dependency upgrades (especially `transformers` major version and `weaviate-client` API shift) can introduce runtime or test regressions; CI runner change may also surface environment-specific failures. > > **Overview** > Bumps the release to `0.21.7` and updates dependency pins in `pyproject.toml`, notably moving to `wrapt` 2.x+, `transformers` 5.x, and `weaviate-client` 4.x (including constraint updates). > > Updates the Weaviate staging integration test to use the `weaviate.connect_to_embedded()` / `collections.*` API instead of the legacy `Client`/schema API. CI unit-test jobs are moved from `ubuntu-latest` to the `opensource-linux-8core` runner, and `.gitignore` now ignores `.venv*` directories. > > Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 8e26501. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).

…#4266) ## Problem `_patch_current_chars_with_render_mode` is called on every `do_TJ`/`do_Tj` text operator during PDF parsing. The original implementation re-scans the entire `cur_item._objs` list each time, checking `hasattr(item, "rendermode")` to skip already-patched items. For a page with N characters across M text operations, this is O(N*M) — effectively quadratic. Memray profiling showed this function as the #1 allocator: 17.57 GB total across 549M allocations in a session processing just 4 files. ## Fix Track the last-patched index so each call only processes newly-added `LTChar` objects. Reset automatically when `cur_item` changes (new page or figure). **Before:** O(N²) per page — re-scans all accumulated objects on every text operator **After:** O(N) per page — each object visited exactly once --------- Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com> Co-authored-by: Alan Bertl <[email protected]>

The form class currently maps to NarrativeText. This updates it to Form and adds a new class for form.

## Summary Enables partitioning of WAV audio files into document elements by transcribing with an optional speech-to-text (STT) agent, defaulting to Whisper. Closes #4029 ## Changes: - New partition_audio() and routing for FileType.WAV so partition() supports audio. - Pluggable STT layer: SpeechToTextAgent interface and SpeechToTextAgentWhisper implementation. - Optional extra audio in pyproject.toml (openai-whisper); all-docs includes audio. - Config: STT_AGENT (and STT_AGENT_MODULES_WHITELIST) for choosing the STT implementation. ## Usage pip install "unstructured[audio]" then partition("file.wav") or partition_audio("file.wav", language="en").

This checks if a pdf file is likely a complex document like mini-holistic-3-v1-Eng_Civil-Structural-Drawing_p001.pdf that is mostly vector graphics by comparing the ratio of vector images to text elements. This limits the overhead to every file by setting a minimum file size before running the check.

## Summary - Sets `fail-build: false` on the Anchore `scan-action@v3` step in the CI workflow - Critical vulnerability findings will still be reported in the scan output, but will no longer block the pipeline ## Test plan - [ ] Verify CI pipeline runs and the Anchore scan step completes without failing the build - [ ] Confirm scan results are still visible in the workflow logs 🤖 Generated with [Claude Code](https://claude.com/claude-code)  --- > [!NOTE] > **Low Risk** > Low risk workflow-only change; CI will no longer block merges on critical vulnerability findings, which reduces enforcement rather than altering runtime behavior. > > **Overview** > Updates the CI `test_dockerfile` job to set `fail-build: false` for the `anchore/scan-action@v3` container scan. > > Critical (fixed) vulnerabilities will still be reported in the scan output, but they will no longer fail the pipeline. > > Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit b01f263. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).  Co-authored-by: Claude Opus 4.6 <[email protected]>

## Summary Closes #3940 ## Changes - **Behavior:** `scarf_analytics()` sends the ping only when `UNSTRUCTURED_TELEMETRY_ENABLED=true` (or `1`). Opt-out env vars `DO_NOT_TRACK` and `SCARF_NO_ANALYTICS` are still respected and take precedence. - **Docs:** README Analytics section and logger comment updated to describe the new default and opt-in/opt-out. - **Tests:** New `DescribeScarfAnalytics` tests for default off, opt-in (`true`/`1`), and opt-out overriding opt-in. - **Changelog:** Entry under 0.21.13. --------- Co-authored-by: Lawrence Elitzer (LoLo) <[email protected]>

## Summary Closes #4006 - Adds support for custom Markdown `extensions` when calling `partition_md`, defaulting to `["tables"]` for backward compatibility. - Invalid `extensions` values log a warning and fall back to `["tables"]`. ## Motivation Fixes incorrect parsing when fenced code blocks contain `#` lines (treated as headings without `fenced_code`). ## How to use ```python from unstructured.partition.md import partition_md elements = partition_md(text=md, extensions=["fenced_code"])

> [!NOTE] > **Medium Risk** > Changes core table-chunking behavior by adding new metadata fields and reconstruction logic; risk is mainly around backward compatibility and correct ordering/HTML merging of split tables. > > **Overview** > Adds end-to-end support for reassembling split tables after chunking. `TableChunk` now receives stable sequencing metadata (`table_id`, `chunk_index`) when a `Table` is split, and a new `reconstruct_table_from_chunks()` helper in `unstructured.chunking.dispatch` groups and merges `TableChunk`s back into full `Table` elements (including merged `text_as_html` when available). > > Updates `ElementMetadata` to carry the new fields (dropped during consolidation), bumps version to `0.22.4`, and adds unit tests covering reconstruction across mixed element streams and edge cases like missing `chunk_index`. > > Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 1e732a3. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).  --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> Co-authored-by: cragwolfe <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comparing changes

Open a pull request

Commits on Feb 24, 2026

Commits on Feb 25, 2026

Commits on Feb 27, 2026

Commits on Mar 3, 2026

Commits on Mar 4, 2026

Commits on Mar 16, 2026

Commits on Mar 20, 2026

Commits on Mar 25, 2026

Commits on Mar 26, 2026

This comparison is taking too long to generate.

Uh oh!