fix(chunking): prevent infinite loop in token-based overlap splitting - #4392

…4501)

No open issue covers this. None. Found while reading the three sibling
validators in `_validate()` against the parameters they guard.

`ChunkingOptions._validate()` at `unstructured/chunking/base.py` rejects
a negative `new_after_n_chars` and a negative `new_after_n_tokens`. It
never checks `overlap`. A negative overlap then corrupts output two
ways, with no error and no warning.

`base.py:2731` computes the remainder as `s[maxlen - self._opts.overlap
:]`. With `overlap = -5` that remainder starts five characters past the
end of the fragment, so five characters of the document disappear at
every split boundary.

`base.py:755` computes `self._text[-overlap:]`. With `overlap = -5` that
is a forward slice, so the tail carried into the next chunk is most of
the current chunk.

Before, on a 60-character input of unique characters at
`max_characters=20`:

```
$ python repro.py
overlap=   0: ['ABCDEFGHIJKLMNOPQRST', 'UVWXYZabcdefghijklmn', 'opqrstuvwxyz01234567']
           dropped from output entirely: ''
overlap=  -5: ['ABCDEFGHIJKLMNOPQRST', 'Zabcdefghijklmnopqrs', 'yz01234567']
           dropped from output entirely: 'UVWXYtuvwx'
```

The `overlap=0` line is the control: same input, same commit, nothing
lost.

With `overlap_all=True`, three 10-character elements at
`max_characters=12`:

```
overlap=  0 -> ['AAAAAAAAAA', 'BBBBBBBBBB', 'CCCCCCCCCC']
overlap= -4 -> ['AAAAAAAAAA', 'AAAAAA\n\nBBBB', 'BB', 'AA\n\nBBBBBBBB', 'CCCCCCCCCC']
```

Text duplicated and out of order.

Reachable from all three public entry points: `chunk_elements(...)`,
`chunk_by_title(...)`, and `partition(filename=...,
chunking_strategy="basic", overlap=-5)`. All three drop the same
characters.

After, the four added lines mirror the existing `new_after_n_chars`
check and raise `ValueError`.

```
$ python -m pytest test_unstructured/chunking -q
484 passed, 24 skipped in 15.04s
```

Baseline on `1bedf7b` is `482 passed, 24 skipped`, so the delta is the
two new parametrised cases. Reverting `unstructured/chunking/base.py`
and keeping the test gives `2 failed, 482 passed, 24 skipped`, both of
them `Failed: DID NOT RAISE ValueError`. `ruff check` is clean on both
touched files.

I did not run `make test` or `make check` in full, and no PDF, image or
OCR test ran here: the install was `-e ".[csv,docx,md,pptx,xlsx]"`
without the inference extras. `make check-version` passes, with
`CHANGELOG.md` and `__version__.py` both at `0.27.10`.

Duplicate check: all 130 open pull requests enumerated, 10 touching
`unstructured/chunking/`. I grepped each diff for added lines matching
`_validate|overlap.*must be|overlap_arg`. The only hit is #4487, which
caps `combine_text_under_n_chars` in `title.py::_validate`: different
file, different function, different parameter. #4392 is the nearest
neighbour and changes `_split_by_tokens` rather than validation, so the
two do not collide.

@cragwolfe you merged the last chunking change here. Would you rather
this raised, or clamped a negative overlap to zero with a warning?


<!-- This is an auto-generated description by cubic. -->
<a
href="https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4501?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->