Skip to content

fix(adapters): read bfloat16 adapters in adapters check and adapters diff (#2058) - #2063

Merged
MakazhanAlpamys merged 3 commits into
MakazhanAlpamys:mainfrom
drakeo338:claude/2058-fix
Oct 23, 2026
Merged

MakazhanAlpamys merged 3 commits into
MakazhanAlpamys:mainfrom
drakeo338:claude/2058-fix

Conversation

@drakeo338

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes #2058.

soup adapters check and soup adapters diff read weights with safe_open(framework="numpy"), and numpy has no bfloat16 dtype. A bf16 adapter therefore failed with data type 'bfloat16' not understood, reported as "Corrupt or unreadable" by check. Both commands now share one reader that parses the safetensors header and widens bfloat16 to float32 exactly. The size cap, tensor-count cap and symlink checks are unchanged.

Type of change

  • Bug fix
  • New feature
  • Refactor / cleanup
  • Documentation
  • Tests

Checklist

  • ruff check src/soup_cli/ scripts/ tests/ passes (ran on the three touched files only)
  • pytest tests/ -v passes (ran only the adapter tests: 37 passed; the new tests fail on the old source)
  • Updated relevant docs (README.md and the matching page under docs/) if needed (no docs change)
  • Added a changelog fragment, or this change has no user-visible impact
  • My contribution is licensed under Apache-2.0, the project's own terms (see CONTRIBUTING.md)

…rs diff` (MakazhanAlpamys#2058)

Both commands loaded the weights with safe_open(framework="numpy"), which cannot
return bfloat16 because numpy has no such dtype. An adapter saved from a bf16 run
therefore failed with "data type 'bfloat16' not understood" (diff) or "Corrupt or
unreadable safetensors weights" (check).

They now share one reader, _read_safetensors_numpy in utils/adapter_diff.py. It
parses the header with read_header (offsets and sizes validated, no mapping) and
reads the payload directly; a bfloat16 tensor is widened to float32 exactly, as
the top 16 bits of a float32. Other dtypes are read as before. The size cap in
adapters check, the tensor-count cap (now checked before any payload is read) and
the symlink checks are unchanged.
@MakazhanAlpamys

Copy link
Copy Markdown
Owner

Thanks, @drakeo338, #2063 follows option 1 of #2058 closely: one shared header-based reader, BF16 widened exactly, the size and tensor caps kept, and tests for check, diff, truncation and the cap. One process note: #2058 carries area:cli, and its maintainer (@Nari-Ab, #1936) has a first refusal until 2026-10-22 19:40Z; the pull request arrived without a claim comment, so review waits for that window or their word. Nothing to change meanwhile. It also removes the safetensors import, which overlaps #2056; a line about that on #2056 would help whoever takes it.

@MakazhanAlpamys MakazhanAlpamys left a comment •

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Process note first, to square this with my earlier comment: the merge still waits for the area:cli first-refusal window on #2058 (until 2026-10-22 19:40Z) or @Nari-Ab's word. I reviewed the code now so that the change below can be made in the meantime.

Thanks for this, @drakeo338. It is a clean fix and it follows option 1 of #2058: one shared reader that parses the safetensors header, widens bfloat16 to float32 exactly, and keeps the size cap, the tensor-count cap and the symlink checks. I checked it against the branch:

  • The widening is exact. I wrote all 65536 bfloat16 bit patterns with torch and compared: the float32 values are bitwise identical to tensor.float(), including NaN, infinity and negative zero.
  • The other dtypes did not move: f16, f32, f64, i8..i64, u8, bool, a scalar and an empty tensor come back with the same dtype, shape and values as safe_open(framework="numpy").
  • On real torch-saved bf16 adapters, soup adapters check prints alive (exit 0) with a norm within bf16 rounding of the float32 copy, a zero lora_B bf16 adapter gives inactive: all lora_B layers are zero (1/1) (exit 2), and soup adapters diff works. Merge and scan also read bf16 and float64 adapters now, since they share the loader.
  • Mutation checks: reverting the two source files fails 5 of the new tests, changing the shift from 16 to 15 fails 4, and turning off the tensor-count cap fails the cap test. test_issue1721_adapters_check.py passes (also with FORCE_COLOR=1), as do the adapter diff, merge and scan test files and ruff.

One change is needed before this can merge. Main has moved since your base: #2061 added the FP8 spellings (F8_E4M3, F8_E5M2, ...) to safetensors_reader._DTYPES. With that change, _read_safetensors_numpy does _NUMPY_DTYPES[entry.dtype] on a dtype that is not in the table and raises a bare KeyError. I reproduced it with main's reader: soup adapters check prints "Corrupt or unreadable safetensors weights ...: 'float8_e4m3fn'" and soup adapters diff prints "Error: KeyError: 'float8_e4m3fn'". On your base the same file is refused cleanly by read_header, so the tests cannot show it. Please look the dtype up with .get and raise a ValueError that names the tensor and the dtype, add a test that writes an F8_E4M3 header by hand and expects that ValueError from check_adapter and load_adapter_weights (it passes before and after a rebase), and rebase onto main.

Smaller things, none blocking:

  • The dtype tests cover bf16, f16 and f32 only. Changing the float64, int64, int8 and uint8 entries of _NUMPY_DTYPES, or disabling the len(raw) != entry.nbytes branch, keeps all tests green. One parametrized test over every mapped dtype compared with safe_open(..., framework="numpy") would close that; the in-loop truncation branch can only trigger if the file changes after the header was checked, so either test it with a patched read_header or drop it.
  • test_bf16_truncated_payload_is_a_named_refusal also passes on the old code, because the old wrapper used the same "Corrupt or unreadable" words. Asserting on text only the new path produces, such as "past the end of the file", would make it pin the new reader.
  • uint16, uint32 and uint64 tensors were readable through numpy and are now refused as "Corrupt or unreadable". LoRA adapters do not store them, so this is fine, but "unsupported dtype" is a better description than "corrupt".
  • The message repeats the file name ("in adapter_model.safetensors: adapter_model.safetensors: tensor ..."); drop the path prefix that read_header adds.
  • The arrays are read-only now. I found no in-place use in check, diff, merge, scan or the evolutionary merge, so it is fine; a sentence in the docstring would help the next caller.
  • The new assert res.exit_code == ... lines would be easier to debug with (res.output, repr(res.exception)) as the message.

Please also note that probe_kernel.py, sae_diff.py and steering.py use the same framework="numpy" call, as the issue mentions; a follow-up issue is the right place for those.

…eyError (MakazhanAlpamys#2058)

Since MakazhanAlpamys#2061 read_header accepts the FP8 spellings (F8_E4M3, F8_E5M2, ...), so
_read_safetensors_numpy hit _NUMPY_DTYPES[entry.dtype] with a dtype it does not
map and raised a bare KeyError ("Error: KeyError: 'float8_e4m3fn'" from adapters
diff). The dtype is now looked up with .get and a missing one raises a ValueError
naming the tensor and the dtype.

Review follow-ups: a dtype read_header refuses (U16/U32/U64) is reported as
"Unsupported" rather than "Corrupt or unreadable"; the read_header path prefix is
dropped so the file is named once; the docstring notes the arrays are read-only.

Tests: a hand-written F8_E4M3/F8_E5M2 header is refused by check_adapter and
load_adapter_weights; every mapped dtype (matrix, scalar, empty) matches
safe_open(framework="numpy"); the truncation test pins "past the end of the
file"; the in-loop truncation branch is covered with a patched read_header; CLI
exit-code asserts carry the output and exception.
@drakeo338

Copy link
Copy Markdown
Contributor Author

Pushed d89f441: merged main and made the adapter numpy reader use .get, so an FP8 tensor raises a ValueError naming the tensor and dtype instead of a KeyError, with F8_E4M3/F8_E5M2 tests, plus the review nits.

@MakazhanAlpamys MakazhanAlpamys added the ci:full Run the full CI matrix on this PR (a maintainer adds it at approval) label Oct 21, 2026
@MakazhanAlpamys
MakazhanAlpamys merged commit a1921ee into MakazhanAlpamys:main Oct 23, 2026
24 checks passed
@MakazhanAlpamys

Copy link
Copy Markdown
Owner

Thank you, @drakeo338. The reader follows option 1 of #2058: one header-based path, bfloat16 widened exactly (all 65536 bit patterns match tensor.float()), caps and symlink checks kept. When main added the FP8 spellings you turned the KeyError into a named refusal with a test for check and load, and took the dtype-coverage, truncation-text, file-name and read-only items in the same push. Checked on d89f441: test_issue1721_adapters_check.py 67 passed (also with FORCE_COLOR=1), the adapter diff, merge and scan files pass, ruff is clean; mutations of the FP8 guard, the bf16 shift, the tensor cap, the dtype table entries and the truncation branch each fail a named test.

MakazhanAlpamys added a commit that referenced this pull request Oct 23, 2026
Display names from `gh api users/<login>`.
Nari-Ab added a commit to Nari-Ab/Soup that referenced this pull request Oct 23, 2026
…kazhanAlpamys#2056)

- Differentiate dependency requirements between read-only and write commands:
  - check and diff require numpy only (accommodating header parsing introduced in MakazhanAlpamys#2063).
  - merge and arithmetic require both numpy and safetensors.
- Implement READ_ADAPTER_DEPS and WRITE_ADAPTER_DEPS with clear installation hints.
- Add pre-flight dependency validation to adapters merge and adapters arithmetic before scanning to avoid false scan bypass prompts.
- Document numpy requirement for soup adapters check in docs/adapters-and-governance.md.
- Rename test suite to tests/test_adapter_dependencies.py per CONTRIBUTING guidelines.
- Add acceptance tests asserting failure when dependencies are missing, and pinning MakazhanAlpamys#2063 where check/diff succeed without safetensors.
- Add changelog fragment in canonical format.
MakazhanAlpamys pushed a commit that referenced this pull request Oct 26, 2026
) (#2141)

fix(adapters): name the missing numpy or safetensors package in adapter commands (#2141)

`soup adapters check`, `diff` and `scan` need numpy only (since #2063); `merge` and `arithmetic` need numpy and safetensors and now refuse before the backdoor-scan gate, with exit 1 and the install hint, instead of reporting corrupt weights or a scan failure.
A partial install where `safetensors` imports but `safetensors.numpy` does not is reported the same way, and a fresh interpreter without numpy reaches the refusal from `soup adapters check`.

Closes #2056
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:full Run the full CI matrix on this PR (a maintainer adds it at approval)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

adapters check and diff cannot read a bfloat16 adapter (dtype not understood)

2 participants