Repository navigation
fix(adapters): read bfloat16 adapters in adapters check and adapters diff (#2058) - #2063
Conversation
…rs diff` (MakazhanAlpamys#2058) Both commands loaded the weights with safe_open(framework="numpy"), which cannot return bfloat16 because numpy has no such dtype. An adapter saved from a bf16 run therefore failed with "data type 'bfloat16' not understood" (diff) or "Corrupt or unreadable safetensors weights" (check). They now share one reader, _read_safetensors_numpy in utils/adapter_diff.py. It parses the header with read_header (offsets and sizes validated, no mapping) and reads the payload directly; a bfloat16 tensor is widened to float32 exactly, as the top 16 bits of a float32. Other dtypes are read as before. The size cap in adapters check, the tensor-count cap (now checked before any payload is read) and the symlink checks are unchanged.
|
Thanks, @drakeo338, #2063 follows option 1 of #2058 closely: one shared header-based reader, BF16 widened exactly, the size and tensor caps kept, and tests for check, diff, truncation and the cap. One process note: #2058 carries |
There was a problem hiding this comment.
Process note first, to square this with my earlier comment: the merge still waits for the area:cli first-refusal window on #2058 (until 2026-10-22 19:40Z) or @Nari-Ab's word. I reviewed the code now so that the change below can be made in the meantime.
Thanks for this, @drakeo338. It is a clean fix and it follows option 1 of #2058: one shared reader that parses the safetensors header, widens bfloat16 to float32 exactly, and keeps the size cap, the tensor-count cap and the symlink checks. I checked it against the branch:
- The widening is exact. I wrote all 65536 bfloat16 bit patterns with torch and compared: the float32 values are bitwise identical to
tensor.float(), including NaN, infinity and negative zero. - The other dtypes did not move: f16, f32, f64, i8..i64, u8, bool, a scalar and an empty tensor come back with the same dtype, shape and values as
safe_open(framework="numpy"). - On real torch-saved bf16 adapters,
soup adapters checkprintsalive(exit 0) with a norm within bf16 rounding of the float32 copy, a zerolora_Bbf16 adapter givesinactive: all lora_B layers are zero (1/1)(exit 2), andsoup adapters diffworks. Merge and scan also read bf16 and float64 adapters now, since they share the loader. - Mutation checks: reverting the two source files fails 5 of the new tests, changing the shift from 16 to 15 fails 4, and turning off the tensor-count cap fails the cap test.
test_issue1721_adapters_check.pypasses (also withFORCE_COLOR=1), as do the adapter diff, merge and scan test files and ruff.
One change is needed before this can merge. Main has moved since your base: #2061 added the FP8 spellings (F8_E4M3, F8_E5M2, ...) to safetensors_reader._DTYPES. With that change, _read_safetensors_numpy does _NUMPY_DTYPES[entry.dtype] on a dtype that is not in the table and raises a bare KeyError. I reproduced it with main's reader: soup adapters check prints "Corrupt or unreadable safetensors weights ...: 'float8_e4m3fn'" and soup adapters diff prints "Error: KeyError: 'float8_e4m3fn'". On your base the same file is refused cleanly by read_header, so the tests cannot show it. Please look the dtype up with .get and raise a ValueError that names the tensor and the dtype, add a test that writes an F8_E4M3 header by hand and expects that ValueError from check_adapter and load_adapter_weights (it passes before and after a rebase), and rebase onto main.
Smaller things, none blocking:
- The dtype tests cover bf16, f16 and f32 only. Changing the float64, int64, int8 and uint8 entries of
_NUMPY_DTYPES, or disabling thelen(raw) != entry.nbytesbranch, keeps all tests green. One parametrized test over every mapped dtype compared withsafe_open(..., framework="numpy")would close that; the in-loop truncation branch can only trigger if the file changes after the header was checked, so either test it with a patchedread_headeror drop it. test_bf16_truncated_payload_is_a_named_refusalalso passes on the old code, because the old wrapper used the same "Corrupt or unreadable" words. Asserting on text only the new path produces, such as "past the end of the file", would make it pin the new reader.uint16,uint32anduint64tensors were readable through numpy and are now refused as "Corrupt or unreadable". LoRA adapters do not store them, so this is fine, but "unsupported dtype" is a better description than "corrupt".- The message repeats the file name ("in adapter_model.safetensors: adapter_model.safetensors: tensor ..."); drop the path prefix that
read_headeradds. - The arrays are read-only now. I found no in-place use in check, diff, merge, scan or the evolutionary merge, so it is fine; a sentence in the docstring would help the next caller.
- The new
assert res.exit_code == ...lines would be easier to debug with(res.output, repr(res.exception))as the message.
Please also note that probe_kernel.py, sae_diff.py and steering.py use the same framework="numpy" call, as the issue mentions; a follow-up issue is the right place for those.
…eyError (MakazhanAlpamys#2058) Since MakazhanAlpamys#2061 read_header accepts the FP8 spellings (F8_E4M3, F8_E5M2, ...), so _read_safetensors_numpy hit _NUMPY_DTYPES[entry.dtype] with a dtype it does not map and raised a bare KeyError ("Error: KeyError: 'float8_e4m3fn'" from adapters diff). The dtype is now looked up with .get and a missing one raises a ValueError naming the tensor and the dtype. Review follow-ups: a dtype read_header refuses (U16/U32/U64) is reported as "Unsupported" rather than "Corrupt or unreadable"; the read_header path prefix is dropped so the file is named once; the docstring notes the arrays are read-only. Tests: a hand-written F8_E4M3/F8_E5M2 header is refused by check_adapter and load_adapter_weights; every mapped dtype (matrix, scalar, empty) matches safe_open(framework="numpy"); the truncation test pins "past the end of the file"; the in-loop truncation branch is covered with a patched read_header; CLI exit-code asserts carry the output and exception.
|
Pushed d89f441: merged main and made the adapter numpy reader use |
|
Thank you, @drakeo338. The reader follows option 1 of #2058: one header-based path, bfloat16 widened exactly (all 65536 bit patterns match |
Display names from `gh api users/<login>`.
…kazhanAlpamys#2056) - Differentiate dependency requirements between read-only and write commands: - check and diff require numpy only (accommodating header parsing introduced in MakazhanAlpamys#2063). - merge and arithmetic require both numpy and safetensors. - Implement READ_ADAPTER_DEPS and WRITE_ADAPTER_DEPS with clear installation hints. - Add pre-flight dependency validation to adapters merge and adapters arithmetic before scanning to avoid false scan bypass prompts. - Document numpy requirement for soup adapters check in docs/adapters-and-governance.md. - Rename test suite to tests/test_adapter_dependencies.py per CONTRIBUTING guidelines. - Add acceptance tests asserting failure when dependencies are missing, and pinning MakazhanAlpamys#2063 where check/diff succeed without safetensors. - Add changelog fragment in canonical format.
) (#2141) fix(adapters): name the missing numpy or safetensors package in adapter commands (#2141) `soup adapters check`, `diff` and `scan` need numpy only (since #2063); `merge` and `arithmetic` need numpy and safetensors and now refuse before the backdoor-scan gate, with exit 1 and the install hint, instead of reporting corrupt weights or a scan failure. A partial install where `safetensors` imports but `safetensors.numpy` does not is reported the same way, and a fresh interpreter without numpy reaches the refusal from `soup adapters check`. Closes #2056
What does this PR do?
Fixes #2058.
soup adapters checkandsoup adapters diffread weights withsafe_open(framework="numpy"), and numpy has no bfloat16 dtype. A bf16 adapter therefore failed withdata type 'bfloat16' not understood, reported as "Corrupt or unreadable" bycheck. Both commands now share one reader that parses the safetensors header and widens bfloat16 to float32 exactly. The size cap, tensor-count cap and symlink checks are unchanged.Type of change
Checklist
ruff check src/soup_cli/ scripts/ tests/passes (ran on the three touched files only)pytest tests/ -vpasses (ran only the adapter tests: 37 passed; the new tests fail on the old source)README.mdand the matching page underdocs/) if needed (no docs change)CONTRIBUTING.md)