Skip to content

Release 0.7.1: complete the binding surface, the test suite, and the packaging - #21

Merged
giacomocaironi merged 92 commits into
masterfrom
dev
Aug 1, 2026
Merged

giacomocaironi merged 92 commits into
masterfrom
dev

Conversation

@fametrano

@fametrano fametrano commented Jul 29, 2026 •

Copy link
Copy Markdown
Member

Brings dev up to master for the 0.7.1 release: 57 commits that widen the binding surface, replace the single smoke test with a real suite validated against external vectors, and rebuild packaging and CI around it.

API surface

The wrappers now cover the modules that were previously reachable only through the raw extension:

  • keys, xonly — key algebra, taproot tweaking, key ordering
  • dsa, ssa — ECDSA signature forms, variable-length signing, tagged hashing
  • ecdh, recovery, ellswift, mult — the remaining upstream modules
  • context — the shared context, documented and tested under concurrent use
  • hashes, _scalar — tagged hashing, and one helper that normalizes every scalar argument

Invalid arguments are reported consistently, return codes are checked, and what libsecp256k1 says through its own callbacks is surfaced rather than swallowed. MuSig2 is stated to belong to btclib; the ECDH hash function stays unexposed, with the reason recorded.

Testing

tests/test.py is gone, replaced by tests exercised against external vectors only — official BIP340, RFC6979/DER, and btclib's own ECDSA vectors — plus property tests that assert invariants over derived inputs rather than chosen ones. Coverage is gated on a ratchet and measures only reachable code.

Packaging and CI

  • Vendored libsecp256k1 built with CMake on every platform
  • Wheels for macOS arm64, Linux aarch64, Windows x64 and arm64; free-threaded interpreter tested, not merely built for
  • uv manages the development environment; uv.lock is committed
  • ruff replaces eight linters and formatters; mypy --strict type-checks the cffi boundary
  • Least-privilege tokens, SHA-pinned actions, Dependabot for Actions and the submodule
  • Tag-triggered PyPI release workflow with Trusted Publishing, able to rehearse itself on TestPyPI (a rehearsal is never tagged)
  • RELEASING.md and SECURITY.md added; README and HISTORY brought current

Note before merging

pyproject.toml still declares version = "0.7.1rc1", set for the TestPyPI rehearsal. That needs to become 0.7.1 before the release tag is cut.

🤖 Generated with Claude Code

Summary by Sourcery

Finalize the 0.7.1 release by widening the libsecp256k1 binding surface, hardening argument validation and error reporting, rebuilding the CFFI/CMake-based extension and wheel packaging, and replacing the ad-hoc tests with a comprehensive, coverage-gated suite driven by external vectors and property tests.

New Features:

  • Expose key algebra, x-only taproot operations, tagged hashing, ECDSA/Schnorr utilities, ECDH, recovery, ElligatorSwift, and raw MuSig2 access through dedicated binding modules.
  • Add a shared, randomized libsecp256k1 context with per-thread callback reporting and document its concurrent, free-threaded-safe use.
  • Provide versioned distribution metadata including PEP 639 licensing, __version__, and sdist/wheel contents suitable for Trusted Publishing with PEP 740 attestations.

Bug Fixes:

  • Prevent invalid or out-of-range scalar, key, and signature inputs from crossing the CFFI boundary by normalizing and validating them consistently.
  • Fix dynamic builds and sdists by moving callback stubs out of the vendored sources and ensuring CMake/CFFI subprocess failures are surfaced.
  • Correct shared library discovery for dynamic wheels and ensure platform tags and deployment targets reflect the actual build environment.

Enhancements:

  • Unify the vendored libsecp256k1 build on CMake across all platforms, avoid mutating the submodule, and support static and dynamic extension builds including cross-compiled Windows wheels.
  • Refine ECDSA and Schnorr APIs to return booleans, support custom-length messages and auxiliary randomness, and expose normalization, malleability, and encoding conversion helpers.
  • Strengthen type checking via strict mypy with dedicated stubs for the CFFI boundary, and introduce a scalar helper that standardizes all 32-byte numeric arguments.

Build:

  • Switch to CMake as a declared build requirement for the vendored library, add a custom hatchling hook for CFFI modules, and tune wheel tags for static vs dynamic builds across CPython and PyPy, including free-threaded interpreters.
  • Define sdist exclusions to ship pristine sources only and configure check-wheel-contents to accept ABI-mode layouts.
  • Add uv/PEP 735 dependency groups and lockfile for reproducible local builds and benchmarking dependencies.

CI:

  • Replace multiple workflows with a single main pipeline that runs linting, coverage, multi-platform wheel builds, dynamic builds, sdist builds, sdist install tests, distribution validation, and matrix testing across CPython and PyPy.
  • Harden CI with least-privilege GITHUB_TOKEN, SHA-pinned actions, concurrency cancellation, and expanded platform matrices for macOS arm64, Linux aarch64, Windows x64 and arm64, and free-threaded Python.
  • Introduce a release workflow that reuses the main pipeline, gates PyPI/TestPyPI uploads via environments and Trusted Publishing, and validates artifacts with twine and check-wheel-contents before publishing.

Deployment:

  • Enable tag-triggered PyPI releases and manual TestPyPI rehearsals with clear release instructions and environment-based approval gates.
  • Document and enforce the release versioning scheme and yanking policy for broken releases.

Documentation:

  • Rewrite and expand the README to cover module coverage, versioning, MuSig2 rationale, thread safety, vendored-library decisions, build/test/dev workflow, and platform support.
  • Add RELEASING.md detailing Trusted Publishing, rehearsals on TestPyPI, and one-time index setup, plus SECURITY.md describing the vulnerability-reporting process and the boundary with upstream libsecp256k1.
  • Update HISTORY.md with a comprehensive 0.7.1 changelog and align metadata URLs with the current documentation and changelog locations.
  • Document build and benchmark scripts in scripts/README.md and clarify their non-API status.

Tests:

  • Replace the legacy single test script with a structured test suite covering BIP340 official vectors, RFC6979 and btclib ECDSA vectors, DER edge cases, libsecp256k1 module behaviour (ECDH, recovery, ElligatorSwift, MuSig2), core signing and input validation, extension loading, callbacks, concurrency, key algebra invariants, and tagged hashing.
  • Introduce property-style tests that sweep derived inputs via SHA256 chains to exercise scalar/point algebra, ECDH/XDH symmetry, taproot tweaking, key ordering, and rare cases such as leading-zero x coordinates and short DER signatures.
  • Configure pytest, coverage (branch mode with 100% fail-under ratchet and unreachable RuntimeError paths excluded), and cibuildwheel test hooks to ensure every wheel and interpreter combination is exercised.

Chores:

  • Replace the linting/formatting stack (black, isort, flake8, bandit, pyupgrade, etc.) with ruff and update the pre-commit configuration, including mypy mirror settings and markdownlint updates.
  • Add Dependabot configuration for GitHub Actions and the secp256k1 submodule and pin pre-commit hook revisions for regular updates.
  • Drop deprecated Python versions (3.7, 3.8), add support for newer ones (3.13, 3.14 and free-threaded 3.14t), and adjust cibuildwheel and CI matrices accordingly.

fametrano and others added 30 commits July 29, 2026 10:59
- Wrap libsecp256k1 0.7.1 (1a53f49)
- Support Python 3.9-3.14 (drop EOL 3.7/3.8, add 3.13/3.14)
- Align release numbers with the wrapped libsecp256k1 version
- Modernize CI: current runners and actions, cibuildwheel 4.x,
  test matrix 3.9-3.14 + PyPy
- Update all pre-commit hooks to their latest versions
- Build the callback stubs as a separate compilation unit instead of
  mutating the vendored sources, fixing sdist rebuilds with multi-pass
  PEP 517 frontends such as uv (#20)
- Fail fast on build subprocess errors
- Use secp256k1_schnorrsig_sign32 instead of the deprecated alias

Co-Authored-By: Claude Fable 5 <[email protected]>
- Exclude vendored-tree build artifacts from the sdist: stale
  autotools/make output from a local build (Makefile, config.status,
  .libs, ...) would break or pollute builds on the installing machine;
  sdist size drops from 5.7MB to 2.1MB
- Point the CI badge to the main.yml workflow (build.yml/test.yml no
  longer exist) and drop the pyright badge (tool not used)

Co-Authored-By: Claude Fable 5 <[email protected]>
Extend the cibuildwheel build matrix and the static-wheel test matrix
with the arm64 GitHub runners (macos-latest, ubuntu-24.04-arm).

Also document in HISTORY.md why Python 3.7/3.8 support was dropped.

Co-Authored-By: Claude Fable 5 <[email protected]>
The tag was hardcoded to x86_64 (with a fake macOS 10.16 minimum
version): a dynamic wheel built on an arm64 machine was tagged as
x86_64, hence wrong and not even installable where it was built.
Only the mingw cross-compilation keeps a fixed win_amd64 tag, as the
target machine cannot be inspected.

Merge the duplicated linux/macos dynamic-wheel jobs into one matrix
job covering arm64 too (ubuntu-24.04-arm, macos-latest), move the
sdist to its own job, and extend the dynamic-wheel test matrix to the
arm64 runners.

Co-Authored-By: Claude Fable 5 <[email protected]>
The host version does not affect the produced wheels; 3.14 is the
current stable and the one all local verification runs on.

Co-Authored-By: Claude Fable 5 <[email protected]>
On native Windows the vendored library is now built with CMake/MSVC
(BUILD_SHARED_LIBS=OFF, external default callbacks ON) and the cffi
extension with the standard setuptools/MSVC toolchain, defining
SECP256K1_STATIC and compiling the callback stubs into the extension
itself. The autotools/mingw path is unchanged and still used for
POSIX builds and for the cross-compiled dynamic Windows wheel.

windows-latest joins the cibuildwheel and static-test matrices
(CPython only: PyPy wheels are skipped on Windows for now).

Co-Authored-By: Claude Fable 5 <[email protected]>
- All libsecp256k1 return codes are now checked, raising ValueError
  with a clear message: malformed keys and signatures were previously
  parsed into uninitialized structs and verified against garbage
- Fixed-size buffers the C side reads blindly (msg32, sig64, seckey)
  are length-checked before the call
- The three per-module contexts are replaced by a single shared one
  (context.py), created with the modern SECP256K1_CONTEXT_NONE flag
  instead of the deprecated SIGN|VERIFY (769) and randomized against
  side-channel leakage, as recommended by libsecp256k1
- New negative tests cover the error paths

Co-Authored-By: Claude Fable 5 <[email protected]>
Weekly grouped PRs for action bumps, monthly for submodule movement
(upstream default branch: release bumps stay manual, aligned with the
tagged version). pre-commit hooks are already covered by pre-commit.ci.

Co-Authored-By: Claude Fable 5 <[email protected]>
- BIP340: the 19 vectors from bitcoin/bips test-vectors.csv (vendored);
  signing is exercised where the bindings allow it (32-byte messages),
  verification on every vector including the invalid-key/signature ones
- RFC6979 deterministic ECDSA on secp256k1: (k, r, s) vectors published
  on bitcointalk, as vendored by trezor-firmware and bitcoinjs-lib;
  each vector is additionally self-checked asserting r == x(k*G)
- the ECDSA vectors used by btclib (vendored there from
  rustyrussell/secp256k1-py): 199 deterministic signatures reproduced
  byte-for-byte, plus 199 custom-nonce signatures verified and anchored
  via r == x(k*G); btclib's rfc6979.json is not imported as it only
  covers NIST curves, not secp256k1
- DER: canonical encodings from trezor-firmware must parse (and fail
  verification); trailing garbage and non-minimal padding must be
  rejected; the leniencies of secp256k1_ecdsa_signature_parse_der
  (unsigned reading of high-bit integers, zeroing of out-of-range
  values) are pinned as regression expectations

Co-Authored-By: Claude Fable 5 <[email protected]>
- permissions: contents: read at workflow level: anything running in a
  job (actions, pip dependencies, build scripts) can read the
  GITHUB_TOKEN, so it must not be able to write to the repository;
  artifact upload/download uses the runtime token and is unaffected
- all actions pinned to full commit SHAs (with the version as a
  comment): version tags are mutable and a compromised action repo can
  repoint them, as in the tj-actions incident of March 2025; Dependabot
  understands SHA pins and keeps them updated

Co-Authored-By: Claude Fable 5 <[email protected]>
The Intel macOS runners are scarce and queue for a long time: today a
dev run sat 40+ minutes with only the two macos-13 build jobs pending.
Development pushes now skip them; the full Intel coverage still gates
every PR to master and every push to master.

Co-Authored-By: Claude Fable 5 <[email protected]>
- release.yml runs on v* tags: it reuses the whole main.yml build/test
  pipeline via workflow_call (single source of truth for the build
  jobs), then publishes every wheel and the sdist to PyPI
- publishing uses Trusted Publishing (OIDC): no long-lived PyPI token
  exists anywhere; the job only adds id-token: write, and PEP 740
  attestations are generated by default
- the publish job runs in the pypi environment, so required reviewers
  can gate every release behind a manual approval
- main.yml push trigger restricted to branches (tags belong to the
  release workflow) and the macos-13 jobs enabled on tag refs, so
  releases ship Intel macOS wheels
- release process documented in README.md, including the one-time
  PyPI/GitHub setup

Co-Authored-By: Claude Fable 5 <[email protected]>
- every optional libsecp256k1 module is now compiled in and declared to
  cffi: ecdh, musig and ellswift were already compiled in (they are on
  by default upstream) but never declared, so their symbols were
  unreachable; recovery was the only one actually disabled
- all the wrapped modules are requested explicitly at configure time on
  both build paths: upstream defaults are not part of its API, and the
  autotools and CMake paths were enabling different module sets
- new ecdh, recovery, and ellswift binding modules; the header
  concatenation order now matters, as the #include directives are
  stripped before preprocessing and musig needs the extrakeys types
- MuSig2 has no binding module: its two-round protocol needs an
  interface that makes secret nonce reuse hard, still to be designed;
  it is reachable through the raw cffi bindings, and tests/test_modules
  runs a complete 2-of-2 session as usage example
- the new tests cross-check the bindings against each other instead of
  against vendored constants: the ECDH secret is recomputed from the
  shared point, the recoverable signature is compared with the
  deterministic ECDSA one, and the MuSig2 aggregate signature is
  verified as a plain BIP340 signature by ssa

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
An in-place dynamic build copies the shared libsecp256k1 next to the
bindings; *.so and *.dll were already ignored, so the artifact only
showed up as untracked on macOS.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
- new keys module: private and public key algebra (verify, negate,
  tweak add and multiply, combine) plus public key parsing and
  serialization; pubkey_tweak_mul is the multiplication of an arbitrary
  point, which mult, being limited to the generator, cannot express
- new xonly module: the BIP341 taproot primitives, i.e. the tweaking of
  an x-only public key and of its private key, so that a key path
  spending can be signed with ssa; tweak_add_check verifies a taproot
  commitment without recomputing the tweak
- dsa now exposes the malleability primitives (normalize, is_low_s) and
  the DER to compact conversions: signatures coming from a system that
  does not enforce lower-s must be normalized before verification, as
  libsecp256k1 rejects them
- dsa.verify and ssa.verify return bool instead of int; being a subtype
  of int, with True == 1, callers cannot tell the difference
- dsa.sign and dsa.verify now share the DER parse and serialize helpers
  with the new conversions, instead of inlining them
- the tests cross-check the scalar and point operations against mult
  modulo the group order, the x-only tweaking against the plain public
  key one (a distinct libsecp256k1 code path), and run a full taproot
  key path: tweak the key, sign with the tweaked private key, verify
  against the tweaked x-only public key; the malleated signature of the
  low-s test is built through the new compact conversions

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Two holes in the pipeline: the pre-commit hooks (formatting, linting,
typing) ran only on the developer's machine, so nothing stopped a
non-conforming commit from landing; and the sdist was built and
uploaded as an artifact without ever being installed.

- new lint job: pre-commit run --all-files on a single runner, with the
  hook environments cached on the hash of .pre-commit-config.yaml; it
  is the cheap gate that must fail before the expensive build matrix
- new test-sdist job: installs the built sdist and runs the test suite
  on Linux and macOS. Installing from sdist is the only path that has
  to compile the vendored libsecp256k1 on the target machine, i.e. the
  path the sdist exclude list in pyproject.toml exists to support;
  until now a broken sdist would have been discovered on PyPI

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
black, isort, flake8 (plus the Flake8-pyproject shim to read its config
from pyproject.toml), autoflake, pyupgrade, bandit, pydocstringformatter
and yesqa all did one slice of the job, each with its own configuration
section and its own exclusion of the vendored secp256k1 tree. ruff does
all of it: two hooks instead of nine, one config section, seconds
instead of a minute.

- the rule families selected in pyproject.toml stand in one to one for
  the tools they replace (F/E/W for flake8, I for isort, UP for
  pyupgrade, S for bandit, D for pydocstringformatter, C90 for the
  McCabe complexity limit), plus C4, SIM, PTH, RET, PL and RUF; the
  pylint section that no hook ever ran is gone
- D202 and PLR2004 are ignored project wide: the blank line after a
  docstring is the style of this codebase, and comparing against a key,
  signature or hash length is the domain, not a magic value
- the ~150 "# nosec B101" markers and the comment blocks quoting bandit
  rule ids and URLs are dead with bandit gone: ruff reads suppressions
  from the per-file-ignores table, where the justification now lives
- the copyright-notice hook matched "python" as a filename pattern, so
  it silently checked no file at all; with the pattern fixed to \.py$
  it found noxfile.py and scripts/hatch_build.py missing the notice
- three module docstrings ("... according." / "to BIP324: ...") had a
  summary line split mid sentence: rewritten so the first line stands
  on its own, as D212 requires
- assorted fixes ruff found and the previous chain did not: os.remove
  and glob.glob replaced by pathlib equivalents, redundant str() inside
  f-strings, superfluous else after return, if/else blocks that are
  plain ternaries, an unused glob import
- the five README badges advertising isort, black, flake8, pylint and
  bandit are replaced by the ruff one

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Coverage was measured by hand, with the source and the report options
spelled out on the command line, and never in CI: nothing recorded what
the suite reached, so nothing noticed it regressing.

- [tool.coverage] in pyproject.toml: branch mode (a wrapper is made of
  guard clauses, and statement coverage calls a half-tested if fully
  covered), source, and fail_under = 84, the level the suite reaches
  today. It is a ratchet, not a target: the remaining gap is error paths
  that need libsecp256k1 itself to fail, plus the dlopen fallback in
  __init__.py, unreachable in a static build and the only covered
  branch in a dynamic one, which is why the static build is the one
  measured
- new coverage job in CI, on a single runner: coverage is a property of
  the python layer, not of the platform
- pytest now runs with --strict-markers and --strict-config (a typo in a
  marker or in a config key is an error, not a silent no-op),
  xfail_strict (an xfail that passes is a stale test) and
  filterwarnings = error: across a 3.9 to 3.14 spread a deprecation
  warning from cffi or CPython is the cheapest possible early warning
- testpaths replaces the python_files / python_functions widening, so
  tests/test.py, the only file needing it, is renamed test_core.py
  after what it covers: round-trips, input validation and safe aborts

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The package ships py.typed, so its annotations are what btclib
type-checks against downstream, but mypy ran with strict = false and,
worse, "import _btclib_libsecp256k1  # type: ignore" made ffi and lib
plain Any: every wrapper in the package is a call through those two
objects, so the whole codebase type-checked vacuously.

- new stubs/_btclib_libsecp256k1.pyi, on mypy_path: the extension is
  generated at install time and has no source to inspect. It declares
  only the surface the package uses (new, unpack, NULL, dlopen) and
  keeps lib as Any on purpose, its members being the ~100 C entry
  points generated at build time. Typing unpack as returning bytes is
  what removed 20 "Returning Any from function declared to return
  bytes" errors, i.e. 20 functions whose return type nothing checked
- strict = true, and the annotations it demanded: the opaque cffi
  handles are now spelled CData (an alias exported by the package, so
  that public signatures like keys.parse say what they return instead
  of Any), and FFIExtension declares the five attributes its subclass
  is required to set, which mypy was right to flag as undefined
- the mypy mirror injects --ignore-missing-imports, which silently turns
  any unresolved or misspelled import into Any: dropped, so that the
  optional comparands of the benchmark (btclib, coincurve, secp256k1)
  have to be declared as overrides in pyproject.toml. secp256k1 needs
  an explicit skip either way, or the vendored C tree is picked up as a
  namespace package of the same name
- the extension import path is restructured into _load_lib(): the dlopen
  fallback assigned lib inside a nested loop, so a failure to load left
  the name unbound and the import failed with a bare NameError instead
  of a diagnosable error. It now raises ImportError naming the directory
  searched, and no longer keeps looking after a successful load
- the stub had to be exempted from the .gitignore rule that hides the
  build artifacts of the extension

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Three tools overlapped: the hatch environments in pyproject.toml, the
noxfile, and pre-commit, each installing its own copy of the same
linters. uv replaces the first two; pre-commit stays, as the gate that
pins the authoritative revisions of the hooks.

- development dependencies become PEP 735 groups (test, lint, build),
  which are standard and not tied to one project manager: pip reads them
  too, with --group. The hatch environments and their scripts are gone,
  and so is the noxfile
- hatchling remains the build backend: uv is the installer and resolver,
  it has nothing to do with how the extension is compiled
- uv.lock is committed, so that "uv sync" gives the same development
  environment to everybody
- .python-version pins the interpreter the environment is built on (the
  newest supported one) and is now tracked instead of ignored: uv
  installs that interpreter if missing, which is what makes pyenv
  unnecessary
- CI: the lint job runs pre-commit through uvx, without installing the
  project, since linting does not need libsecp256k1 compiled; the
  coverage job uses uv sync and inherits the interpreter from
  .python-version, exactly as a developer does
- README documents the uv workflow, including the one sharp edge: uv
  keys its build cache on the sources, which do not tell it that the
  compiled extension is ABI specific, so testing against another
  interpreter needs --no-cache

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
- PEP 639 license metadata: license = "MIT" as an SPDX expression plus
  license-files, replacing the deprecated license table; the
  License :: OSI Approved classifier is now redundant and removed. The
  built distributions carry Metadata-Version 2.4 with License-Expression
  and both LICENSE and COPYRIGHT
- the package exposes __version__, read from the installed distribution
  metadata so that pyproject.toml stays the single place to bump at
  release time, as the documented release process assumes
- new check-dist job: twine check --strict and check-wheel-contents over
  every wheel and the sdist. It is the same validation PyPI applies on
  upload, run before the release workflow reaches the point of no
  return, since broken metadata or a README that does not render cannot
  be fixed by reuploading, only yanked. check-wheel-contents needs W003
  and W009 ignored: a dynamic wheel is meant to ship the shared
  libsecp256k1 as a non-module at the top level
- the changelog project URL pointed at
  github.com/btclib-org/btclib_libsecp256k1/HISTORY.md, a 404: it is
  missing the /blob/master/ path, and PyPI shows that link
- HISTORY.md records this whole round of infrastructure work

Not touched, as it is a decision about the published support matrix
rather than hygiene: requires-python is still >= 3.9, though 3.9 has
been end of life since October 2025.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The same five lines, serializing an int private key big endian and
checking that the result is 32 bytes, existed in nine places: inlined in
dsa, ecdh, ellswift (twice), recovery, ssa and mult, and as two
byte-identical private _scalar helpers in keys and xonly.

The new private _scalar module holds the one copy. Behaviour is
unchanged, error messages included: every call site passes the same
argument name the inlined message used ("private key", "tweak",
"scalar"), so the ValueError texts the tests match on are the same.

Coverage goes from 84.4% to 85.5% for free, and the fail_under ratchet
with it: eight partly covered length guards became one that the suite
exercises on both the int and the bytes path.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Building and testing on Windows is supported since the native
CMake/MSVC path: describe the per-platform toolchains instead.

Co-Authored-By: Claude Fable 5 <[email protected]>
Not a temporary limitation: PyPy is best served by the ABI-mode
dynamic wheel (the cffi mode recommended on PyPy), which pip falls
back to; a static pp3 wheel would take precedence over it and
perform worse.

Co-Authored-By: Claude Fable 5 <[email protected]>
dsa.sign did not check the length of its optional nonce contribution,
as ssa.sign and recovery.sign do: an ndata longer than 32 bytes left
the left padding empty, libsecp256k1 read 32 bytes through the bare
pointer, and the excess was dropped in silence.

An int scalar outside the 32-byte range surfaced the OverflowError of
int.to_bytes, where every other out-of-domain argument raises
ValueError; whether the value is a valid scalar, i.e. in [1, n-1],
remains for libsecp256k1 to say.

The lint dependency group did not carry the stubs the mypy pre-commit
hook declares, so mypy run from the development environment reported
three errors the gate does not see. The two have to agree, or neither
is worth running.

The build writes src/btclib_default_callbacks.c into the submodule
tree, as automake resolves the sources it is added to relative to it:
the untracked file left the submodule permanently reported as
modified, which only hides a real change.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The coverage ratchet stood at 85% because the validation errors of
recovery, ssa, xonly and dsa.verify were never exercised: the gap was
not made of unreachable code, it was untested code. Now at 89.9%, and
what remains uncovered is only the RuntimeError paths, which need
libsecp256k1 itself to fail, plus the dlopen fallback of __init__.py.

- recovery: a short message hash in recover, a short signature in
  to_der, and a signature whose r is zero, which parses but recovers no
  point
- ssa: a short message hash and an over-long aux_rand32 in sign
- xonly: an unparsable public key in from_pubkey, and the two tweaks
  that land on the point at infinity, i.e. by the negation of the
  private key of the even y point
- dsa: a short message hash in verify, and the nonce contribution of
  sign, which no test passed at all: it now checks that a custom nonce
  changes the deterministic signature, that the result still verifies,
  and that a shorter one is left padded. A one-line ternary had hidden
  this, being covered by the ndata=None path alone
- the int out of range guard of scalar(), on both sides of the range
- xonly.tweak_add is also exercised with a full public key, which was
  the only untested branch of _parse: the public key of 11 has odd y, so
  the result matches the x-only one only because the key is lifted to
  its even y point first

pre-commit.ci and the lint job of the main workflow were both running
the hooks, with two badges reporting it. The gate is the lint job, in
the same run as the build and test matrices; pre-commit.ci is declared
as what it alone provides, the weekly pull request bumping the pinned
hook revisions, with autofix disabled (a bot committing to a branch is
at odds with a workflow token that cannot write and actions pinned to
SHAs) and targeting dev, as Dependabot does. Its badge is dropped.

The documentation URL pointed at btclib.readthedocs.io, which documents
btclib and not these bindings; it now points at the README.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
keys.prvkey_tweak_mul with a zero tweak, whose product is zero and so no
private key, and keys.pubkey_tweak_add by the negation of the private
key, which lands on the point at infinity.

Coverage goes from 89.9% to 90.6%, and the ratchet with it. Every
ValueError the bindings raise is now exercised: what is left uncovered
is the RuntimeError paths, unreachable through the API since they need
libsecp256k1 itself to fail, and the dlopen fallback of __init__.py,
which is unreachable code in a static build and the only covered branch
in a dynamic one. Raising the ratchet further means testing the
untestable.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
A cryptographic package without one leaves whoever finds a flaw with no
channel but a public issue. There are two: GitHub private vulnerability
reporting, now enabled on the repository, and the address btclib
already publishes in its own policy, so that the organization has one
mailbox rather than one per project.

What the policy has to say beyond that is where the boundary of this
project lies. The cryptography is libsecp256k1, which has a policy and
an address of its own; ours is the layer driving it: the validation of
the arguments crossing the cffi boundary, the default callbacks
replaced with do-nothing stubs (so that nothing below the wrapper
modules catches an illegal argument, and code reaching the raw lib
bindings is on its own), which optional modules are compiled in, the
commit the submodule is pinned to, and the distributions published to
PyPI.

It also states what is not a vulnerability, to spare both sides the
exchange: secret material lives in immutable Python objects that are
never zeroized, and the constant-time properties of libsecp256k1 stop
at the boundary.

Only the latest release is supported: version numbers track the wrapped
libsecp256k1, so a fix is published as a new release and nothing is
backported.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The ratchet was gating on a number the suite could not raise. Of the 28
uncovered points, 18 were the RuntimeError paths, which report a
libsecp256k1 call failing where no argument can make it fail: excluded
from the measure, so that what is left is code a test can reach. The
distinction is the one the bindings already draw, ValueError for an
argument out of the domain and RuntimeError for the impossible, and
counting the two together meant a new untested guard clause could hide
inside the slack.

The rest was the dlopen fallback of _load_lib, unreachable in a static
build because the branch taken is the other one. It now takes the
extension as an argument, so a stand-in module pointing at an empty
directory drives the search: what is asserted is that a directory
holding no library is reported as such, and that a file the loader
rejects is skipped rather than ending the search, which is what a wheel
repaired by auditwheel or delocate needs.

Coverage is now 100% of lines and branches, and the ratchet is set
there: the only value that fails on a single new untested one. It
assumes the static extension the coverage job builds; on a dynamic one
the linked-in branch is the unexecuted one.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Windows was covered on x86_64 only: cibuildwheel ran on windows-latest
and the mingw cross-compiled dynamic wheel is x86_64 by construction, so
Windows on ARM had no wheel at all and fell back to the sdist, i.e. to
compiling libsecp256k1 with CMake/MSVC on the user's machine.

- windows-11-arm joins the build-cibuildwheel and test-static matrices.
  No source change is needed: CMake and MSVC target the host, so the
  win_arm64 wheel comes out of the same native path as win_amd64, and
  the artifacts reach check-dist on their own, being collected by pattern
- cp39 and cp310 are skipped for win_arm64, and excluded from the test
  matrix: CPython has no Windows arm64 build before 3.11. cibuildwheel
  does list those identifiers, having cross-compiled them from an x86_64
  host in the past, so without the skip it would look for an interpreter
  that cannot exist on a native arm64 runner
- the dynamic wheel stays x86_64: mingw cross-compilation targets it
  alone, as dynamic_platform_tag already documented

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
fametrano and others added 12 commits July 30, 2026 12:53
b803e1a moved the three packaging tools from uvx to the check group of
pyproject.toml, but left the setup-uv comment above them claiming they
are "fetched by uvx on every run". That cache holds the wheels of the
check group, keyed on uv.lock, and the comment twenty lines below
already said as much: the two disagreed, and a comment that carries the
reasoning teaches nobody to trust it once one of them is stale. The
wording now matches the lint workflow, where the same job caches the
same kind of group.

The unpinned `pip install -U pip pytest` of test-sdist, test-dynamic and
test-static is the choice it looks like, so it is written down rather
than left to be rediscovered as an oversight: those three jobs hold no
uv and no lock because their subject is pip resolving and building a
published artifact the way a user does, and pinning pytest would
reintroduce the resolution they exist to exclude. The price is named
too, because it is not hypothetical: filterwarnings = error applies to
them, so a pytest release can fail them on a day nothing here changed.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The comment told a maintainer to configure required reviewers on the
"pypi" environment so publishing waits for a manual approval. That is
done: the environment has a required_reviewers rule, and the gate is now
a fact about repository settings rather than an instruction in a file
that cannot assert it. An imperative comment left in place after it has
been carried out reads as work still to do. The testpypi job above never
carried the equivalent note, so the two environment blocks now read the
same; RELEASING.md remains where the approval step is documented.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The bindings validated their arguments in two registers, and only one of
them was theirs. Checking the size of a buffer that reaches a bare
pointer is memory safety: libsecp256k1 reads msg32 or input32 without
ever seeing a length, so a short one is read past its end and no return
code or callback can report what nothing in C observed. Deciding what a
caller meant is not the same thing: an ndata of four bytes was left
padded into a valid nonce contribution, and a full public key handed to
ssa.verify or xonly.tweak_add was reinterpreted as its even y point,
which is a different key from the one passed whenever y is odd.

The second register is gone. The 32 bytes of entropy — ndata of dsa.sign
and recovery.sign, aux_rand32 of ssa and ellswift.create, rnd32 of
ellswift.encode — are 32 bytes or omitted, a shorter value being a
caller mistake rather than a small number. The x-only surface takes the
32-byte form and only it, xonly.from_pubkey being where a caller drops a
y coordinate on purpose; ssa.verify parses through
secp256k1_xonly_pubkey_parse, which is the BIP340 call for what it is
handed, and one conversion and one unreachable RuntimeError go with it.

What stays is what C cannot see, and what libsecp256k1 already answers
stays its answer: whether 32 bytes are a scalar in [1, n-1] is
secp256k1_ec_seckey_verify's to say, not a range test in python. The int
scalar stays too, as the one convenience, with the cost now written
down: a python int is variable length, so serializing it leaks its
magnitude, and no check in front of the call gives that back. None of
these checks branches on the content of a secret, which is why the
constant-time guarantee they sit in front of is untouched.

README gains What the boundary checks, stating the three rules and the
raw lib and ffi path for a caller who wants none of them; CONTRIBUTING
holds a new wrapper to them, SECURITY.md records the int as an inherent
limit, and HISTORY.md names both breaks for 0.7.1.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The README carried a section called "Build, test, develop, and
contribute" which did four things, and only the first of them was the
README's: what the build does on each platform is a fact about the
artifact a user installs, while the submodule commands, uv sync, pytest,
the coverage and pre-commit invocations, the benchmark group, the
other-interpreter caveats and the table of local equivalents for each CI
job are all instructions to someone who has cloned the repository. That
someone is sent to CONTRIBUTING.md by the issue forms, by the pull
request template and by GitHub itself; the README pointed there only in
a closing sentence, after the commands it duplicated the audience of.

The split now follows the one Release process already used: the README
keeps the account of the mechanism — CMake out of tree, the
interpreter's own toolchain, the emulated-interpreter and universal2
cases, the cross-compiled dynamic Windows wheel — and hands off the
steps, as it hands off the release steps to RELEASING.md. CONTRIBUTING
gains a Building and testing section before What a change has to
satisfy, so the commands come before the requirements that cite them,
and Running what CI runs moves with them unchanged.

Three references that pointed "above" into the README are rewired now
that the text sits elsewhere: the editable install names the README for
the toolchain, the benchmark note asks for pkg-config besides the C
toolchain, and the Windows arm64 paragraph drops a dangling "as above".
The new pointer uses the absolute URL the RELEASING.md and SECURITY.md
references use, because the README is rendered on PyPI where a relative
link resolves to nothing.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The docstring of scalar() and the README bullet it belongs to both put
their weight on the wrong half of the cost. Serializing an int was
described as taking a time that depends on the magnitude of the value,
and it does not measurably: to_bytes loops over CPython digits of 30
bits, so a 256-bit scalar uses nine of them and would use eight only
below 2**240, which a uniform scalar is with probability 2**-16.
Measured, the small value, the eight-digit one and the full one land
within noise of one another. What is variable in the magnitude of its
operands is the python arithmetic that produced the int — (a*b) % n
takes 71 ns against 248 ns across that range — and that arithmetic ran
before this function saw anything, leaving unzeroized copies of every
intermediate on the heap. The advice to pass bytes is still right, but
what it buys is narrow, and the next paragraph of the README already
concedes that bytes are not zeroized either: it buys only that no
arithmetic on the secret happened here.

The other half was missing outright. The boundary refuses to pad a short
bytes into validity, and then accepts an int and serializes it to the
same 32 bytes, so scalar(1) passes where scalar(b"\x01") raises. That
asymmetry has a reason and neither text gave it: a bytes states a value
and a width, and accepting a short one means choosing which of the two
to disbelieve, while an int states only a value — the width is the
curve's, not a fact the caller supplied and got wrong. The set of valid
scalars does not change, only the type spelling them, which is why the
int is a convenience rather than a leniency.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Everything here is asked to explain itself: comments say why, the
workflows and the build scripts carry paragraphs, and the documentation
is four files that a contributor is sent to by name. None of it was
checked for spelling. Every hook in this file reads the code and leaves
the prose alone, so a misspelling in a comment or a README section
reached a reader before it reached a check — and the reader it reaches
first is someone deciding whether to trust a cryptography binding.

Nothing needed fixing: the tree is clean under codespell's default
dictionaries. That is a result worth having rather than a reason not to
add the hook, and it was verified the way this repository verifies a
check, by handing it something bad — a typo planted in README.md turns
the hook red — because a check that reports nothing is indistinguishable
from a check that looks at nothing.

The optional dictionaries stay off, and the comment says why: tried, they
report only what this repository means to say. "keypair" is the name of
the upstream secp256k1_keypair type, so the code and usage lists would
have this package renaming an API it wraps, and "master" is a branch that
exists here. Enabling them would mean an ignore list spelling out the
vocabulary of the thing being wrapped, which is more to keep true than
the misspellings it would catch.

The upstream hook rather than a local one, pinned by rev like its
neighbours, so pre-commit.ci moves it in the weekly bump; nothing moves
an additional dependency. The vendored secp256k1 tree is not scanned
because pre-commit passes the files tracked here, and the submodule's
are not among them.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>

@giacomocaironi giacomocaironi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve the changes to the repo to bring it up to standards, support the latest python interpreters and libsecp256k1 versions.
However this PR is made up of two main components, the aforementioned ones and the extension of the repo scope, with the addition of a lot more functions and the related tests. These two things are separate, and need to be addressed in two sequential PRs.

Comment thread README.md Resolved
Comment thread SECURITY.md Resolved
Comment thread pyproject.toml Resolved
Comment thread pyproject.toml Resolved
Comment thread pyproject.toml Resolved
Comment thread .secrets.baseline Resolved
Comment thread .gitignore Resolved
Comment thread scripts/README.md Resolved
Comment thread scripts/cffi_build.py Resolved
Comment thread scripts/cffi_build.py Resolved
@giacomocaironi
giacomocaironi marked this pull request as draft August 1, 2026 07:02
@giacomocaironi
giacomocaironi marked this pull request as ready for review August 1, 2026 07:02

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @giacomocaironi, your pull request is larger than the review limit of 150000 diff characters

@giacomocaironi giacomocaironi reopened this Aug 1, 2026
@giacomocaironi
giacomocaironi merged commit 9563ebb into master Aug 1, 2026
130 checks passed
fametrano added a commit that referenced this pull request Aug 5, 2026
wrong measurement

Added tests-passed, Lint and type-check and CodeQL to the required
status checks on master, matching btclib's own rule and its
convention of binding each to an app_id.

The earlier text measured CodeQL's check runs on #43, a pull request
against dev, and found none -- true, and the wrong pull request to
ask: default setup analyzes pushes to the default branch and pull
requests against it, and no ordinary pull request here targets
master. The one that does is dev into master, #21 among them, and
code-scanning/analyses shows CodeQL ran there. That is the pull
request this rule gates, so naming the check names one it produces.
fametrano added a commit that referenced this pull request Aug 6, 2026
wrong measurement

Added tests-passed, Lint and type-check and CodeQL to the required
status checks on master, matching btclib's own rule and its
convention of binding each to an app_id.

The earlier text measured CodeQL's check runs on #43, a pull request
against dev, and found none -- true, and the wrong pull request to
ask: default setup analyzes pushes to the default branch and pull
requests against it, and no ordinary pull request here targets
master. The one that does is dev into master, #21 among them, and
code-scanning/analyses shows CodeQL ran there. That is the pull
request this rule gates, so naming the check names one it produces.
fametrano added a commit that referenced this pull request Aug 6, 2026
wrong measurement

Added tests-passed, Lint and type-check and CodeQL to the required
status checks on master, matching btclib's own rule and its
convention of binding each to an app_id.

The earlier text measured CodeQL's check runs on #43, a pull request
against dev, and found none -- true, and the wrong pull request to
ask: default setup analyzes pushes to the default branch and pull
requests against it, and no ordinary pull request here targets
master. The one that does is dev into master, #21 among them, and
code-scanning/analyses shows CodeQL ran there. That is the pull
request this rule gates, so naming the check names one it produces.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants