Skip to content

fix(ofp): accept empty-recipient HMAC so bootstrap_peers can connect#6330

Merged
houko merged 3 commits into
mainfrom
fix-ofp-bootstrap-hmac
Jun 26, 2026
Merged

fix(ofp): accept empty-recipient HMAC so bootstrap_peers can connect#6330
houko merged 3 commits into
mainfrom
fix-ofp-bootstrap-hmac

Conversation

@houko

@houko houko commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Problem

Two LibreFang nodes on the same network, each with a matching [network] shared_secret and the other's address in bootstrap_peers, can never connect. Both sides log, symmetrically:

WARN  OFP: failed to connect to bootstrap peer error=Handshake failed: Remote error 403: HMAC authentication failed
DEBUG OFP: inbound connection ended: Handshake failed: HMAC verification failed on incoming Handshake

No configuration change fixes it — not ports, mDNS, shared_secret, or a WireGuard/Netbird overlay. The handshake is reached; HMAC verification is what fails.

Root cause (regression from #3920)

#3920 bound the recipient node ID into the handshake HMAC so a captured handshake cannot be replayed against a different federation node sharing the same shared_secret:

auth_data = nonce | sender_node_id | recipient_node_id

The bootstrap dialer cannot satisfy that binding. bootstrap_peers is a list of addresses, and each node's node_id is derived from its persisted Ed25519 keypair (key_mgr.load_or_generate() in background_lifecycle.rs) — unique per machine, not user-settable. So the kernel dials each bootstrap entry via PeerNode::connect_to_peer, which hardcodes the recipient to the empty string:

// peer.rs
pub async fn connect_to_peer(&self, addr, handle) -> Result<(), WireError> {
    self.connect_to_peer_with_id(addr, handle, "").await   // recipient = ""
}

The receiver, however, verified the HMAC against its own real node ID with no fallback for the empty form:

// peer.rs inbound handshake (pre-fix)
let expected_data = format!("{}|{}|{}", nonce, node_id, node.config.node_id);
if !hmac_verify(&node.config.shared_secret, expected_data.as_bytes(), auth_hmac) { /* 403 */ }

HMAC(nonce|sender|"") never equals HMAC(nonce|sender|<receiver id>), so every bootstrap handshake 403s, in both directions. Commit 9ccf2a42 (#3920) introduced both the empty-recipient bootstrap call and the strict receiver check together, and every test used connect_to_peer_with_id with a known recipient, so the real bootstrap path was never exercised.

Fix

The inbound handshake accepts either binding:

  1. the recipient-bound form (nonce|sender|<our node id>) — the strong anti-replay path, used by send_to_peer once the peer's node ID is known; or
  2. the empty bootstrap form (nonce|sender|) — for an initial bootstrap dial that cannot know our node ID.

Whichever form authenticated is threaded into the subsequent Ed25519 identity verification (the client signed that same auth_data). The remaining authentication layers still fully cover the bootstrap path:

A wrong, non-empty recipient is still rejected, so the #3875 cross-node replay protection is preserved for the established-link path.

Changes

  • crates/librefang-wire/src/peer.rs: inbound handshake HMAC verification accepts the empty-recipient (bootstrap) binding as a fallback to the recipient-bound form; the matched auth_data is reused for Ed25519 identity verification. Doc comment on connect_to_peer updated to describe the accepted bootstrap binding.

Verification

  • cargo test -p librefang-wire --lib69 passed, 0 failed, including:
    • issue_3920_bootstrap_connect_without_recipient_id_succeeds (new) — dials through the real connect_to_peer entry point and asserts the handshake completes in both directions.
    • issue_3920_wrong_nonempty_recipient_still_rejected (new) — a wrong non-empty recipient is still refused.
    • test_handshake_hmac_includes_recipient, issue_3873_*, issue_4269_* — existing replay / identity / KEX coverage unchanged.
  • cargo clippy -p librefang-wire --all-targets -- -D warnings — clean.
  • rustfmt --check — clean.

(Built and tested in the sanctioned Linux dev container; the change is platform-independent.)

Review follow-up (commit 2)

  • Added issue_3920_identity_bootstrap_connect_succeeds_and_pins: the HMAC-only test above uses identity-less nodes (PeerNode::start carries no keypair), so it never exercised the line that threads the matched auth_data into Ed25519 identity verification — the production default, where every node loads a load_or_generate keypair. The new test dials two identity-bearing nodes through the empty-recipient connect_to_peer and asserts both sides pin each other's pubkey. Verified non-vacuous by mutation: feeding the bound form into identity verification makes this test fail with Ed25519 identity signature invalid while the HMAC-only test still passes.
  • Recorded the fix under CHANGELOG.md [Unreleased].
  • Full librefang-wire lib suite: 70 passed, 0 failed; cargo clippy -p librefang-wire --all-targets -- -D warnings clean; rustfmt --check clean.

Out of scope (separate observation, not fixed here)

[network] mdns_enabled is a dead config knob: it is referenced only in config struct / default / API display / validation, with no mDNS discovery implementation anywhere in crates/ (no mdns-sd, ServiceDaemon, or multicast dial). Toggling it has no effect on peering — the only discovery/dial path is bootstrap_peers. Implementing mDNS auto-discovery is feature work in the kernel mesh subsystem and belongs in its own issue/PR.

#3920 bound recipient_node_id into the handshake HMAC so a captured handshake cannot be replayed against a different federation node that shares the same shared_secret.
But the bootstrap dialer (`connect_to_peer`) cannot know the remote node ID in advance — `bootstrap_peers` lists addresses, not identities — so it signs an empty recipient, while the receiver verified the HMAC against its own real node ID.
Every bootstrap connection therefore failed with `403 HMAC authentication failed` in both directions.
No existing test caught it because all of them used `connect_to_peer_with_id` with a known recipient; nothing exercised the real bootstrap path.

The inbound handshake now accepts either the recipient-bound form (the strong anti-replay path, used by `send_to_peer` once the peer's node ID is known) or the empty bootstrap form, and threads whichever matched into the Ed25519 identity verification.
The other authentication layers still cover the bootstrap path — nonce dedup (#3880), Ed25519 TOFU identity pinning (#3873), and the per-handshake ECDH session key (#4269) — so accepting an unbound recipient does not let an attacker who lacks the X25519 ephemeral private key actually communicate.

Adds `issue_3920_bootstrap_connect_without_recipient_id_succeeds`, which dials through the real `connect_to_peer` entry point and asserts the handshake completes in both directions, and `issue_3920_wrong_nonempty_recipient_still_rejected`, which confirms a wrong non-empty recipient is still refused so the #3875 cross-node replay protection is preserved.
@github-actions github-actions Bot added size/M 50-249 lines changed area/security Security systems and auditing labels Jun 26, 2026
claude and others added 2 commits June 26, 2026 05:36
Review of #6330 found the HMAC-only bootstrap regression test used identity-less nodes (`PeerNode::start` passes no keypair), so it never exercised the line that threads the matched `auth_data` into Ed25519 identity verification — the production default, where every node carries a `load_or_generate` keypair.

Adds `issue_3920_identity_bootstrap_connect_succeeds_and_pins`: two identity-bearing nodes complete a bootstrap (empty-recipient) handshake and pin each other's pubkey.
A mutation that feeds the bound form into identity verification makes this test fail with "Ed25519 identity signature invalid" while the HMAC-only test still passes, confirming the new test is non-vacuous.

Records the fix under CHANGELOG `[Unreleased]`.
@github-actions github-actions Bot added the area/docs Documentation and guides label Jun 26, 2026
@houko
houko enabled auto-merge (squash) June 26, 2026 06:09
@houko
houko merged commit 359a97c into main Jun 26, 2026
33 checks passed
@houko
houko deleted the fix-ofp-bootstrap-hmac branch June 26, 2026 06:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs Documentation and guides area/security Security systems and auditing size/M 50-249 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants