Guard against integer overflow in index deserialization storage-size checks#5313
Closed
AgentGymLeader wants to merge 1 commit into
Closed
Guard against integer overflow in index deserialization storage-size checks#5313AgentGymLeader wants to merge 1 commit into
AgentGymLeader wants to merge 1 commit into
Conversation
…checks The flat-index, binary-flat, and quantizer readers validated codes.size() == ntotal * code_size (and final_graph.size() == ntotal * K for NNDescent) with unchecked size_t multiplications that could wrap for a crafted large ntotal, causing the check to pass while ntotal is inconsistent with the actual backing storage. Now uses mul_no_overflow() at all 9 sites to reject overflowing products and throw instead of silently accepting a malformed index, matching the Array/Block inverted-list reader hardening already present in this file (22 existing call sites -> 31).
Contributor
|
@alibeklfc has imported this pull request. If you are a Meta employee, you can view this in D108628013. |
Contributor
|
@alibeklfc merged this pull request in 6f62d01. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Several index deserialization readers validate the backing store with
<vector>.size() == ntotal * element_size, where the multiplication is performed insize_tand is unchecked.read_index_headeraccepts any non-negativentotal(it boundsdbut notntotal), so a crafted index can makentotal * code_sizewrap modulo 2^64 to a small value. The equality check then passes against a small backing vector, and an internally inconsistent index object (hugentotal, tiny backing storage) is accepted.Downstream, code that trusts
ntotalreads past the backing vector — e.g.IndexFlat::reconstruct(key, ...)checks onlykey < ntotalbeforememcpy(recons, &codes[key * code_size], code_size), and the IVF/flat scan loops index intocodes/xbbyi * code_size. The result is an out-of-bounds read (crash / DoS, potential information disclosure) when an application loads an untrusted index file.Fix
Wrap the
ntotal * element_sizemultiplications in the existingmul_no_overflowhelper (faiss/impl/FaissAssert.h), which throws on overflow instead of wrapping. This brings the flat / binary-flat / quantizer readers and the NNDescentfinal_graphcheck to parity with the Array/Block inverted-list readers in this same file, which already usemul_no_overflowfor their deserialization size checks. Well-formed indexes are unaffected — the guard only rejects inputs whose declared sizes overflow.Sites hardened (
faiss/impl/index_read.cpp): IndexFlat, IndexLSH, IndexPQ, IndexResidualQuantizer, IndexLocalSearchQuantizer, IndexProductResidualQuantizer, IndexProductLocalSearchQuantizer, IndexBinaryFlat, and NNDescent. Bare== ntotal/== ntotal + 1checks (no multiplication) are left unchanged.Verification
mul_no_overflowandFAISS_THROW_IF_NOTare already used throughout this file; the change follows the existing call pattern.ntotal * element_sizewould overflowsize_t.