Repository navigation
Fuzzing Crash: MaskedArray canonical dtype mismatch during comparison #5989
Description
Fuzzing Crash Report
Analysis
Crash Location: vortex-array/src/vtable/dyn_.rs:218 (in execute_canonical function)
Error Message:
compare operation should succeed in fuzz test:
Executed canonical dtype mismatch for vortex.masked
Stack Trace:
0: std::backtrace_rs::backtrace::libunwind::trace
1: std::backtrace_rs::backtrace::trace_unsynchronized
2: <std::backtrace::Backtrace>::create
3: execute_canonical<vortex_array::arrays::masked::vtable::MaskedVTable>
at ./vortex-array/src/vtable/dyn_.rs:216:13
4: execute
at ./vortex-array/src/executor.rs:99:35
5: execute<vortex_array::canonical::Canonical>
at ./vortex-array/src/executor.rs:43:9
6: execute
at ./vortex-array/src/executor.rs:129:36
7: execute<vortex_array::arrays::primitive::array::PrimitiveArray>
at ./vortex-array/src/executor.rs:43:9
8: execute_decompress
at ./encodings/alp/src/alp/decompress.rs:88:31
9: execute
at ./encodings/alp/src/alp/array.rs:189:33
10: execute_canonical<vortex_alp::alp::array::ALPVTable>
at ./vortex-array/src/vtable/dyn_.rs:208:22
Root Cause:
The fuzzer discovered a dtype mismatch bug in the MaskedArray canonicalization path. When executing the canonical form of a MaskedArray, the dtype assertion at vortex-array/src/vtable/dyn_.rs:216-220 fails:
vortex_ensure!(
result.as_ref().dtype() == array.dtype(),
"Executed canonical dtype mismatch for {}",
self.id()
);The issue occurs when:
- A ChunkedArray with F64 Nullable dtype containing ALP-encoded data goes through file I/O
- During comparison operations, the array needs to be canonicalized
- The ALP encoding decompresses, involving a MaskedArray somewhere in the execution path
- The MaskedArray's canonical form returns a dtype that doesn't match the original MaskedArray's dtype
Looking at vortex-array/src/arrays/masked/vtable/canonical.rs:15-30, the canonicalization logic:
- Unwraps constant arrays and applies masking
- Or calls
masked_child()which applies an inverted mask via themaskcompute operation
The dtype mismatch likely occurs because:
- The
mask()operation or the child canonicalization changes the nullability - The MaskedArray's dtype should be nullable, but the result is non-nullable (or vice versa)
The test array has:
- ChunkedArray with F64 Nullable dtype
- Multiple chunks with AllInvalid validity (all null values)
- After compression, contains ALP-encoded data
Debug Output
FuzzFileAction {
array: ChunkedArray {
dtype: Primitive(
F64,
Nullable,
),
len: 5,
chunk_offsets: PrimitiveArray {
dtype: Primitive(
U64,
NonNullable,
),
buffer: BufferHandle(
Host(
Buffer<u8> {
length: 32,
alignment: Alignment(
8,
),
as_slice: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...],
},
),
),
validity: NonNullable,
stats_set: ArrayStats {
inner: RwLock {
data: StatsSet {
values: [],
},
},
},
},
chunks: [
PrimitiveArray {
dtype: Primitive(
F64,
Nullable,
),
buffer: BufferHandle(
Host(
Buffer<u8> {
length: 0,
alignment: Alignment(
8,
),
as_slice: [],
},
),
),
validity: AllInvalid,
stats_set: ArrayStats {
inner: RwLock {
data: StatsSet {
values: [],
},
},
},
},
PrimitiveArray {
dtype: Primitive(
F64,
Nullable,
),
buffer: BufferHandle(
Host(
Buffer<u8> {
length: 8,
alignment: Alignment(
8,
),
as_slice: [255, 255, 255, 255, 1, 0, 0, 0],
},
),
),
validity: AllInvalid,
stats_set: ArrayStats {
inner: RwLock {
data: StatsSet {
values: [],
},
},
},
},
PrimitiveArray {
dtype: Primitive(
F64,
Nullable,
),
buffer: BufferHandle(
Host(
Buffer<u8> {
length: 32,
alignment: Alignment(
8,
),
as_slice: [211, 255, 255, 255, 255, 255, 255, 223, 191, 191, 191, 189, 191, 191, 191, 191, ...],
},
),
),
validity: AllValid,
stats_set: ArrayStats {
inner: RwLock {
data: StatsSet {
values: [],
},
},
},
},
],
stats_set: ArrayStats {
inner: RwLock {
data: StatsSet {
values: [],
},
},
},
},
projection_expr: None,
filter_expr: None,
compressor_strategy: Default,
}
Summary
- Target:
file_io - Crash File:
crash-9de4b2cde8f6f3b96538b8c6ce2b47c200992295 - Branch: develop
- Commit: 4bbafe7
- Crash Artifact: https://github.com/vortex-data/vortex/actions/runs//artifacts/
Reproduction
-
Download the crash artifact:
- Direct download: https://github.com/vortex-data/vortex/actions/runs//artifacts/
- Or find
fuzzer-crashes-file_ioat: https://github.com/vortex-data/vortex/actions/runs/ - Extract the zip file
-
Reproduce locally:
# The artifact contains file_io/crash-9de4b2cde8f6f3b96538b8c6ce2b47c200992295
cargo +nightly fuzz run -D --sanitizer=none file_io file_io/crash-9de4b2cde8f6f3b96538b8c6ce2b47c200992295 -- -rss_limit_mb=0- Get full backtrace:
RUST_BACKTRACE=full cargo +nightly fuzz run -D --sanitizer=none file_io file_io/crash-9de4b2cde8f6f3b96538b8c6ce2b47c200992295 -- -rss_limit_mb=0Auto-created by fuzzing workflow with Claude analysis
Activity
Fix: masked dtype mismatch (#6008)
Fix: masked dtype mismatch (#6008)