Repository navigation
Description
Fuzz tests should have configurable time budgets, be easy to reproduce, and explore new seeds over time. Existing implementations often miss one or more of these properties.
Adopt the centralized FuzzTest trait and driver from #27048 to:
- Make these conventions a required contract for every fuzzer.
- Share repetition, budgets, seed reporting, and replay, keeping implementations simple.
Spill pool will be migrated in the initial PR. The later plan is
- Refactor other existing fuzzing UTs to use the centralized trait and runner
- Require new fuzz tests follow this convention
Tracker
- Add documentation for fuzzing UTs and new trait docs: clarify fuzz testing guidance #27198
The remaining 30 suite groups below cover 152 test definitions, grouped by source file.
-
Aggregation —
async fn test_min() {(source).
Also:test_first_val,test_last_val,test_max,test_sum,test_count,test_median,
streaming_aggregate_test,group_by_strings. -
String DISTINCT counts —
async fn distinct_count_string_test() {(source). -
Equivalence: ordering —
fn test_ordering_satisfy_with_equivalence_random() -> Result<()> {(source).
Also:test_ordering_satisfy_with_equivalence_complex_random,
test_ordering_satisfy_with_casts_and_windows_random. -
Equivalence: projection —
fn project_orderings_random() -> Result<()> {(source).
Also:ordering_satisfy_after_projection_random. -
Equivalence: properties —
fn test_find_longest_permutation_random() -> Result<()> {(source).
Also:test_find_longest_permutation_with_casts_and_windows_random. -
Joins —
async fn test_inner_join_1k_filtered() {(source).
Also:test_inner_join_1k,test_left_join_1k,test_left_join_1k_filtered,test_right_join_1k,
test_right_join_1k_filtered,test_full_join_1k,test_full_join_1k_filtered,test_left_semi_join_1k,
test_left_semi_join_1k_filtered,test_right_semi_join_1k,test_right_semi_join_1k_filtered,
test_left_anti_join_1k,test_left_anti_join_1k_filtered,test_right_anti_join_1k,
test_right_anti_join_1k_filtered,test_left_mark_join_1k,test_left_mark_join_1k_filtered,
test_right_mark_join_1k,test_right_mark_join_1k_filtered,test_inner_join_1k_binary_filtered,
test_inner_join_1k_binary,test_left_join_1k_binary,test_left_join_1k_binary_filtered,
test_right_join_1k_binary,test_right_join_1k_binary_filtered,test_full_join_1k_binary,
test_full_join_1k_binary_filtered,test_left_semi_join_1k_binary,
test_left_semi_join_1k_binary_filtered,test_right_semi_join_1k_binary,
test_right_semi_join_1k_binary_filtered,test_left_anti_join_1k_binary,
test_left_anti_join_1k_binary_filtered,test_right_anti_join_1k_binary,
test_right_anti_join_1k_binary_filtered,test_left_mark_join_1k_binary,
test_left_mark_join_1k_binary_filtered,test_right_mark_join_1k_binary,
test_right_mark_join_1k_binary_filtered,test_inner_join_1k_low_cardinality,
test_inner_join_1k_low_cardinality_filtered,test_left_join_1k_low_cardinality,
test_left_join_1k_low_cardinality_filtered,test_right_join_1k_low_cardinality,
test_right_join_1k_low_cardinality_filtered,test_full_join_1k_low_cardinality,
test_full_join_1k_low_cardinality_filtered,test_filtered_join_spill_fuzz,
test_nested_loop_join_spill_fuzz,fuzz_pwmj_matches_nested_loop. -
Leaf-expression pushdown —
async fn leaf_pushdown_fuzz() {(source).
Also:leaf_pushdown_parquet_fuzz,leaf_pushdown_fuzz_extended. -
Sort / Top-K limits —
async fn test_sort_topk_i32() {(source).
Also:test_sort_topk_f64,test_sort_topk_str,test_sort_topk_i64str. -
Stream merging —
async fn test_merge_2() {(source).
Also:test_merge_2_no_overlap,test_merge_3,test_merge_3_gaps. -
Null-aware joins —
async fn null_aware_join_matches_reference() {(source). -
UTF-8 pruning —
async fn test_utf8_eq() {(source).
Also:test_utf8_not_eq,test_utf8_lt,test_utf8_lt_eq,test_utf8_gt,test_utf8_gt_eq,
test_utf8_like,test_utf8_not_like,test_utf8_like_prefix,test_utf8_like_suffix,
test_utf8_not_like_prefix,test_utf8_not_like_ecsape,test_utf8_not_like_suffix,
test_utf8_not_like_suffix_one. -
ScalarValue conversions —
fn scalar_value_eq_array_consistency() {(source).
Also:scalar_value_iter_to_array_roundtrip,
scalar_value_iter_to_array_rejects_mixed_parameterized_types,scalar_value_to_array_roundtrip. -
Sorting and spilling —
async fn test_sort_10k_mem() {(source).
Also:test_sort_100k_mem,test_sort_strings_100k_mem,test_sort_multi_columns_100k_mem,
test_sort_unlimited_mem. -
Sort-preserving repartition —
async fn stream_merge_multi_order_preserve() -> Result<()> {(source).
Also:sort_preserving_repartition_test. -
Sort queries —
async fn sort_query_fuzzer_runner() {(source). -
Top-K filter pushdown —
async fn test_fuzz_topk_filter_pushdown() {(source). -
Window aggregation —
async fn window_bounded_window_random_comparison() -> Result<()> {(source). -
Timestamp arithmetic —
fn timestamp_op_random_tests() {(source). -
Memory accounting —
fn test_randomized_count_uncount_sequence() {(source). -
LRU cache —
fn test_fuzzy() {(source). -
Grouped accumulation —
fn accumulate_fuzz() {(source). -
Moving min/max —
fn moving_min_tests() -> Result<()> {(source).
Also:moving_max_tests. -
List filtering —
fn list_filter_matches_reference_on_random_lists() {(source). -
Boolean conjunctions —
fn test_conjunction_matches_kleene_and() -> Result<()> {(source). -
Unmarked join pairs —
fn unmarked_pairs_match_brute_force() -> Result<()> {(source). -
Routed hash lookups —
fn routed_lookup_fuzz() -> Result<()> {(source). -
Key-range bitmap union —
fn union_fuzz() {(source). -
Partitioned Top-K —
async fn test_partitioned_topk_matches_bruteforce() -> Result<()> {(source).
Also:test_partitioned_topk_store_bookkeeping_holds_under_stress,
test_partitioned_topk_dense_rank_matches_bruteforce,
test_partitioned_topk_dense_rank_contents_bytes_tracks_recompute,
test_partitioned_topk_rank_bookkeeping_tracks_recompute. -
Symmetric hash joins —
async fn complex_join_all_one_ascending_numeric((source).
Also:join_all_one_ascending_numeric,join_without_sort_information,join_without_filter,
join_all_one_descending_numeric_particular,testing_with_temporal_columns,test_with_interval_columns,
testing_ascending_float_pruning. -
Interval constraint propagation —
integer_float_case_1!(case_1_i32, generate_case_i32, i32, Int32);(source).
Also:case_1_i64,case_1_f64,case_1_f32,case_2_i32,case_2_i64,case_2_f64,case_2_f32,
case_3_i32,case_3_i64,case_3_f64,case_3_f32,case_4_i32,case_4_i64,case_4_f64,
case_4_f32,case_5_i32,case_5_i64,case_5_f64,case_5_f32.
Activity
Awesome06 commentedon Nov 8, 2026
Hi @2010YOUY01, I'd like to work on migrating the Key-range bitmap union fuzz test (
datafusion/physical-plan/src/joins/key_range_bitmap.rs) to theFuzzTesttrait and driver indatafusion-fuzz.test: refactor key-range bitmap union fuzz test to FuzzTest trait
docs: clarify fuzz testing guidance (apache#27198)
Nagato-Yuzuru commentedon Nov 10, 2026
Hi @2010YOUY01, I'm working on #24243 (adding NULLs to fuzz test data). Most of its remaining items are also on this tracker. Like limit, sort, merge, sort-preserving repartition, join, and the old aggregate path.
Would you like me to migrate each one to
FuzzTestas I add NULLs, or keep the two efforts separate? I'd start withlimit_fuzz.rs.If migrating makes sense: these tests use
SessionContextand the DataFrame API, so thedatafusion-fuzzcrate would needdatafusionas a dev-dependency. Is that OK with you?2010YOUY01 commentedon Nov 10, 2026
I think #24243 is a valid improvement, but I don't think expanding the existing internal fuzz tests is the highest-ROI way to improve fuzzing coverage.
My preference is to:
I think this would give us better coverage in the long term, with less duplicated maintenance effort.
Nagato-Yuzuru commentedon Nov 10, 2026
Thanks, that makes sense. I can take the equivalence and limit suites for the migration.
test: refactor key-range bitmap union fuzz test to FuzzTest trait
test: refactor key-range bitmap union fuzz test to FuzzTest trait
test: refactor key-range bitmap union fuzz test to FuzzTest trait (ap…