Repository navigation
Expr::Not and PhysicalExpr::NotExpr have different metadata propagation rules #27155
Description
Describe the bug
The logical an physical representations of NOT propagate metadata differently.
I'm actually not sure which one is more correct...some metadata uses like "sensor" or "where this value came from" probably are not lying after a NOT operation; however, it's also a little like a function call which would discard its output metadata.
To Reproduce
#[test]
fn not_metadata_differs_between_logical_and_physical() -> Result<()> {
let schema = Schema::new(vec![Field::new("flag", DataType::Boolean, false)]);
let df_schema = DFSchema::try_from(schema.clone())?;
let metadata = FieldMetadata::from(HashMap::from([(
"source".to_string(),
"input".to_string(),
)]));
let expr = Expr::Not(Box::new(
col("flag").alias_with_metadata("flag", Some(metadata)),
));
let logical_field = expr.to_field(&df_schema)?.1;
let physical_expr = create_physical_expr(
&expr,
&df_schema,
&ExecutionProps::new(),
&PhysicalPlanningContext::default(),
)?;
let physical_field = physical_expr.return_field(&schema)?;
// Logical NOT creates a metadata-free Boolean field.
assert_eq!(logical_field.metadata().get("source"), None);
// Physical NotExpr forwards its child's field metadata.
assert_eq!(
physical_field.metadata().get("source").map(String::as_str),
Some("input")
);
Ok(())
}Expected behavior
I'd expect the result to match
Additional context
No response
Willingness to contribute
I would like to contribute a fix for this bug, but request guidance from the DataFusion community
Activity
james-willis commentedon Nov 10, 2026
copying a comment I left on #27143:
paleolimbot commentedon Nov 10, 2026
Yes, to do this properly we need a pluggable "common type" calculation and definition of a cast between two arbitrary types.