Support column reference as percentile arg for percentile_cont and approx_percentile_cont

Is your feature request related to a problem or challenge?

percentile_cont and approx_percentile_cont only accept a literal for the percentile argument (see here).

We could extend support for column reference or projections as well.

Describe the solution you'd like

Add support for ColumnarValue::Array for both percentile_cont and approx_percentile_cont.

Describe alternatives you've considered

No response

Additional context

No response

take

what would be the behaviour if it supports ColumnarValue::Array for the percentile? or is this assuming the array is len 1 anyway, akin to a scalar?

@Jefffrey

@Jefffrey We're hitting an issue when the percentile comes from a projection. The behavior would be the same as for Trino: the percentile argument must be constant for all inputs. So yes, essentially it would be a ScalarValue (not always folded into a ScalarValue).

does that seem reasonable to you?

datafusion/datafusion/functions-aggregate/src/utils.rs

Lines 30 to 48 in 04ca5ac

    
           pub(crate) fn get_scalar_value(expr: &Arc<dyn PhysicalExpr>) -> Result<ScalarValue> { 
        
               let empty_schema = Arc::new(Schema::empty()); 
        
               let batch = RecordBatch::new_empty(Arc::clone(&empty_schema)); 
        
               if let ColumnarValue::Scalar(s) = expr.evaluate(&batch)? { 
        
                   Ok(s) 
        
               } else { 
        
                   internal_err!("Didn't expect ColumnarValue::Array") 
        
               } 
        
           } 
        
           /// Validates that a percentile expression is a literal float value between 0.0 and 1.0. 
        
           /// 
        
           /// Used by both `percentile_cont` and `approx_percentile_cont` to validate their 
        
           /// percentile parameters. 
        
           pub(crate) fn validate_percentile_expr( 
        
               expr: &Arc<dyn PhysicalExpr>, 
        
               fn_name: &str, 
        
           ) -> Result<f64> { 
        
               let scalar_value = get_scalar_value(expr).map_err(|_e| {

so is this about fixing line 33 here to potentially accept a ColumnarValue::Array with length 1? Since for some reason its not coming as a ColumnarValue::Scalar?

@Jefffrey

@Jefffrey Not exactly: this is not sufficient because you can have empty batches and then would not be able to resolve the percentile argument.

feat: support col reference for percentiles (apache#25337)

Navigation Menu

Repository navigation

Support column reference as percentile arg for `percentile_cont` and `approx_percentile_cont` #24337

Is your feature request related to a problem or challenge?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	pub(crate) fn get_scalar_value(expr: &Arc<dyn PhysicalExpr>) -> Result<ScalarValue> {
	let empty_schema = Arc::new(Schema::empty());
	let batch = RecordBatch::new_empty(Arc::clone(&empty_schema));
	if let ColumnarValue::Scalar(s) = expr.evaluate(&batch)? {
	Ok(s)
	} else {
	internal_err!("Didn't expect ColumnarValue::Array")
	}
	}

	/// Validates that a percentile expression is a literal float value between 0.0 and 1.0.
	///
	/// Used by both `percentile_cont` and `approx_percentile_cont` to validate their
	/// percentile parameters.
	pub(crate) fn validate_percentile_expr(
	expr: &Arc<dyn PhysicalExpr>,
	fn_name: &str,
	) -> Result<f64> {
	let scalar_value = get_scalar_value(expr).map_err(\|_e\| {

Support column reference as percentile arg for percentile_cont and approx_percentile_cont #24337

Description

Is your feature request related to a problem or challenge?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Activity

dd-annarose commented on Aug 13, 2026

Jefffrey commented on Aug 20, 2026

dd-annarose commented on Sep 11, 2026

Jefffrey commented on Sep 15, 2026

dd-annarose commented on Sep 15, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Support column reference as percentile arg for `percentile_cont` and `approx_percentile_cont` #24337