Skip to content

Add support for ignore nulls for LEAD, LAG in WindowAggExec. #9456

@mustafasrepo

Description

@mustafasrepo

Is your feature request related to a problem or challenge?

Currently LEAD and LAG window functions have IGNORE NULLS support when they are calculated from BoundedWindowAggExec. However, when they are calculated from WindowAggExec they don't have this support.

Currently datafusion gives error: IGNORE NULLS mode for LAG and LEAD is not supported for WindowAggExec when WindowAggExec is used with ignore nulls option.

As an example test below in the window.slt file

select lag(a, 2, null) ignore nulls over (order by id desc) as x1,
       lag(a, 2, 'def') ignore nulls over (order by id desc) as x2,
       lag(a, 2, null) respect nulls over (order by id desc) as x4,
       lag(a, 2, 'def') respect nulls over (order by id desc) as x5,
       sum(id) over (order by id desc ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) as sum_id
from (select 2 id, 'b' a union all select 1 id, null a union all select 3 id, null union all select 4 id, 'x')

triggers this error.

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

The error is triggered at evaluate_all method of the WindowShiftEvaluator. This implementation should be updated to support this feature.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions