Skip to content

Substrait roundtrip fails for Sort with a fetch #13860

@robtandy

Description

@robtandy

Describe the bug

Substrait round trip fails for a query that produces a logical plan where the Sort node includes a fetch.

As an example,

Sort: data.b ASC NULLS LAST, fetch=2
  Filter: data.a > Int64(1)
    TableScan: data projection=[a, b, c, d, e, f], partial_filters=[data.a > Int64(1)]

Upon returning from the round trip produces

Sort: data.b ASC NULLS LAST
  Filter: data.a > Int64(1)
    TableScan: data projection=[a, b, c, d, e, f], partial_filters=[data.a > Int64(1)]

the fetch is lost.

I'm happy to take and fix this issue, but I wasn't sure where the fix should be made. Should this fix the Sort Rel in substrait that cannot hold a fetch member? Or should this be a fix in the substrait producer to produce a plan with a Limit and a Sort?

To Reproduce

Add a test to datafusion/substrait/tests/cases/round_trip_logical_plan.rs such as

#[tokio::test]
async fn select_with_filter_sort_limit() -> Result<()> {
    roundtrip("SELECT * FROM data WHERE a > 1 ORDER BY b ASC LIMIT 2").await
}

in this repo, run

cd datafusion/substrait
 cargo test -- --nocapture select_with_filter_sort_limit

Expected behavior

A round trip for this plan should work.

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions