Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: apache/datafusion
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 50.2.0
Choose a base ref
...
head repository: apache/datafusion
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 50.3.0
Choose a head ref
  • 9 commits
  • 43 files changed
  • 11 contributors

Commits on Oct 18, 2025

  1. [branch-50] chore: Fix no space left on device (#18141)

    ## Which issue does this PR close?
    
    <!--
    We generally require a GitHub issue to be filed for all bug fixes and
    enhancements and this helps us generate change logs for our releases.
    You can link an issue to this PR using the GitHub syntax. For example
    `Closes #123` indicates that this PR will close issue #123.
    -->
    
    - Closes #18135 
    ## Rationale for this change
    
    <!--
    Why are you proposing this change? If this is already explained clearly
    in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand
    your changes and offer better suggestions for fixes.
    -->
    
    ## What changes are included in this PR?
    
    <!--
    There is no need to duplicate the description in the issue here but it
    is sometimes worth providing a summary of the individual changes in this
    PR.
    -->
    
    ## Are these changes tested?
    
    <!--
    We typically require tests for all PRs in order to:
    1. Prevent the code from being accidentally broken by subsequent changes
    2. Serve as another way to document the expected behavior of the code
    
    If tests are not included in your PR, please explain why (for example,
    are they covered by existing tests)?
    -->
    
    ## Are there any user-facing changes?
    
    <!--
    If there are user-facing changes then we may require documentation to be
    updated before approving the PR.
    -->
    
    <!--
    If there are any breaking changes to public APIs, please add the `api
    change` label.
    -->
    
    ---------
    
    Co-authored-by: Andrew Lamb <[email protected]>
    comphead and alamb authored Oct 18, 2025
    Configuration menu
    Copy the full SHA
    28ad4ef View commit details
    Browse the repository at this point in the history
  2. [branch-50]: chore: cherry pick concat to 50.3.0 (#18128)

    ## Which issue does this PR close?
    
    <!--
    We generally require a GitHub issue to be filed for all bug fixes and
    enhancements and this helps us generate change logs for our releases.
    You can link an issue to this PR using the GitHub syntax. For example
    `Closes #123` indicates that this PR will close issue #123.
    -->
    
    - Closes #.
    
    ## Rationale for this change
    
    <!--
    Why are you proposing this change? If this is already explained clearly
    in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand
    your changes and offer better suggestions for fixes.
    -->
    
    ## What changes are included in this PR?
    
    <!--
    There is no need to duplicate the description in the issue here but it
    is sometimes worth providing a summary of the individual changes in this
    PR.
    -->
    
    ## Are these changes tested?
    
    <!--
    We typically require tests for all PRs in order to:
    1. Prevent the code from being accidentally broken by subsequent changes
    2. Serve as another way to document the expected behavior of the code
    
    If tests are not included in your PR, please explain why (for example,
    are they covered by existing tests)?
    -->
    
    ## Are there any user-facing changes?
    
    <!--
    If there are user-facing changes then we may require documentation to be
    updated before approving the PR.
    -->
    
    <!--
    If there are any breaking changes to public APIs, please add the `api
    change` label.
    -->
    
    ---------
    
    Co-authored-by: Andrew Lamb <[email protected]>
    comphead and alamb authored Oct 18, 2025
    Configuration menu
    Copy the full SHA
    ce4eb18 View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2025

  1. [branch-50] Backport Fix bug in LimitPushPastWindows (#18029) (#18107)

    ## Which issue does this PR close?
    
    - Related to  #18072.
    - Backports #18029
    
    ## Rationale for this change
    
    Return correct results
    
    ## What changes are included in this PR?
    
    A fix for the PushPastLimits rule to accommodate the special needs of
    `LEAD()`
    - Backport of #18029
    
    ## Are these changes tested?
    
    An 800 line file of slts was added. There are never enough.
    
    ## Are there any user-facing changes?
    
    Queries using `lead()` with a `limit` should return correct results
    again (but also go fast)
    avantgardnerio authored Oct 19, 2025
    Configuration menu
    Copy the full SHA
    d554f1c View commit details
    Browse the repository at this point in the history
  2. [branch-50] fix(SubqueryAlias): use maybe_project_redundant_column (#…

    …17478) (#18130)
    
    ## Which issue does this PR close?
    
    <!--
    We generally require a GitHub issue to be filed for all bug fixes and
    enhancements and this helps us generate change logs for our releases.
    You can link an issue to this PR using the GitHub syntax. For example
    `Closes #123` indicates that this PR will close issue #123.
    -->
    
    - Related to #17405
    - Related to #18072
    
    ## Rationale for this change
    
    <!--
    Why are you proposing this change? If this is already explained clearly
    in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand
    your changes and offer better suggestions for fixes.
    -->
    
    See #17478
    
    ## What changes are included in this PR?
    
    <!--
    There is no need to duplicate the description in the issue here but it
    is sometimes worth providing a summary of the individual changes in this
    PR.
    -->
     
    See #17478
    
    ## Are these changes tested?
    
    <!--
    We typically require tests for all PRs in order to:
    1. Prevent the code from being accidentally broken by subsequent changes
    2. Serve as another way to document the expected behavior of the code
    
    If tests are not included in your PR, please explain why (for example,
    are they covered by existing tests)?
    -->
    
    ## Are there any user-facing changes?
    
    <!--
    If there are user-facing changes then we may require documentation to be
    updated before approving the PR.
    -->
    
    <!--
    If there are any breaking changes to public APIs, please add the `api
    change` label.
    -->
    
    Co-authored-by: Filippo Rossi <[email protected]>
    Co-authored-by: Andrew Lamb <[email protected]>
    3 people authored Oct 19, 2025
    Configuration menu
    Copy the full SHA
    5226650 View commit details
    Browse the repository at this point in the history
  3. [branch-50]: fix: Add overflow checks to SparkDateAdd/Sub to avoid pa…

    …nics (#18013) (#18131)
    
    ## Which issue does this PR close?
    
    <!--
    We generally require a GitHub issue to be filed for all bug fixes and
    enhancements and this helps us generate change logs for our releases.
    You can link an issue to this PR using the GitHub syntax. For example
    `Closes #123` indicates that this PR will close issue #123.
    -->
    
    - Related to apache/datafusion-comet#2539
    - Related to #18072
    
    ## Rationale for this change
    
    <!--
    Why are you proposing this change? If this is already explained clearly
    in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand
    your changes and offer better suggestions for fixes.
    -->
    
    Return errors rather than panicking.
    
    ## What changes are included in this PR?
    
    <!--
    There is no need to duplicate the description in the issue here but it
    is sometimes worth providing a summary of the individual changes in this
    PR.
    -->
    
    ## Are these changes tested?
    
    <!--
    We typically require tests for all PRs in order to:
    1. Prevent the code from being accidentally broken by subsequent changes
    2. Serve as another way to document the expected behavior of the code
    
    If tests are not included in your PR, please explain why (for example,
    are they covered by existing tests)?
    -->
    
    ## Are there any user-facing changes?
    
    <!--
    If there are user-facing changes then we may require documentation to be
    updated before approving the PR.
    -->
    
    <!--
    If there are any breaking changes to public APIs, please add the `api
    change` label.
    -->
    
    Co-authored-by: Andy Grove <[email protected]>
    Co-authored-by: Andrew Lamb <[email protected]>
    3 people authored Oct 19, 2025
    Configuration menu
    Copy the full SHA
    290e3bb View commit details
    Browse the repository at this point in the history
  4. [branch-50] Fix ambiguous column names in substrait conversion #17299 (

    …#18077)
    
    ## Which issue does this PR close?
    
    - Related to #18072 
    - Related to #17294
    - Backports #17299 to branch-50
    
    
    ## Rationale for this change
    
    Fix ambiguous column names in substrait conversion as a result of
    literals before and after a join being assigned the same name.
    
    More information in the issue, but say you have a NULL literal before a
    join called "column1" and then you create a new NULL column after the
    join called column2, you would get an ambiguous column name errors like.
    
    ```
    Error: SchemaError(AmbiguousReference { field: Column { relation: Some(Bare { table: "left" }), name: "UTF8(NULL)" } }, Some(""))
    ```
    
    
    ## What changes are included in this PR?
    
    Simply alias all literals as they're converted to have a UUID name.
    
    ## Are these changes tested?
    
    Yes.
    Tested by using substrait-java with this query
    
    ```
    ./isthmus-cli/build/graal/isthmus --create "create table A (a int); create table B (a int, c int); create table C (a int, d int)" "select t.*, C.d, CAST(NULL AS VARCHAR) as e from (select a, CAST(NULL AS VARCHAR) as c from A UNION ALL select a, c from B) t LEFT JOIN C ON t.a = C.a"
    ```
    
    ## Are there any user-facing changes?
    
    Co-authored-by: Xander <[email protected]>
    Co-authored-by: Xander Bailey <[email protected]>
    Co-authored-by: Andrew Lamb <[email protected]>
    4 people authored Oct 19, 2025
    Configuration menu
    Copy the full SHA
    e3f8e37 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2025

  1. [branch-50] Extend datatype semantic equality check to include timest…

    …amps (#17777) (#18129)
    
    ## Which issue does this PR close?
    
    <!--
    We generally require a GitHub issue to be filed for all bug fixes and
    enhancements and this helps us generate change logs for our releases.
    You can link an issue to this PR using the GitHub syntax. For example
    `Closes #123` indicates that this PR will close issue #123.
    -->
    
    - Related to #17776
    - Related to #18072
    
    ## Rationale for this change
    
    <!--
    Why are you proposing this change? If this is already explained clearly
    in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand
    your changes and offer better suggestions for fixes.
    -->
    
    If I've understood semantic equality correctly, any two timestamps
    should meet the bar for equality regardless of time units and timezones,
    but the current code doesn't reflect that.
    
    ## What changes are included in this PR?
    
    <!--
    There is no need to duplicate the description in the issue here but it
    is sometimes worth providing a summary of the individual changes in this
    PR.
    -->
    
    Adds a branch to this method for timestamps.
    
    ## Are these changes tested?
    
    <!--
    We typically require tests for all PRs in order to:
    1. Prevent the code from being accidentally broken by subsequent changes
    2. Serve as another way to document the expected behavior of the code
    
    If tests are not included in your PR, please explain why (for example,
    are they covered by existing tests)?
    -->
    
    Yes
    
    ## Are there any user-facing changes?
    
    <!--
    If there are user-facing changes then we may require documentation to be
    updated before approving the PR.
    -->
    
    Yes
    
    <!--
    If there are any breaking changes to public APIs, please add the `api
    change` label.
    -->
    
    ---------
    
    Co-authored-by: Shiv Bhatia <[email protected]>
    Co-authored-by: Shiv Bhatia <[email protected]>
    Co-authored-by: Andrew Lamb <[email protected]>
    4 people authored Oct 20, 2025
    Configuration menu
    Copy the full SHA
    ade5232 View commit details
    Browse the repository at this point in the history
  2. [branch-50] perf: Fix NLJ slow join with condition array_has (#18161)…

    … (#18179)
    
    ## Which issue does this PR close?
    
    <!--
    We generally require a GitHub issue to be filed for all bug fixes and
    enhancements and this helps us generate change logs for our releases.
    You can link an issue to this PR using the GitHub syntax. For example
    `Closes #123` indicates that this PR will close issue #123. -->
    
    - Related to #18070
    - Part of #18072
    
    ## Rationale for this change
    
    Fix performance regression in Datafusion 50
    
    ## What changes are included in this PR?
    
    Backport #18161 to `branch-50`
    
    ## Are these changes tested?
    
    Yes
    ## Are there any user-facing changes?
    
    Fix performance regression
    
    Co-authored-by: Yongting You <[email protected]>
    alamb and 2010YOUY01 authored Oct 20, 2025
    Configuration menu
    Copy the full SHA
    0c62e84 View commit details
    Browse the repository at this point in the history
  3. [branch-50] Prepare 50.3.0 release version number and README (#18173)

    ## Which issue does this PR close?
    
    - part of #18072
    
    ## Rationale for this change
    
    Get read for release
    
    ## What changes are included in this PR?
    
    release version number and README
    
    ## Are these changes tested?
    By CI
    
    ## Are there any user-facing changes?
    
    New version
    alamb authored Oct 20, 2025
    Configuration menu
    Copy the full SHA
    7909909 View commit details
    Browse the repository at this point in the history
Loading