Repository navigation
Commit c48d4ed
Projection Pushdown (#211)
* [GITHUB ACTION] Refactor for license and actions (#148)
* Delete datafusion main publication
* Adding licence information, refactoring prunibility issues
* Update SYNNADA-CONTRIBUTIONS.txt
* Update rat_exclude_files.txt
* Enhanced Pipeline Execution: Now Supporting Complex Query Plans for Improved Performance (#132)
* Very initial test passing algorithm
* Working except a minor bug in interval calculations
* After clippy
* Plan
* initial implemantation
* Before prune check ability is added.
Order equivalence implementations will vanish after we send a seperate PR
* minor changes
* Fix bug, ordering equivalence random head
* minor changes
* Add ordering equivalence for sort merge join
* Improvement on tests
* Upstream changes
* Add ordering equivalence for sort merge join
* Fmt issues
* Update comment
* Add ordering equivalence support for hash join
* Make 1 file
* Code enhancements/comment improvements
* Add projection cast handling
* Fix output ordering for sort merge join
* projection bug fix
* Minor changes
* minor changes
* simplify sort_merge_join
* Update equivalence implementation
* Update test_utils.rs
* Update cast implementation
* More idiomatic code
* After merge
* Comments visisted
* Add key swap according to the children orders
* Refactoring
* After merge refactor
* Update sort_enforcement.rs
* Update datafusion/core/src/physical_optimizer/join_selection.rs
Co-authored-by: Mustafa Akur <[email protected]>
* Comments are applied
* Feature/determine prunability (#139)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
* Determine prunability of tables for join operations (#90)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* fix the tables' unboundedness
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* Comment improvements and minor code improvements
* Splitting the order based join selection
* Update rat_exclude_files.txt
* Revert "Feature/determine prunability (#139)"
This reverts commit cf56105.
* Commented
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Berkay Şahin <[email protected]>
* Bug fix: Fix lexicographical column search among provided ordering (#156)
* License update (#157)
This extension adds Synnada license information to the existing one.
* Sliding Nested Join Algorithm (#142)
* Sliding Hash Join Algorithm (SWHJ) (#147)
* Fix errors introduced during rebase
* Keep Track of Global Ordering Requirement (#165)
* Prunability of Join Filter Physical Expressions (#161)
* BinaryExpr Equivalence (#116)
* Fix errors introduced during rebase
* Support multiple ordered columns on joins and expression graph (#163)
* SlidingHashJoin and SlidingNestedLoopJoin planner integration (#171)
* Add license, add contribution hash commits, minor changes, add scripts to automate hash generation,Delete docs yaml file
* Resolve errors introduced during rebase
* Synnada Streaming SQL Tests (#190)
* Adds a new method to construct window function for the given input
* Protobuf implementations with roundrobin tests (#193)
* Protobuf implementations with roundrobin
* Proto
* Update mod.rs
* [GITHUB ACTION] Refactor for license and actions (#148)
* Delete datafusion main publication
* Adding licence information, refactoring prunibility issues
* Update SYNNADA-CONTRIBUTIONS.txt
* Update rat_exclude_files.txt
* Enhanced Pipeline Execution: Now Supporting Complex Query Plans for Improved Performance (#132)
* Very initial test passing algorithm
* Working except a minor bug in interval calculations
* After clippy
* Plan
* initial implemantation
* Before prune check ability is added.
Order equivalence implementations will vanish after we send a seperate PR
* minor changes
* Fix bug, ordering equivalence random head
* minor changes
* Add ordering equivalence for sort merge join
* Improvement on tests
* Upstream changes
* Add ordering equivalence for sort merge join
* Fmt issues
* Update comment
* Add ordering equivalence support for hash join
* Make 1 file
* Code enhancements/comment improvements
* Add projection cast handling
* Fix output ordering for sort merge join
* projection bug fix
* Minor changes
* minor changes
* simplify sort_merge_join
* Update equivalence implementation
* Update test_utils.rs
* Update cast implementation
* More idiomatic code
* After merge
* Comments visisted
* Add key swap according to the children orders
* Refactoring
* After merge refactor
* Update sort_enforcement.rs
* Update datafusion/core/src/physical_optimizer/join_selection.rs
Co-authored-by: Mustafa Akur <[email protected]>
* Comments are applied
* Feature/determine prunability (#139)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
* Determine prunability of tables for join operations (#90)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* fix the tables' unboundedness
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* Comment improvements and minor code improvements
* Splitting the order based join selection
* Update rat_exclude_files.txt
* Revert "Feature/determine prunability (#139)"
This reverts commit cf56105.
* Commented
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Berkay Şahin <[email protected]>
* Bug fix: Fix lexicographical column search among provided ordering (#156)
* License update (#157)
This extension adds Synnada license information to the existing one.
* Sliding Nested Join Algorithm (#142)
* Sliding Hash Join Algorithm (SWHJ) (#147)
* Fix errors introduced during rebase
* Keep Track of Global Ordering Requirement (#165)
* Prunability of Join Filter Physical Expressions (#161)
* BinaryExpr Equivalence (#116)
* Fix errors introduced during rebase
* Support multiple ordered columns on joins and expression graph (#163)
* SlidingHashJoin and SlidingNestedLoopJoin planner integration (#171)
* Add license, add contribution hash commits, minor changes, add scripts to automate hash generation,Delete docs yaml file
* Resolve errors introduced during rebase
* Synnada Streaming SQL Tests (#190)
* Adds a new method to construct window function for the given input
* Protobuf implementations with roundrobin tests (#193)
* Protobuf implementations with roundrobin
* Proto
* Update mod.rs
* Fix linter errors, compile errors after rebase, Update commit hashes, regenerate proto
* Rewrite Filter Predicate (#192)
* Global join selection (#183)
* Feature/determine prunability (#139)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
* Determine prunability of tables for join operations (#90)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* fix the tables' unboundedness
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* [GITHUB ACTION] Refactor for license and actions (#148)
* Delete datafusion main publication
* Adding licence information, refactoring prunibility issues
* Update SYNNADA-CONTRIBUTIONS.txt
* Update rat_exclude_files.txt
* Enhanced Pipeline Execution: Now Supporting Complex Query Plans for Improved Performance (#132)
* Very initial test passing algorithm
* Working except a minor bug in interval calculations
* After clippy
* Plan
* initial implemantation
* Before prune check ability is added.
Order equivalence implementations will vanish after we send a seperate PR
* minor changes
* Fix bug, ordering equivalence random head
* minor changes
* Add ordering equivalence for sort merge join
* Improvement on tests
* Upstream changes
* Add ordering equivalence for sort merge join
* Fmt issues
* Update comment
* Add ordering equivalence support for hash join
* Make 1 file
* Code enhancements/comment improvements
* Add projection cast handling
* Fix output ordering for sort merge join
* projection bug fix
* Minor changes
* minor changes
* simplify sort_merge_join
* Update equivalence implementation
* Update test_utils.rs
* Update cast implementation
* More idiomatic code
* After merge
* Comments visisted
* Add key swap according to the children orders
* Refactoring
* After merge refactor
* Update sort_enforcement.rs
* Update datafusion/core/src/physical_optimizer/join_selection.rs
Co-authored-by: Mustafa Akur <[email protected]>
* Comments are applied
* Feature/determine prunability (#139)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
* Determine prunability of tables for join operations (#90)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* fix the tables' unboundedness
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* Comment improvements and minor code improvements
* Splitting the order based join selection
* Update rat_exclude_files.txt
* Revert "Feature/determine prunability (#139)"
This reverts commit cf56105.
* Commented
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Berkay Şahin <[email protected]>
* Bug fix: Fix lexicographical column search among provided ordering (#156)
* Initial comments on working
* License update (#157)
This extension adds Synnada license information to the existing one.
* Adding comments
* Update sort_hash_join.rs
* After merge silent error
* Change the query in HashJoin
* Revert "Feature/determine prunability (#139)"
This reverts commit cf56105.
* Update rat_exclude_files.txt
* Clippy solving.
* [GITHUB ACTION] Refactor for license and actions (#148)
* Delete datafusion main publication
* Adding licence information, refactoring prunibility issues
* Update SYNNADA-CONTRIBUTIONS.txt
* Update rat_exclude_files.txt
* Enhanced Pipeline Execution: Now Supporting Complex Query Plans for Improved Performance (#132)
* Very initial test passing algorithm
* Working except a minor bug in interval calculations
* After clippy
* Plan
* initial implemantation
* Before prune check ability is added.
Order equivalence implementations will vanish after we send a seperate PR
* minor changes
* Fix bug, ordering equivalence random head
* minor changes
* Add ordering equivalence for sort merge join
* Improvement on tests
* Upstream changes
* Add ordering equivalence for sort merge join
* Fmt issues
* Update comment
* Add ordering equivalence support for hash join
* Make 1 file
* Code enhancements/comment improvements
* Add projection cast handling
* Fix output ordering for sort merge join
* projection bug fix
* Minor changes
* minor changes
* simplify sort_merge_join
* Update equivalence implementation
* Update test_utils.rs
* Update cast implementation
* More idiomatic code
* After merge
* Comments visisted
* Add key swap according to the children orders
* Refactoring
* After merge refactor
* Update sort_enforcement.rs
* Update datafusion/core/src/physical_optimizer/join_selection.rs
Co-authored-by: Mustafa Akur <[email protected]>
* Comments are applied
* Feature/determine prunability (#139)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
* Determine prunability of tables for join operations (#90)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* fix the tables' unboundedness
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* Comment improvements and minor code improvements
* Splitting the order based join selection
* Update rat_exclude_files.txt
* Revert "Feature/determine prunability (#139)"
This reverts commit cf56105.
* Commented
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Berkay Şahin <[email protected]>
* Bug fix: Fix lexicographical column search among provided ordering (#156)
* License update (#157)
This extension adds Synnada license information to the existing one.
* [GITHUB ACTION] Refactor for license and actions (#148)
* Delete datafusion main publication
* Adding licence information, refactoring prunibility issues
* Update SYNNADA-CONTRIBUTIONS.txt
* Update rat_exclude_files.txt
* Enhanced Pipeline Execution: Now Supporting Complex Query Plans for Improved Performance (#132)
* Very initial test passing algorithm
* Working except a minor bug in interval calculations
* After clippy
* Plan
* initial implemantation
* Before prune check ability is added.
Order equivalence implementations will vanish after we send a seperate PR
* minor changes
* Fix bug, ordering equivalence random head
* minor changes
* Add ordering equivalence for sort merge join
* Improvement on tests
* Upstream changes
* Add ordering equivalence for sort merge join
* Fmt issues
* Update comment
* Add ordering equivalence support for hash join
* Make 1 file
* Code enhancements/comment improvements
* Add projection cast handling
* Fix output ordering for sort merge join
* projection bug fix
* Minor changes
* minor changes
* simplify sort_merge_join
* Update equivalence implementation
* Update test_utils.rs
* Update cast implementation
* More idiomatic code
* After merge
* Comments visisted
* Add key swap according to the children orders
* Refactoring
* After merge refactor
* Update sort_enforcement.rs
* Update datafusion/core/src/physical_optimizer/join_selection.rs
Co-authored-by: Mustafa Akur <[email protected]>
* Comments are applied
* Feature/determine prunability (#139)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
* Determine prunability of tables for join operations (#90)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* fix the tables' unboundedness
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* Comment improvements and minor code improvements
* Splitting the order based join selection
* Update rat_exclude_files.txt
* Revert "Feature/determine prunability (#139)"
This reverts commit cf56105.
* Commented
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Berkay Şahin <[email protected]>
* Bug fix: Fix lexicographical column search among provided ordering (#156)
* License update (#157)
This extension adds Synnada license information to the existing one.
* Sliding Nested Join Algorithm (#142)
* Sliding Hash Join Algorithm (SWHJ) (#147)
* Fix errors introduced during rebase
* [GITHUB ACTION] Refactor for license and actions (#148)
* Delete datafusion main publication
* Adding licence information, refactoring prunibility issues
* Update SYNNADA-CONTRIBUTIONS.txt
* Update rat_exclude_files.txt
* Enhanced Pipeline Execution: Now Supporting Complex Query Plans for Improved Performance (#132)
* Very initial test passing algorithm
* Working except a minor bug in interval calculations
* After clippy
* Plan
* initial implemantation
* Before prune check ability is added.
Order equivalence implementations will vanish after we send a seperate PR
* minor changes
* Fix bug, ordering equivalence random head
* minor changes
* Add ordering equivalence for sort merge join
* Improvement on tests
* Upstream changes
* Add ordering equivalence for sort merge join
* Fmt issues
* Update comment
* Add ordering equivalence support for hash join
* Make 1 file
* Code enhancements/comment improvements
* Add projection cast handling
* Fix output ordering for sort merge join
* projection bug fix
* Minor changes
* minor changes
* simplify sort_merge_join
* Update equivalence implementation
* Update test_utils.rs
* Update cast implementation
* More idiomatic code
* After merge
* Comments visisted
* Add key swap according to the children orders
* Refactoring
* After merge refactor
* Update sort_enforcement.rs
* Update datafusion/core/src/physical_optimizer/join_selection.rs
Co-authored-by: Mustafa Akur <[email protected]>
* Comments are applied
* Feature/determine prunability (#139)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
* Determine prunability of tables for join operations (#90)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* fix the tables' unboundedness
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* Comment improvements and minor code improvements
* Splitting the order based join selection
* Update rat_exclude_files.txt
* Revert "Feature/determine prunability (#139)"
This reverts commit cf56105.
* Commented
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Berkay Şahin <[email protected]>
* Bug fix: Fix lexicographical column search among provided ordering (#156)
* License update (#157)
This extension adds Synnada license information to the existing one.
* Sliding Nested Join Algorithm (#142)
* Sliding Hash Join Algorithm (SWHJ) (#147)
* Fix errors introduced during rebase
* Keep Track of Global Ordering Requirement (#165)
* Prunability of Join Filter Physical Expressions (#161)
* BinaryExpr Equivalence (#116)
* Fix errors introduced during rebase
* Support multiple ordered columns on joins and expression graph (#163)
* After merge
* SlidingHashJoin and SlidingNestedLoopJoin planner integration (#171)
* Before clippy fmt etc.
* lazy loading tables
* mini test
* [GITHUB ACTION] Refactor for license and actions (#148)
* Delete datafusion main publication
* Adding licence information, refactoring prunibility issues
* Update SYNNADA-CONTRIBUTIONS.txt
* Update rat_exclude_files.txt
* Enhanced Pipeline Execution: Now Supporting Complex Query Plans for Improved Performance (#132)
* Very initial test passing algorithm
* Working except a minor bug in interval calculations
* After clippy
* Plan
* initial implemantation
* Before prune check ability is added.
Order equivalence implementations will vanish after we send a seperate PR
* minor changes
* Fix bug, ordering equivalence random head
* minor changes
* Add ordering equivalence for sort merge join
* Improvement on tests
* Upstream changes
* Add ordering equivalence for sort merge join
* Fmt issues
* Update comment
* Add ordering equivalence support for hash join
* Make 1 file
* Code enhancements/comment improvements
* Add projection cast handling
* Fix output ordering for sort merge join
* projection bug fix
* Minor changes
* minor changes
* simplify sort_merge_join
* Update equivalence implementation
* Update test_utils.rs
* Update cast implementation
* More idiomatic code
* After merge
* Comments visisted
* Add key swap according to the children orders
* Refactoring
* After merge refactor
* Update sort_enforcement.rs
* Update datafusion/core/src/physical_optimizer/join_selection.rs
Co-authored-by: Mustafa Akur <[email protected]>
* Comments are applied
* Feature/determine prunability (#139)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
* Determine prunability of tables for join operations (#90)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* fix the tables' unboundedness
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* Comment improvements and minor code improvements
* Splitting the order based join selection
* Update rat_exclude_files.txt
* Revert "Feature/determine prunability (#139)"
This reverts commit cf56105.
* Commented
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Berkay Şahin <[email protected]>
* Bug fix: Fix lexicographical column search among provided ordering (#156)
* License update (#157)
This extension adds Synnada license information to the existing one.
* Sliding Nested Join Algorithm (#142)
* Sliding Hash Join Algorithm (SWHJ) (#147)
* Fix errors introduced during rebase
* Keep Track of Global Ordering Requirement (#165)
* Prunability of Join Filter Physical Expressions (#161)
* BinaryExpr Equivalence (#116)
* Fix errors introduced during rebase
* Support multiple ordered columns on joins and expression graph (#163)
* SlidingHashJoin and SlidingNestedLoopJoin planner integration (#171)
* Add license, add contribution hash commits, minor changes
* Before rebase merge
* Add license, add contribution hash commits, minor changes, add scripts to automate hash generation,Delete docs yaml file
* Update utils.rs
* Print deletion
* Update Cargo.lock
* Refactor for review
* Working without slt
* [GITHUB ACTION] Refactor for license and actions (#148)
* Delete datafusion main publication
* Adding licence information, refactoring prunibility issues
* Update SYNNADA-CONTRIBUTIONS.txt
* Update rat_exclude_files.txt
* Enhanced Pipeline Execution: Now Supporting Complex Query Plans for Improved Performance (#132)
* Very initial test passing algorithm
* Working except a minor bug in interval calculations
* After clippy
* Plan
* initial implemantation
* Before prune check ability is added.
Order equivalence implementations will vanish after we send a seperate PR
* minor changes
* Fix bug, ordering equivalence random head
* minor changes
* Add ordering equivalence for sort merge join
* Improvement on tests
* Upstream changes
* Add ordering equivalence for sort merge join
* Fmt issues
* Update comment
* Add ordering equivalence support for hash join
* Make 1 file
* Code enhancements/comment improvements
* Add projection cast handling
* Fix output ordering for sort merge join
* projection bug fix
* Minor changes
* minor changes
* simplify sort_merge_join
* Update equivalence implementation
* Update test_utils.rs
* Update cast implementation
* More idiomatic code
* After merge
* Comments visisted
* Add key swap according to the children orders
* Refactoring
* After merge refactor
* Update sort_enforcement.rs
* Update datafusion/core/src/physical_optimizer/join_selection.rs
Co-authored-by: Mustafa Akur <[email protected]>
* Comments are applied
* Feature/determine prunability (#139)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
* Determine prunability of tables for join operations (#90)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* fix the tables' unboundedness
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* Comment improvements and minor code improvements
* Splitting the order based join selection
* Update rat_exclude_files.txt
* Revert "Feature/determine prunability (#139)"
This reverts commit cf56105.
* Commented
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Berkay Şahin <[email protected]>
* Bug fix: Fix lexicographical column search among provided ordering (#156)
* License update (#157)
This extension adds Synnada license information to the existing one.
* Sliding Nested Join Algorithm (#142)
* Sliding Hash Join Algorithm (SWHJ) (#147)
* Fix errors introduced during rebase
* Keep Track of Global Ordering Requirement (#165)
* Prunability of Join Filter Physical Expressions (#161)
* BinaryExpr Equivalence (#116)
* Fix errors introduced during rebase
* Support multiple ordered columns on joins and expression graph (#163)
* SlidingHashJoin and SlidingNestedLoopJoin planner integration (#171)
* Add license, add contribution hash commits, minor changes, add scripts to automate hash generation,Delete docs yaml file
* Change in test folders
* Update join_pipeline_selection.rs
* Update utils.rs
* Before clippy
* Before SLT
* Tests are passing and clippy OK.
* [GITHUB ACTION] Refactor for license and actions (#148)
* Delete datafusion main publication
* Adding licence information, refactoring prunibility issues
* Update SYNNADA-CONTRIBUTIONS.txt
* Update rat_exclude_files.txt
* Enhanced Pipeline Execution: Now Supporting Complex Query Plans for Improved Performance (#132)
* Very initial test passing algorithm
* Working except a minor bug in interval calculations
* After clippy
* Plan
* initial implemantation
* Before prune check ability is added.
Order equivalence implementations will vanish after we send a seperate PR
* minor changes
* Fix bug, ordering equivalence random head
* minor changes
* Add ordering equivalence for sort merge join
* Improvement on tests
* Upstream changes
* Add ordering equivalence for sort merge join
* Fmt issues
* Update comment
* Add ordering equivalence support for hash join
* Make 1 file
* Code enhancements/comment improvements
* Add projection cast handling
* Fix output ordering for sort merge join
* projection bug fix
* Minor changes
* minor changes
* simplify sort_merge_join
* Update equivalence implementation
* Update test_utils.rs
* Update cast implementation
* More idiomatic code
* After merge
* Comments visisted
* Add key swap according to the children orders
* Refactoring
* After merge refactor
* Update sort_enforcement.rs
* Update datafusion/core/src/physical_optimizer/join_selection.rs
Co-authored-by: Mustafa Akur <[email protected]>
* Comments are applied
* Feature/determine prunability (#139)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
* Determine prunability of tables for join operations (#90)
* ready to review
* license added
* simplifications
* simplifications
* sort expr's are taken separately for each table
* we can return the sort info of the expression now
* check filter conditions
* simplifications
* simplifications
* functions are implemented for SortInfo calculations
* node specialized tableSide functions
* NotImplemented errors are added, test comments are added
* Comment change
* Simplify comparison node calculations
* Simplfications and better commenting
* is_prunable function is updated with new Prunability function
* Indices of sort expressions are updated with intermediate schema columns of the filter
* Unused function is removed
* Future-proof index updating
* An if let check is removed
* simplifications
* Simplifications
* simplifications
* Change if condition
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* fix the tables' unboundedness
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
* Comment improvements and minor code improvements
* Splitting the order based join selection
* Update rat_exclude_files.txt
* Revert "Feature/determine prunability (#139)"
This reverts commit cf56105.
* Commented
---------
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Berkay Şahin <[email protected]>
* Bug fix: Fix lexicographical column search among provided ordering (#156)
* License update (#157)
This extension adds Synnada license information to the existing one.
* Sliding Nested Join Algorithm (#142)
* Sliding Hash Join Algorithm (SWHJ) (#147)
* Fix errors introduced during rebase
* Keep Track of Global Ordering Requirement (#165)
* Prunability of Join Filter Physical Expressions (#161)
* BinaryExpr Equivalence (#116)
* Fix errors introduced during rebase
* Support multiple ordered columns on joins and expression graph (#163)
* SlidingHashJoin and SlidingNestedLoopJoin planner integration (#171)
* Add license, add contribution hash commits, minor changes, add scripts to automate hash generation,Delete docs yaml file
* Resolve errors introduced during rebase
* After merge
* Update rat_exclude_files.txt
* Comments visited
* Synnada Streaming SQL Tests (#190)
* Adds a new method to construct window function for the given input
* For mustafa
* Final
* Update rat_exclude_files.txt
* More commenting
* Fix linter errors, compile errors after rebase, Update commit hashes
* After merge refactors
* Dir
* Additional test for coverage
* Update join_disable_repartition_joins.slt
* Review changes, remove code duplicates
* Update subdirectory hashes
---------
Co-authored-by: Berkay Şahin <[email protected]>
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
Co-authored-by: Mustafa Akur <[email protected]>
* work in progress
* fix after merge
* joins are ok
* code cleaning
* CaseExpr handling
* tests are updated
* Handling sequential projections
* Simplifications
* partition expr update
* ProjectionPushdown becomes a rule.bash de
* fix after merge
* remove the subrule
* tpch update
* Minor comment changes
* remove unnecessary state struct
* coalesce batches does not let to push projection down
* tpch changes removed
* minor changes
* addresses reviews
* Update projection_pushdown.rs
* minor
* Review Part 1
* Simplify the API of plan handlers
* Review Part 2
* Review Part 3
* Review Part 4
* Review Part 5
* Review Part 6
* Review Part 7
* Remove duplication of physical_expr matching
* fix documentation
* Minor changes
* Take upstream changes
---------
Co-authored-by: Metehan Yıldırım <[email protected]>
Co-authored-by: Mustafa Akur <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
Co-authored-by: Mustafa Akur <[email protected]>1 parent 8bd6fca commit c48d4ed
11 files changed
Lines changed: 217 additions & 130 deletions
File tree
- datafusion
- core/src/physical_optimizer
- expr/src
- physical-expr/src
- expressions
- sqllogictest/test_files
- dev/release
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
| 37 | + | |
| 38 | + | |
38 | 39 | | |
39 | | - | |
40 | 40 | | |
41 | 41 | | |
42 | | - | |
43 | | - | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
44 | 45 | | |
45 | 46 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
27 | | - | |
28 | 27 | | |
29 | 28 | | |
30 | 29 | | |
31 | 30 | | |
32 | 31 | | |
33 | 32 | | |
34 | 33 | | |
| 34 | + | |
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
18 | 21 | | |
19 | 22 | | |
20 | 23 | | |
| |||
29 | 32 | | |
30 | 33 | | |
31 | 34 | | |
32 | | - | |
| 35 | + | |
33 | 36 | | |
34 | | - | |
35 | | - | |
| 37 | + | |
| 38 | + | |
36 | 39 | | |
37 | 40 | | |
38 | 41 | | |
| |||
133 | 136 | | |
134 | 137 | | |
135 | 138 | | |
| 139 | + | |
| 140 | + | |
136 | 141 | | |
137 | 142 | | |
138 | 143 | | |
| |||
679 | 684 | | |
680 | 685 | | |
681 | 686 | | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
| 734 | + | |
| 735 | + | |
| 736 | + | |
| 737 | + | |
| 738 | + | |
| 739 | + | |
| 740 | + | |
| 741 | + | |
| 742 | + | |
| 743 | + | |
| 744 | + | |
| 745 | + | |
| 746 | + | |
| 747 | + | |
| 748 | + | |
| 749 | + | |
| 750 | + | |
| 751 | + | |
| 752 | + | |
| 753 | + | |
| 754 | + | |
| 755 | + | |
| 756 | + | |
| 757 | + | |
| 758 | + | |
| 759 | + | |
| 760 | + | |
| 761 | + | |
| 762 | + | |
| 763 | + | |
| 764 | + | |
| 765 | + | |
682 | 766 | | |
683 | 767 | | |
684 | 768 | | |
| |||
876 | 960 | | |
877 | 961 | | |
878 | 962 | | |
879 | | - | |
880 | 963 | | |
881 | 964 | | |
882 | 965 | | |
| |||
1156 | 1239 | | |
1157 | 1240 | | |
1158 | 1241 | | |
| 1242 | + | |
| 1243 | + | |
| 1244 | + | |
| 1245 | + | |
| 1246 | + | |
| 1247 | + | |
| 1248 | + | |
| 1249 | + | |
| 1250 | + | |
| 1251 | + | |
| 1252 | + | |
| 1253 | + | |
| 1254 | + | |
| 1255 | + | |
| 1256 | + | |
| 1257 | + | |
| 1258 | + | |
| 1259 | + | |
| 1260 | + | |
1159 | 1261 | | |
1160 | 1262 | | |
1161 | 1263 | | |
| |||
1170 | 1272 | | |
1171 | 1273 | | |
1172 | 1274 | | |
1173 | | - | |
| 1275 | + | |
1174 | 1276 | | |
1175 | 1277 | | |
1176 | 1278 | | |
| |||
1192 | 1294 | | |
1193 | 1295 | | |
1194 | 1296 | | |
1195 | | - | |
| 1297 | + | |
1196 | 1298 | | |
1197 | 1299 | | |
1198 | 1300 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
249 | 249 | | |
250 | 250 | | |
251 | 251 | | |
252 | | - | |
| 252 | + | |
253 | 253 | | |
254 | 254 | | |
255 | 255 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| 24 | + | |
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
33 | 36 | | |
| 37 | + | |
34 | 38 | | |
35 | 39 | | |
36 | 40 | | |
37 | 41 | | |
38 | 42 | | |
39 | 43 | | |
| 44 | + | |
40 | 45 | | |
41 | 46 | | |
42 | 47 | | |
43 | | - | |
| 48 | + | |
44 | 49 | | |
45 | 50 | | |
46 | | - | |
47 | 51 | | |
48 | 52 | | |
49 | 53 | | |
50 | 54 | | |
51 | | - | |
52 | | - | |
53 | 55 | | |
54 | 56 | | |
55 | 57 | | |
| |||
979 | 981 | | |
980 | 982 | | |
981 | 983 | | |
982 | | - | |
983 | | - | |
| 984 | + | |
| 985 | + | |
984 | 986 | | |
985 | 987 | | |
986 | 988 | | |
| |||
990 | 992 | | |
991 | 993 | | |
992 | 994 | | |
993 | | - | |
994 | | - | |
| 995 | + | |
995 | 996 | | |
996 | 997 | | |
997 | 998 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
255 | 255 | | |
256 | 256 | | |
257 | 257 | | |
258 | | - | |
259 | 258 | | |
260 | 259 | | |
261 | 260 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
283 | 283 | | |
284 | 284 | | |
285 | 285 | | |
286 | | - | |
287 | | - | |
288 | | - | |
289 | | - | |
290 | | - | |
291 | | - | |
292 | | - | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
293 | 292 | | |
294 | 293 | | |
295 | 294 | | |
| |||
374 | 373 | | |
375 | 374 | | |
376 | 375 | | |
377 | | - | |
378 | | - | |
379 | | - | |
380 | | - | |
381 | | - | |
382 | | - | |
383 | | - | |
384 | | - | |
385 | | - | |
386 | | - | |
387 | | - | |
388 | | - | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
389 | 386 | | |
390 | 387 | | |
391 | 388 | | |
| |||
0 commit comments