Skip to content

feat(sqllogictest): Add support for iceberg datafusion sqllogictest integration#1764

Merged
liurenjie1024 merged 11 commits intoapache:mainfrom
lliangyu-lin:sqllogic-datafusion
Oct 24, 2025
Merged

feat(sqllogictest): Add support for iceberg datafusion sqllogictest integration#1764
liurenjie1024 merged 11 commits intoapache:mainfrom
lliangyu-lin:sqllogic-datafusion

Conversation

@lliangyu-lin
Copy link
Contributor

@lliangyu-lin lliangyu-lin commented Oct 20, 2025 •
edited
Loading

Which issue does this PR close?

What changes are included in this PR?

  • Add sqllogictests entry point sqllogictests.rs
  • Update Schedule to track EngineRunner instead of Engine
  • Update datafusion engine to run slt file with sqllogictest

Are these changes tested?

  • cargo test --test sqllogictests
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.46s
     Running tests/sqllogictests.rs (target/debug/deps/sqllogictests-8ee6c1e0fdedebe4)

running 1 test
test schedule: df_test.toml ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

lliangyu-lin
lliangyu-lin commented Oct 20, 2025
View reviewed changes
@lliangyu-lin
Copy link
Contributor Author

lliangyu-lin commented Oct 21, 2025

Getting No space left on device during compilation in CI. Will check if we could increase the memory or if any extra dependencies were added.

liurenjie1024
liurenjie1024 reviewed Oct 21, 2025
View reviewed changes
Copy link
Contributor

@liurenjie1024 liurenjie1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @lliangyu-lin for this pr!

@lliangyu-lin lliangyu-lin mentioned this pull request Oct 23, 2025
Open
lliangyu-lin
lliangyu-lin commented Oct 23, 2025
View reviewed changes
.github/workflows/ci.yml
- macos-latest
- windows-latest
steps:
- name: Maximize build space (Ubuntu)
Copy link
Contributor Author

@lliangyu-lin lliangyu-lin Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is same used in unit test workflow: https://github.com/apache/iceberg-rust/blob/main/.github/workflows/ci.yml#L128-L131

liurenjie1024
liurenjie1024 reviewed Oct 23, 2025
View reviewed changes
Copy link
Contributor

@liurenjie1024 liurenjie1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @lliangyu-lin for this pr, generally LGTM! Just one minor nit.

crates/sqllogictest/testdata/schedules/df_test.toml Outdated
[engines]
df = { type = "datafusion" }

[catalog]
Copy link
Contributor

@liurenjie1024 liurenjie1024 Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need this any more? If so, it would be confusing to put it here.

Copy link
Contributor Author

@lliangyu-lin lliangyu-lin Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[catalog] is not needed for now. Removed in the new commit.

liurenjie1024
liurenjie1024 approved these changes Oct 24, 2025
View reviewed changes
Copy link
Contributor

@liurenjie1024 liurenjie1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @lliangyu-lin for this pr!

@liurenjie1024 liurenjie1024 merged commit 406d416 into apache:main Oct 24, 2025
17 checks passed
liurenjie1024 pushed a commit that referenced this pull request Oct 24, 2025
@mbutrovich
fix(build): Pin home version after merging #1764 (#1783)
ddbcae4
mbutrovich added a commit to mbutrovich/iceberg-rust that referenced this pull request Oct 25, 2025
@mbutrovich
fix(build): Pin home version after merging apache#1764 (apache#1783)
b0a58be 
(cherry picked from commit ddbcae4)
@lliangyu-lin lliangyu-lin deleted the sqllogic-datafusion branch October 27, 2025 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@liurenjie1024 liurenjie1024 liurenjie1024 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@lliangyu-lin @liurenjie1024