Skip to content

Add Hudi and DeltaLake table engines#41054

Merged
kssenii merged 44 commits intoClickHouse:masterfrom
rubin-do:hudi
Nov 8, 2022
Merged

Add Hudi and DeltaLake table engines#41054
kssenii merged 44 commits intoClickHouse:masterfrom
rubin-do:hudi

Conversation

@rubin-do
Copy link
Copy Markdown
Contributor

@rubin-do rubin-do commented Sep 6, 2022 •
edited by kssenii
Loading

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Add Hudi and DeltaLake table engines, read-only, only for tables on S3.

@robot-ch-test-poll robot-ch-test-poll added the pr-feature Pull request with new product feature label Sep 6, 2022
@alexey-milovidov alexey-milovidov added the can be tested Allows running workflows for external contributors label Sep 6, 2022
@kssenii kssenii self-assigned this Sep 6, 2022
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Sep 7, 2022 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

kssenii
kssenii reviewed Sep 19, 2022
View reviewed changes
kssenii
kssenii reviewed Sep 19, 2022
View reviewed changes
kssenii
kssenii reviewed Sep 21, 2022
View reviewed changes
kssenii
kssenii reviewed Oct 3, 2022
View reviewed changes
@alexey-milovidov alexey-milovidov mentioned this pull request Oct 8, 2022
Closed
@kssenii
Copy link
Copy Markdown
Member

kssenii commented Nov 8, 2022

Integration tests (tsan) [1/4] — fail: 1

test_grpc_protocol_ssl/test.py::test_secure_channel

@kssenii kssenii merged commit d2b777c into ClickHouse:master Nov 8, 2022
@ucasfl
Copy link
Copy Markdown
Collaborator

ucasfl commented Nov 15, 2022

Question about these two table engines: currently implementation will get all data files when creating the table, but the iceberg/hudi table may have data written after table creation, while we just can read the data at the moment of table creation.

@kssenii
Copy link
Copy Markdown
Member

kssenii commented Nov 15, 2022

Question about these two table engines: currently implementation will get all data files when creating the table, but the iceberg/hudi table may have data written after table creation, while we just can read the data at the moment of table creation.

create a PR with the improvement?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kssenii kssenii kssenii approved these changes

Assignees

@kssenii kssenii

Labels

can be tested Allows running workflows for external contributors pr-feature Pull request with new product feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@rubin-do @CLAassistant @kssenii @ucasfl @alexey-milovidov @robot-ch-test-poll