Python: Alter table plumbing and REST support by Fokko · Pull Request #6323 · apache/iceberg

Fokko · 2022-11-30T13:05:41Z

The plumbing to do an table.alter().commit() with actions in between. We can reuse this later on to do multi table commits:

from datetime import datetime

now = str(datetime.now())

with Transaction(catalog) as transaction:
    table.alter().setProperties(updated_at=now)
    table.alter().setProperties(updated_at=now)

# Multi table commit done

python/mkdocs/docs/index.md

python/mkdocs/docs/api.md

python/mkdocs/docs/feature-support.md

python/pyiceberg/catalog/null.py

python/pyiceberg/catalog/rest.py

python/pyiceberg/table/__init__.py

…lter-table

python/mkdocs/docs/api.md

rdblue · 2023-06-19T23:56:57Z

python/pyiceberg/catalog/rest.py


        return self.load_table(to_identifier)

+    def _commit(self, *table_requests: CommitTableRequest) -> CommitTableResponse:


Do we want to combine these? Why not have separate commit_table and commit_transaction methods? Then we don't need the check that only one request is supported. I also like that you could previously pass requirements and updates to a public method. Are you trying to restrict access to those in the "public" API for some reason?

+1, given this is called afterwards in Transaction, it feels more nature to call it just commit_table so we an use self._table.catalog.commit_table instead of self._table.catalog._commit

Yes, thinking of it, that makes sense. Also, the return type is different for a table and a transaction. Thanks!

rdblue · 2023-06-20T02:01:01Z

Looks good to me. We can always update the method signature later since this one is internal. I'd merge, but the lock file has conflicts.

jackye1995

mostly looks good to me, +1 for merging so we can start adding implementations for the other catalogs.

jackye1995 · 2023-06-21T21:23:26Z

python/pyiceberg/catalog/rest.py


        return self.load_table(to_identifier)

+    def _commit(self, *table_requests: CommitTableRequest) -> CommitTableResponse:


+1, given this is called afterwards in Transaction, it feels more nature to call it just commit_table so we an use self._table.catalog.commit_table instead of self._table.catalog._commit

jackye1995 · 2023-06-21T21:24:24Z

python/pyiceberg/table/__init__.py

 ALWAYS_TRUE = AlwaysTrue()


+class Transaction:


Do we plan to have another MiltiTableTransaction for transactions across tables?

I'd rather expect a List[Transaction]

Fokko · 2023-06-21T22:06:59Z

Thank @jackye1995 for chiming in here! 🙏🏻

github-actions bot added the python label Nov 30, 2022

rdblue reviewed Nov 30, 2022

View reviewed changes

python/mkdocs/docs/index.md Outdated Show resolved Hide resolved

Fokko closed this Feb 28, 2023

Fokko mentioned this pull request Mar 31, 2023

Python: Ability to the Metadata JSON #7257

Closed

Fokko deleted the fd-alter-table branch April 26, 2023 09:49

Fokko restored the fd-alter-table branch April 26, 2023 09:49

Fokko reopened this Apr 26, 2023

Fokko force-pushed the fd-alter-table branch 2 times, most recently from d4b6a5b to d7c795c Compare April 28, 2023 10:46

Fokko added this to the PyIceberg 0.5.0 release milestone Apr 28, 2023

Fokko changed the title ~~Python: Alter table for table properties~~ Python: Alter table plumbing and REST support Apr 28, 2023

Alter table

68c9a03

Fokko force-pushed the fd-alter-table branch from d7c795c to 68c9a03 Compare April 28, 2023 11:05

Fokko added 4 commits May 1, 2023 23:48

Merge branch 'master' of github.com:apache/iceberg into fd-alter-table

4e575ba

Merge branch 'master' of github.com:apache/iceberg into fd-alter-table

e7a4251

Make the CI happy

aa767e3

Merge branch 'master' of github.com:apache/iceberg into fd-alter-table

ebc8fda

Fokko modified the milestones: PyIceberg 0.5.0 release, PyIceberg 0.4.0 release May 15, 2023