Repository navigation
[FEATURE] Nessie integration of Iceberg REST - #7043
Merged
Merged
Conversation
snazy
force-pushed
the
feature/nessie-catalog-server
branch
3 times, most recently
from
July 11, 2023 15:44
ae85c09 to
5783a01
Compare
snazy
force-pushed
the
feature/nessie-catalog-server
branch
4 times, most recently
from
July 19, 2023 17:13
422dd34 to
9b4e0fb
Compare
snazy
force-pushed
the
feature/nessie-catalog-server
branch
7 times, most recently
from
August 3, 2023 08:04
694c558 to
899ce14
Compare
snazy
force-pushed
the
feature/nessie-catalog-server
branch
from
May 2, 2024 15:25
fd576c7 to
d696495
Compare
snazy
force-pushed
the
feature/nessie-catalog-server
branch
from
May 13, 2024 15:06
c34104b to
47de890
Compare
snazy
marked this pull request as ready for review
May 13, 2024 15:06
snazy
force-pushed
the
feature/nessie-catalog-server
branch
2 times, most recently
from
May 17, 2024 07:57
b8c6df1 to
b1b16ff
Compare
Really use a non-default warehouse, configure the warehouse _name_ via the misleadingly named property `CatalogProperties.WAREHOUSE_LOCATION`. Also eliminate some duplicate code.
and add some unit tests also remove unused `CatalogServerConfig`
…8539) The Nessie client used from Nessie Catalog is currently a "simple" one. This change lets it use a client that's configured against the actual config+tree resources, which also perform authZ.
Start collecting all base locations in `NessieTableSnapshot`. This change is a start to collect _all_ (base) table locations, to handle cases when a table's (meta)data files are "spread" across multiple base locations. Ideally (and probably usually), each table has only one base location.
* Make `nessie.catalog.default-warehouse` just the optional name of the default warehouse (prevent duplicate configuration). * Remove `WarehouseConfig.name()` - the name is implicitly defined via the map key of `CatalogConfig.warehouses()`
The handling of secrets is a bit awkward at the moment. We declare "IDs" on the objects, which are to be looked up via the `SecretsProvider`, which is currently just a `Map<String, String>`. This change changes the behavior: The (default) behavior to lookup via another map is gone. Secrets are now generally looked up in the following order: 1. If the value is defined on the "per bucket" config, it is used as is. 2. If a secret for the bucket can be resolved, it is used. 3. If the value is defined on the "default" config, it is used as is. 4. If a secret for the "default" config can be resolved, it is used. 5. The secret is assumed to not exist. The above logic is implemented in the new `SecretsHelper` class. It calls the `SecretProvider` using a collection of keys to look up, which shall return a map with the existing, resolved secrets. Follow-up, after #7043 is merged to `main`: refactor secrets handlings in "Nessie Core", where/if necessary.
Co-authored-by: Alexandre Dutra <[email protected]>
- Client `client1` has access to everything. - Client `client2` has *NO* access to the `sales` namespace. - Client `client3` has *NO* access to the `eng` namespace. ---------- Co-authored-by: Alexandre Dutra <[email protected]>
`CONNECT TO` and `USE` working, `SHOW TABLE/VIEW` displays the table/view metadata. Also adds the `ON initialReference` clause to `CONNECT TO`.
snazy
force-pushed
the
feature/nessie-catalog-server
branch
from
May 27, 2024 18:55
5e176f9 to
49cf7f7
Compare
Member
Author
Member
|
I ran a quick smoke test in my local on bd33b94 - it seems to work fine 🎉 |
dimas-b
approved these changes
May 28, 2024
adutra
approved these changes
May 28, 2024
adutra
left a comment
Contributor
There was a problem hiding this comment.
Approving although I think we have at least one issue with base location that we just discussed offline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.