-
Notifications
You must be signed in to change notification settings - Fork 101
Add databricks labs ucx migrate-tables and a related workflow #670
Description
databricks labs ucx migrate-tables to apply table mapping from CSV file with the following columns: src_workspace_name,dst_catalog,src_schema,dst_schema,src_table,dst_table . Consider the following stages:
Initial assessment
We scan all tables, generate an inventory, get all perms, etc.
We provide a mapping file the customer can customize, but seed the UC table name with something like the workspace name or ID as the catalog.
Customer can choose to modify the mapping or not
Initial migration
We use the mapping file to sync tables, apply perms, and persist the mapping data in inventory
Repeated migration - this is probably a job that runs on schedule and the user should be instructed to configure accordingly
We use the mapping data in inventory to re-sync any tables, but not re-apply permissions as these probably changed.
Fine-grained:
Upstream dependencies:
- Automatically assign a UC Metastore to a workspace #341
- [FEATURE]: Create UC External Location, Schema, and Table Grants based on workspace-wide Azure SPN mount points #94
- Create EXTERNAL LOCATIONs to map to External Tables (Azure) #100
- Add
databricks labs ucx table-mappingcommand #669
Next quarter:
- Allow skipping DBFS Root Delta Table if size exceeds preconfigured threshold #334
- [FEATURE]: Connect
_migrate_table_create_ctastomigrate-tablesworkflow #332 - Migration corner-case: what if the schema changed upstream in Hive.
SYNCis supposed to be idempotent and intended to run multiple times. #319 - Add
databricks labs ucx validate-table-locationscommand #673
Metadata
Metadata
Labels
Type
Projects
Status