-
Notifications
You must be signed in to change notification settings - Fork 101
Test failure: test_table_migration_job_refreshes_migration_status[regular-migrate-tables] #1628
Copy link
Copy link
Closed
Description
❌ test_table_migration_job_refreshes_migration_status[regular-migrate-tables]: AssertionError: No migration status found for TableType.MANAGED hive_metastore.migrate_zilg5.ucx_tsvdl (11m47.311s)
AssertionError: No migration status found for TableType.MANAGED hive_metastore.migrate_zilg5.ucx_tsvdl
assert 0 == 1
+ where 0 = len([])
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/GNRr
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_zilg5: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_zilg5', metastore_id=None, name='migrate_zilg5', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_zilg5.ucx_tsvdl: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5/ucx_tsvdl
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_tsvdl', metastore_id=None, name='ucx_tsvdl', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_zilg5/ucx_tsvdl', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_zilg5.ucx_tur4f: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5/ucx_tur4f
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_tur4f', metastore_id=None, name='ucx_tur4f', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/GNRr', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_zilg5.ucx_t99k5: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5/ucx_t99k5
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_t99k5', metastore_id=None, name='ucx_t99k5', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_zilg5.ucx_tsvdl', view_dependencies=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_zilg5.ucx_tnq3d: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5/ucx_tnq3d
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_tnq3d', metastore_id=None, name='ucx_tnq3d', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_zilg5.ucx_t99k5', view_dependencies=None)
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1714806233915, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8yj4', isolation_mode=<IsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c8yj4', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1714806233915, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c8yj4.migrate_zilg5: https://DATABRICKS_HOST/explore/data/ucx_c8yj4/migrate_zilg5
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c8yj4', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8yj4.migrate_zilg5', metastore_id=None, name='migrate_zilg5', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_swyzz: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_swyzz
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_swyzz', metastore_id=None, name='ucx_swyzz', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
[gw2] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/GNRr
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_zilg5: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_zilg5', metastore_id=None, name='migrate_zilg5', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_zilg5.ucx_tsvdl: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5/ucx_tsvdl
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_tsvdl', metastore_id=None, name='ucx_tsvdl', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_zilg5/ucx_tsvdl', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_zilg5.ucx_tur4f: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5/ucx_tur4f
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_tur4f', metastore_id=None, name='ucx_tur4f', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/GNRr', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_zilg5.ucx_t99k5: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5/ucx_t99k5
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_t99k5', metastore_id=None, name='ucx_t99k5', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_zilg5.ucx_tsvdl', view_dependencies=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_zilg5.ucx_tnq3d: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5/ucx_tnq3d
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_tnq3d', metastore_id=None, name='ucx_tnq3d', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_zilg5.ucx_t99k5', view_dependencies=None)
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1714806233915, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8yj4', isolation_mode=<IsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c8yj4', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1714806233915, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c8yj4.migrate_zilg5: https://DATABRICKS_HOST/explore/data/ucx_c8yj4/migrate_zilg5
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c8yj4', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8yj4.migrate_zilg5', metastore_id=None, name='migrate_zilg5', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_swyzz: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_swyzz
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_swyzz', metastore_id=None, name='ucx_swyzz', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:04 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QtBC/config.yml) doesn't exist.
07:04 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
07:04 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
07:04 INFO [databricks.labs.ucx.install] Fetching installations...
07:04 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QtBC is corrupted. Skipping...
07:04 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
07:04 DEBUG [tests.integration.conftest] Waiting for clusters to start...
07:12 DEBUG [tests.integration.conftest] Waiting for clusters to start...
07:12 INFO [databricks.labs.ucx.install] Installing UCX v0.22.1+3220240504071213
07:12 INFO [databricks.labs.ucx.install] Creating ucx schemas...
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
07:12 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QtBC/README for the next steps.
07:12 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/1016215544949849
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/GNRr
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_zilg5: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_zilg5', metastore_id=None, name='migrate_zilg5', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_zilg5.ucx_tsvdl: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5/ucx_tsvdl
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_tsvdl', metastore_id=None, name='ucx_tsvdl', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_zilg5/ucx_tsvdl', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_zilg5.ucx_tur4f: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5/ucx_tur4f
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_tur4f', metastore_id=None, name='ucx_tur4f', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/GNRr', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_zilg5.ucx_t99k5: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5/ucx_t99k5
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_t99k5', metastore_id=None, name='ucx_t99k5', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_zilg5.ucx_tsvdl', view_dependencies=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_zilg5.ucx_tnq3d: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_zilg5/ucx_tnq3d
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_tnq3d', metastore_id=None, name='ucx_tnq3d', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_zilg5.ucx_t99k5', view_dependencies=None)
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1714806233915, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8yj4', isolation_mode=<IsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c8yj4', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1714806233915, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c8yj4.migrate_zilg5: https://DATABRICKS_HOST/explore/data/ucx_c8yj4/migrate_zilg5
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c8yj4', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8yj4.migrate_zilg5', metastore_id=None, name='migrate_zilg5', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:03 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_swyzz: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_swyzz
07:03 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_swyzz', metastore_id=None, name='ucx_swyzz', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:04 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QtBC/config.yml) doesn't exist.
07:04 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
07:04 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
07:04 INFO [databricks.labs.ucx.install] Fetching installations...
07:04 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QtBC is corrupted. Skipping...
07:04 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
07:04 DEBUG [tests.integration.conftest] Waiting for clusters to start...
07:12 DEBUG [tests.integration.conftest] Waiting for clusters to start...
07:12 INFO [databricks.labs.ucx.install] Installing UCX v0.22.1+3220240504071213
07:12 INFO [databricks.labs.ucx.install] Creating ucx schemas...
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
07:12 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
07:12 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QtBC/README for the next steps.
07:12 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/1016215544949849
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/GNRr
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1714806233915, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8yj4', isolation_mode=<IsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c8yj4', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1714806233915, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while catalog CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1714806233915, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8yj4', isolation_mode=<IsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c8yj4', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1714806233915, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d') teardown: Catalog 'ucx_c8yj4' does not exist.
07:15 INFO [databricks.labs.ucx.install] Deleting UCX v0.22.1+3220240504071521 from https://DATABRICKS_HOST
07:15 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_swyzz
07:15 INFO [databricks.labs.ucx.install] Deleting jobs
07:15 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=1116149974119258.
07:15 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=234037995859769.
07:15 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=316689123165635.
07:15 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=483070915129766.
07:15 INFO [databricks.labs.ucx.install] Deleting assessment job_id=565632414989636.
07:15 INFO [databricks.labs.ucx.install] Deleting failing job_id=275197782356187.
07:15 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=1016215544949849.
07:15 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=218478226601944.
07:15 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=721932942380578.
07:15 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=191109020166073.
07:15 INFO [databricks.labs.ucx.install] Deleting cluster policy
07:15 ERROR [databricks.labs.ucx.install] UCX Policy already deleted
07:15 INFO [databricks.labs.ucx.install] Deleting secret scope
07:15 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 4 table fixtures
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_tsvdl', metastore_id=None, name='ucx_tsvdl', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_zilg5/ucx_tsvdl', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_tur4f', metastore_id=None, name='ucx_tur4f', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/GNRr', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_t99k5', metastore_id=None, name='ucx_t99k5', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_zilg5.ucx_tsvdl', view_dependencies=None)
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_zilg5.ucx_tnq3d', metastore_id=None, name='ucx_tnq3d', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='migrate_zilg5', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_zilg5.ucx_t99k5', view_dependencies=None)
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_zilg5', metastore_id=None, name='migrate_zilg5', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c8yj4', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c8yj4.migrate_zilg5', metastore_id=None, name='migrate_zilg5', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_swyzz', metastore_id=None, name='ucx_swyzz', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
[gw2] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
Running from nightly #50
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels