Skip to content

Test failure: test_running_real_assessment_job_ext_hms #1616

Closed
Closed
Test failure: test_running_real_assessment_job_ext_hms#1616
Assignees
JCZuurmond
@github-actions

Description

@github-actions
github-actions
bot
opened on May 3, 2024
❌ test_running_real_assessment_job_ext_hms: KeyError: 'ucx_tnIh' (25m7.398s) 
KeyError: 'ucx_tnIh'
[gw8] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
07:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='[email protected]', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='[email protected]')], entitlements=[], external_id=None, groups=[], id='3594324638715029', name=Name(family_name=None, given_name='[email protected]'), roles=[], schemas=[<UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_USER: 'urn:ietf:params:scim:schemas:core:2.0:User'>, <UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_EXTENSION_WORKSPACE_2_0_USER: 'urn:ietf:params:scim:schemas:extension:workspace:2.0:User'>], user_name='[email protected]')
07:05 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group ucx_tnIh: https://DATABRICKS_HOST#setting/accounts/groups/373244128321231
07:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='ucx_tnIh', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='373244128321231', members=[ComplexValue(display='[email protected]', primary=None, ref='Users/3594324638715029', type=None, value='3594324638715029')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[<GroupSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_GROUP: 'urn:ietf:params:scim:schemas:core:2.0:Group'>])
07:05 INFO [databricks.labs.ucx.mixins.fixtures] Account group ucx_tnIh: https://accounts.CLOUD_ENVdatabricks.net/users/groups/1001775630102736/members
07:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='ucx_tnIh', entitlements=[], external_id=None, groups=[], id='1001775630102736', members=[ComplexValue(display='[email protected]', primary=None, ref='Users/3594324638715029', type=None, value='3594324638715029')], meta=None, roles=[], schemas=[<GroupSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_GROUP: 'urn:ietf:params:scim:schemas:core:2.0:Group'>])
07:05 INFO [databricks.labs.ucx.mixins.fixtures] Cluster policy: https://DATABRICKS_HOST#setting/clusters/cluster-policies/view/000BA07CC000327C
07:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster policy fixture: CreatePolicyResponse(policy_id='000BA07CC000327C')
07:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster_policy permissions fixture: 000BA07CC000327C [group_name admins CAN_USE] -> [group_name ucx_tnIh CAN_USE]
07:05 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE SCHEMA hive_metastore.ucx_sxsox
07:15 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sxsox: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sxsox
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sxsox', metastore_id=None, name='ucx_sxsox', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:15 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.gE8a/config.yml) doesn't exist.
07:15 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
07:15 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
07:15 INFO [databricks.labs.ucx.install] Fetching installations...
07:15 INFO [databricks.labs.ucx.installer.policy] Setting up an external metastore
07:15 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
07:15 DEBUG [tests.integration.conftest] Waiting for clusters to start...
07:15 DEBUG [tests.integration.conftest] Waiting for clusters to start...
07:15 INFO [databricks.labs.ucx.install] Installing UCX v0.22.1+2720240503071553
07:15 INFO [databricks.labs.ucx.install] Creating ucx schemas...
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE SCHEMA IF NOT EXISTS hive_metastore.ucx_sxsox
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.CLOUD_ENV_service_principals (application_id STR... (107 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.clusters (cluster_id STRING NOT NULL, succes... (131 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.global_init_scripts (script_id STRING NOT NU... (120 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.jobs (job_id STRING NOT NULL, success LONG N... (79 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.pipelines (pipeline_id STRING NOT NULL, succ... (99 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.external_locations (location STRING NOT NULL... (40 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.mounts (name STRING NOT NULL, source STRING ... (21 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.grants (principal STRING NOT NULL, action_ty... (167 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.groups (id_in_workspace STRING NOT NULL, nam... (179 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.tables (catalog STRING NOT NULL, database ST... (222 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.table_size (catalog STRING NOT NULL, databas... (81 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.table_failures (catalog STRING NOT NULL, dat... (61 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.workspace_objects (path STRING NOT NULL, obj... (63 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.permissions (object_id STRING NOT NULL, obje... (57 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.submit_runs (run_ids STRING NOT NULL, hashed... (58 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.policies (policy_id STRING NOT NULL, policy_... (147 more bytes)
07:16 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.migration_status (src_schema STRING NOT NULL... (115 more bytes)
07:16 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.udfs (catalog STRING NOT NULL, database STRI... (239 more bytes)
07:16 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.logs (timestamp LONG NOT NULL, job_id LONG N... (179 more bytes)
07:17 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE OR REPLACE VIEW hive_metastore.ucx_sxsox.objects AS SELECT "jobs" AS object_type, job_id ... (1639 more bytes)
07:17 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE OR REPLACE VIEW hive_metastore.ucx_sxsox.grant_detail AS SELECT CASE WHEN anonymous_funct... (1037 more bytes)
07:17 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE OR REPLACE VIEW hive_metastore.ucx_sxsox.table_estimates AS select concat(catalog, ".", d... (1227 more bytes)
07:17 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE OR REPLACE VIEW hive_metastore.ucx_sxsox.misc_patterns AS SELECT col1 AS commandLanguage,... (559 more bytes)
07:17 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE OR REPLACE VIEW hive_metastore.ucx_sxsox.code_patterns AS SELECT col1 AS pattern, col2 AS... (3796 more bytes)
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
07:17 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.gE8a/README for the next steps.
07:17 DEBUG [databricks.labs.ucx.installer.workflows] starting assessment job: https://DATABRICKS_HOST#job/552932542669137
07:30 DEBUG [databricks.labs.ucx.installer.workflows] Validating assessment workflow: https://DATABRICKS_HOST#job/552932542669137
07:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='[email protected]', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='[email protected]')], entitlements=[], external_id=None, groups=[], id='3594324638715029', name=Name(family_name=None, given_name='[email protected]'), roles=[], schemas=[<UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_USER: 'urn:ietf:params:scim:schemas:core:2.0:User'>, <UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_EXTENSION_WORKSPACE_2_0_USER: 'urn:ietf:params:scim:schemas:extension:workspace:2.0:User'>], user_name='[email protected]')
07:05 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group ucx_tnIh: https://DATABRICKS_HOST#setting/accounts/groups/373244128321231
07:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='ucx_tnIh', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='373244128321231', members=[ComplexValue(display='[email protected]', primary=None, ref='Users/3594324638715029', type=None, value='3594324638715029')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[<GroupSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_GROUP: 'urn:ietf:params:scim:schemas:core:2.0:Group'>])
07:05 INFO [databricks.labs.ucx.mixins.fixtures] Account group ucx_tnIh: https://accounts.CLOUD_ENVdatabricks.net/users/groups/1001775630102736/members
07:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='ucx_tnIh', entitlements=[], external_id=None, groups=[], id='1001775630102736', members=[ComplexValue(display='[email protected]', primary=None, ref='Users/3594324638715029', type=None, value='3594324638715029')], meta=None, roles=[], schemas=[<GroupSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_GROUP: 'urn:ietf:params:scim:schemas:core:2.0:Group'>])
07:05 INFO [databricks.labs.ucx.mixins.fixtures] Cluster policy: https://DATABRICKS_HOST#setting/clusters/cluster-policies/view/000BA07CC000327C
07:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster policy fixture: CreatePolicyResponse(policy_id='000BA07CC000327C')
07:05 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster_policy permissions fixture: 000BA07CC000327C [group_name admins CAN_USE] -> [group_name ucx_tnIh CAN_USE]
07:05 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE SCHEMA hive_metastore.ucx_sxsox
07:15 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sxsox: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sxsox
07:15 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sxsox', metastore_id=None, name='ucx_sxsox', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:15 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.gE8a/config.yml) doesn't exist.
07:15 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
07:15 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
07:15 INFO [databricks.labs.ucx.install] Fetching installations...
07:15 INFO [databricks.labs.ucx.installer.policy] Setting up an external metastore
07:15 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
07:15 DEBUG [tests.integration.conftest] Waiting for clusters to start...
07:15 DEBUG [tests.integration.conftest] Waiting for clusters to start...
07:15 INFO [databricks.labs.ucx.install] Installing UCX v0.22.1+2720240503071553
07:15 INFO [databricks.labs.ucx.install] Creating ucx schemas...
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE SCHEMA IF NOT EXISTS hive_metastore.ucx_sxsox
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.CLOUD_ENV_service_principals (application_id STR... (107 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.clusters (cluster_id STRING NOT NULL, succes... (131 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.global_init_scripts (script_id STRING NOT NU... (120 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.jobs (job_id STRING NOT NULL, success LONG N... (79 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.pipelines (pipeline_id STRING NOT NULL, succ... (99 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.external_locations (location STRING NOT NULL... (40 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.mounts (name STRING NOT NULL, source STRING ... (21 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.grants (principal STRING NOT NULL, action_ty... (167 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.groups (id_in_workspace STRING NOT NULL, nam... (179 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.tables (catalog STRING NOT NULL, database ST... (222 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.table_size (catalog STRING NOT NULL, databas... (81 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.table_failures (catalog STRING NOT NULL, dat... (61 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.workspace_objects (path STRING NOT NULL, obj... (63 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.permissions (object_id STRING NOT NULL, obje... (57 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.submit_runs (run_ids STRING NOT NULL, hashed... (58 more bytes)
07:15 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.policies (policy_id STRING NOT NULL, policy_... (147 more bytes)
07:16 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.migration_status (src_schema STRING NOT NULL... (115 more bytes)
07:16 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.udfs (catalog STRING NOT NULL, database STRI... (239 more bytes)
07:16 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE TABLE IF NOT EXISTS hive_metastore.ucx_sxsox.logs (timestamp LONG NOT NULL, job_id LONG N... (179 more bytes)
07:17 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE OR REPLACE VIEW hive_metastore.ucx_sxsox.objects AS SELECT "jobs" AS object_type, job_id ... (1639 more bytes)
07:17 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE OR REPLACE VIEW hive_metastore.ucx_sxsox.grant_detail AS SELECT CASE WHEN anonymous_funct... (1037 more bytes)
07:17 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE OR REPLACE VIEW hive_metastore.ucx_sxsox.table_estimates AS select concat(catalog, ".", d... (1227 more bytes)
07:17 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE OR REPLACE VIEW hive_metastore.ucx_sxsox.misc_patterns AS SELECT col1 AS commandLanguage,... (559 more bytes)
07:17 DEBUG [tests.integration.test_ext_hms] [api][execute] CREATE OR REPLACE VIEW hive_metastore.ucx_sxsox.code_patterns AS SELECT col1 AS pattern, col2 AS... (3796 more bytes)
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
07:17 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
07:17 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.gE8a/README for the next steps.
07:17 DEBUG [databricks.labs.ucx.installer.workflows] starting assessment job: https://DATABRICKS_HOST#job/552932542669137
07:30 DEBUG [databricks.labs.ucx.installer.workflows] Validating assessment workflow: https://DATABRICKS_HOST#job/552932542669137
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 cluster_policy permissions fixtures
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster_policy permissions fixture: 000BA07CC000327C [group_name admins CAN_USE] -> [group_name ucx_tnIh CAN_USE]
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 cluster policy fixtures
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster policy fixture: CreatePolicyResponse(policy_id='000BA07CC000327C')
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while cluster policy CreatePolicyResponse(policy_id='000BA07CC000327C') teardown: Can't find a cluster policy with id: 000BA07CC000327C.
07:30 INFO [databricks.labs.ucx.install] Deleting UCX v0.22.1+2720240503073003 from https://DATABRICKS_HOST
07:30 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sxsox
07:30 DEBUG [tests.integration.test_ext_hms] [api][execute] DROP SCHEMA IF EXISTS hive_metastore.ucx_sxsox CASCADE
07:30 INFO [databricks.labs.ucx.install] Deleting jobs
07:30 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=1068898461395856.
07:30 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=277236911283661.
07:30 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=1083879690919004.
07:30 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=450536702360503.
07:30 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=904346105919442.
07:30 INFO [databricks.labs.ucx.install] Deleting failing job_id=752593203937251.
07:30 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=1092339076953781.
07:30 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=1034103158179352.
07:30 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=411333076634864.
07:30 INFO [databricks.labs.ucx.install] Deleting assessment job_id=552932542669137.
07:30 INFO [databricks.labs.ucx.install] Deleting cluster policy
07:30 INFO [databricks.labs.ucx.install] Deleting secret scope
07:30 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace user fixtures
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='[email protected]', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='[email protected]')], entitlements=[], external_id=None, groups=[], id='3594324638715029', name=Name(family_name=None, given_name='[email protected]'), roles=[], schemas=[<UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_USER: 'urn:ietf:params:scim:schemas:core:2.0:User'>, <UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_EXTENSION_WORKSPACE_2_0_USER: 'urn:ietf:params:scim:schemas:extension:workspace:2.0:User'>], user_name='[email protected]')
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while workspace user User(active=True, display_name='[email protected]', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='[email protected]')], entitlements=[], external_id=None, groups=[], id='3594324638715029', name=Name(family_name=None, given_name='[email protected]'), roles=[], schemas=[<UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_USER: 'urn:ietf:params:scim:schemas:core:2.0:User'>, <UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_EXTENSION_WORKSPACE_2_0_USER: 'urn:ietf:params:scim:schemas:extension:workspace:2.0:User'>], user_name='[email protected]') teardown: None User with id 3594324638715029 not found.
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 account group fixtures
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] removing account group fixture: Group(display_name='ucx_tnIh', entitlements=[], external_id=None, groups=[], id='1001775630102736', members=[ComplexValue(display='[email protected]', primary=None, ref='Users/3594324638715029', type=None, value='3594324638715029')], meta=None, roles=[], schemas=[<GroupSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_GROUP: 'urn:ietf:params:scim:schemas:core:2.0:Group'>])
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace group fixtures
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace group fixture: Group(display_name='ucx_tnIh', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='373244128321231', members=[ComplexValue(display='[email protected]', primary=None, ref='Users/3594324638715029', type=None, value='3594324638715029')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[<GroupSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_GROUP: 'urn:ietf:params:scim:schemas:core:2.0:Group'>])
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while workspace group Group(display_name='ucx_tnIh', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='373244128321231', members=[ComplexValue(display='[email protected]', primary=None, ref='Users/3594324638715029', type=None, value='3594324638715029')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[<GroupSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_GROUP: 'urn:ietf:params:scim:schemas:core:2.0:Group'>]) teardown: None Group with id 373244128321231 not found.
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures
07:30 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sxsox', metastore_id=None, name='ucx_sxsox', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:30 DEBUG [tests.integration.test_ext_hms] [api][execute] DROP SCHEMA IF EXISTS hive_metastore.ucx_sxsox CASCADE
[gw8] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python

Running from nightly #49

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions