Skip to content

[BUG]: Cannot Install/Uninstall: "Cluster policy not present"/ "Missing required field: policy_id" #1012

@koberlag-vizio

Description

@koberlag-vizio

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

When I try to install, uninstall, or upgrade ucx, I receive errors. I installed successfully the first time and ran the analysis job. When I attempted to uninstall, it seems that it partially uninstalled, then failed, and now I cannot complete an uninstall or an install, no matter what options I select.

Expected Behavior

Successful Install/Uninstall

Steps To Reproduce

No response

Cloud

AWS

Operating System

Linux

Version

latest via Databricks CLI

Relevant log output

$ databricks labs install ucx
22:46:34  INFO [d.l.ucx.install] Installing UCX v0.14.0
22:46:34  INFO [d.l.ucx.install] Installing UCX v0.14.0
22:46:34  INFO [d.l.ucx.install][installing_components_0] Creating dashboards...
22:46:34  INFO [d.l.ucx.install][installing_components_0] Fetching warehouse_id from a config
22:46:35 ERROR [d.l.blueprint.parallel][installing_components_0] installing components task failed: 'NoneType' object is not a mapping: Traceback (most recent call last):
  File "/home/REDACTED/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 127, in inner
    return func(*args, **kwargs), None
  File "/home/REDACTED/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 428, in _create_dashboards
    dash.create_dashboards()
  File "/home/REDACTED/.databricks/labs/ucx/lib/src/databricks/labs/ucx/framework/dashboards.py", line 113, in create_dashboards
    self._install_dashboard(dashboard_name, parent_folder_id, dashboard_ref)
  File "/home/REDACTED/.databricks/labs/ucx/lib/src/databricks/labs/ucx/framework/dashboards.py", line 215, in _install_dashboard
    self._ws.dashboard_widgets.delete(widget.id)
  File "/home/REDACTED/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/sdk/service/sql.py", line 3985, in delete
    self._api.do('DELETE', f'/api/2.0/preview/sql/widgets/{id}', headers=headers)
  File "/home/REDACTED/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 152, in do
    return {**resp, **json}
TypeError: 'NoneType' object is not a mapping
22:46:37 ERROR [d.l.blueprint.parallel][installing_components_2] installing components task failed: Cluster policy not present, please uninstall and reinstall ucx completely.: Traceback (most recent call last):
  File "/home/REDACTED/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 127, in inner
    return func(*args, **kwargs), None
  File "/home/REDACTED/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 596, in create_jobs
    self._upload_cluster_policy(remote_wheel)
  File "/home/REDACTED/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 579, in _upload_cluster_policy
    raise InvalidParameterValue(msg)
databricks.sdk.errors.platform.InvalidParameterValue: Cluster policy not present, please uninstall and reinstall ucx completely.
22:46:40 ERROR [d.l.blueprint.parallel] More than half 'installing components' tasks failed: 0% results available (0/3). Took 0:00:06.391709
Traceback (most recent call last):
  File "/home/REDACTED/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 1013, in <module>
    installer.run()
  File "/home/REDACTED/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 193, in run
    workspace_installation.run()
  File "/home/REDACTED/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 387, in run
    Threads.strict(
  File "/home/REDACTED/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 54, in strict
    raise ManyError(errs)
databricks.labs.blueprint.parallel.ManyError: Detected 2 failures: InvalidParameterValue: Cluster policy not present, please uninstall and reinstall ucx completely., TypeError: 'NoneType' object is not a mapping
Error: installer: exit status 1




$ databricks labs uninstall ucx
Do you want to uninstall ucx from the workspace too, this would remove ucx project folder, dashboards, queries and jobs (default: no): yes
22:42:37  INFO [d.l.ucx.install] Deleting UCX v0.14.0 from https://REDACTED.cloud.databricks.com
Do you want to delete the inventory database ucx too? (default: no): yes
22:42:40  INFO [d.l.ucx.install] Deleting inventory database ucx
22:42:46  INFO [d.l.ucx.install] Deleting jobs
22:42:46  INFO [d.l.ucx.install] Deleting migrate-groups job_id=REDACTED.
22:42:46 ERROR [d.l.ucx.install] Already deleted: migrate-groups job_id=REDACTED.
22:42:46  INFO [d.l.ucx.install] Deleting remove-workspace-local-backup-groups job_id=REDACTED.
22:42:46 ERROR [d.l.ucx.install] Already deleted: remove-workspace-local-backup-groups job_id=REDACTED.
22:42:46  INFO [d.l.ucx.install] Deleting 099-destroy-schema job_id=REDACTED.
22:42:46 ERROR [d.l.ucx.install] Already deleted: 099-destroy-schema job_id=REDACTED.
22:42:46  INFO [d.l.ucx.install] Deleting assessment job_id=REDACTED.
22:42:46 ERROR [d.l.ucx.install] Already deleted: assessment job_id=REDACTED.
22:42:46  INFO [d.l.ucx.install] Deleting cluster policy
Traceback (most recent call last):
  File "/home/REDACTED/.databricks/labs/ucx/lib/src/databricks/labs/ucx/uninstall.py", line 14, in <module>
    installer.uninstall()
  File "/home/REDACTED/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 940, in uninstall
    self._remove_policies()
  File "/home/REDACTED/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 956, in _remove_policies
    self._ws.cluster_policies.delete(policy_id=self.config.policy_id)
  File "/home/REDACTED/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/sdk/service/compute.py", line 5581, in delete
    self._api.do('POST', '/api/2.0/policies/clusters/delete', body=body, headers=headers)
  File "/home/REDACTED/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 130, in do
    response = retryable(self._perform)(method,
  File "/home/REDACTED/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper
    raise err
  File "/home/REDACTED/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper
    return func(*args, **kwargs)
  File "/home/REDACTED/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 238, in _perform
    raise self._make_nicer_error(response=response, **payload) from None
databricks.sdk.errors.platform.InvalidParameterValue: Missing required field: policy_id
Error: uninstall hook: exit status 1

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions