[BUG] databricks labs upgrade fails when dependencies used in install script have breaking changes #1623
Description
Describe the issue
The databricks labs upgrade command fails when dependencies used in install script have breaking changes.
See the following issues for ucx reporting problems related to this:
- [BUG]: Unable to update UCX/reinstall UCX /run UCX commands in Databricks CLI databrickslabs/ucx#2223
- [BUG]: Not able to run databricks labs install ucx databrickslabs/ucx#2220
- [BUG]: cannot import name 'with_user_agent_extra' from 'databricks.sdk.core' databrickslabs/ucx#2102
Steps to reproduce the behavior
Please list the steps required to reproduce the issue, for example:
- Install older ucx version:
databricks labs install [email protected] - Upgrade to latest ucx version:
databricks labs upgrade ucx
Expected Behavior
Upgrade should not fail
Actual Behavior
Upgrade fails due to missing dependencies.
OS and CLI version
Please include the version of the CLI (eg: v0.1.2) and the operating system (eg: windows). You can run databricks --version to get the version of your Databricks CLI
Is this a regression?
No
Debug Logs
Debug log for databricks --debug labs upgrade ucx
~/github/databrickslabs/ucx fix/install-deployment *2 ❯ databricks --debug labs upgrade ucx ▼ Py ucx 11:59:40
11:59:43 INFO start pid=46827 version=0.216.0 args="databricks, --debug, labs, upgrade, ucx"
11:59:43 DEBUG Latest ucx version is: v0.29.0 pid=46827
11:59:43 DEBUG Reading [email protected] from databrickslabs/ucx pid=46827
⣾ Cleaning up previous installation if necessary11:59:43 DEBUG Downloading zipball for v0.29.0 from databrickslabs/ucx pid=46827
⣷ Downloading and unpacking zipball for v0.29.011:59:45 DEBUG Unpacking zipball to: /Users/user/.databricks/labs/ucx/lib pid=46827
11:59:45 DEBUG Writing installed version info to: /Users/user/.databricks/labs/ucx/state/version.json pid=46827
11:59:45 DEBUG Launching installer script src/databricks/labs/ucx/install.py in /Users/user/.databricks/labs/ucx/lib pid=46827
11:59:45 DEBUG Loading login configuration from: /Users/user/.databricks/labs/ucx/config/login.json pid=46827
11:59:45 DEBUG Using workspace-level login profile: DEFAULT pid=46827
11:59:45 DEBUG Loading DEFAULT profile from /Users/user/.databrickscfg pid=46827 sdk=true
11:59:45 DEBUG Resolved login: Config: host=adb-REDACTED.azuredatabricks.net, cluster_id=REDACTED, account_id=REDACTED, profile=DEFAULT, config_file=/user/.databrickscfg. Env: DATABRICKS_HOST, DATABRICKS_CLUSTER_ID, DATABRICKS_ACCOUNT_ID pid=46827 sdk=true
11:59:45 DEBUG Passing down environment variables: DATABRICKS_ACCOUNT_ID, DATABRICKS_AUTH_TYPE, DATABRICKS_HOST, DATABRICKS_CLUSTER_ID pid=46827
11:59:45 DEBUG starting: /Users/user/.databricks/labs/ucx/state/venv/bin/python3 src/databricks/labs/ucx/install.py pid=46827
Traceback (most recent call last):
File "/Users/user/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 27, in <module>
from databricks.labs.lsql.dashboards import DashboardMetadata, Dashboards
ModuleNotFoundError: No module named 'databricks.labs.lsql.dashboards'
Error: installer: exit status 1
11:59:46 ERROR failed execution pid=46827 exit_code=1 error="installer: exit status 1"