The customer is having a credentials issue while trying to install UCX. They have tried a couple of things but still getting the same issue. They tried installing UCX, running the upgrade command and manually deleting the workflows in the workspace and then running the install command. Getting the same error in all scenarios. Even installing UCX in a new workspace failed with the same error.
ERROR [d.l.blueprint.parallel][installing_components_0] installing components task failed: org.apache.hadoop.hive.ql.metadata.HiveException: com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: To use assume role profiles the aws-java-sdk-sts module must be on the class path., com.amazonaws.auth.profile.ProfileCredentialsProvider@51650af3: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@4a4831f7: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/]: Traceback (most recent call last):
File "/home/grobert5/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 158, in inner
return func(*args, **kwargs), None
File "/home/grobert5/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 510, in _create_database
raise err
File "/home/grobert5/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 499, in _create_database
deploy_schema(self._sql_backend, self._config.inventory_database)
File "/home/grobert5/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 85, in deploy_schema
deployer.deploy_schema()
File "/home/grobert5/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/labs/lsql/deployment.py", line 18, in deploy_schema
self._sql_backend.execute(f"CREATE SCHEMA IF NOT EXISTS hive_metastore.{self._inventory_schema}")
File "/home/grobert5/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/labs/lsql/backends.py", line 142, in execute
self._sql.execute(sql, catalog=catalog, schema=schema)
File "/home/grobert5/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/labs/lsql/core.py", line 265, in execute
self._raise_if_needed(result_status)
File "/home/grobert5/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/labs/lsql/core.py", line 479, in _raise_if_needed
raise error_class(error_message)
databricks.sdk.errors.platform.BadRequest: org.apache.hadoop.hive.ql.metadata.HiveException: com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: To use assume role profiles the aws-java-sdk-sts module must be on the class path., com.amazonaws.auth.profile.ProfileCredentialsProvider@51650af3: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@4a4831f7: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/]
14:56:04 ERROR [d.l.blueprint.parallel] More than half 'installing components' tasks failed: 0% results available (0/2). Took 0:03:09.057741
Traceback (most recent call last):
File "/home/grobert5/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 830, in <module>
workspace_installer.run()
File "/home/grobert5/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 195, in run
workspace_installation.run()
File "/home/grobert5/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 486, in run
Threads.strict("installing components", install_tasks)
File "/home/grobert5/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 62, in strict
raise errs[0]
File "/home/grobert5/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 158, in inner
return func(*args, **kwargs), None
File "/home/grobert5/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 510, in _create_database
raise err
File "/home/grobert5/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 499, in _create_database
deploy_schema(self._sql_backend, self._config.inventory_database)
File "/home/grobert5/.databricks/labs/ucx/lib/src/databricks/labs/ucx/install.py", line 85, in deploy_schema
deployer.deploy_schema()
File "/home/grobert5/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/labs/lsql/deployment.py", line 18, in deploy_schema
self._sql_backend.execute(f"CREATE SCHEMA IF NOT EXISTS hive_metastore.{self._inventory_schema}")
File "/home/grobert5/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/labs/lsql/backends.py", line 142, in execute
self._sql.execute(sql, catalog=catalog, schema=schema)
File "/home/grobert5/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/labs/lsql/core.py", line 265, in execute
self._raise_if_needed(result_status)
File "/home/grobert5/.databricks/labs/ucx/state/venv/lib/python3.10/site-packages/databricks/labs/lsql/core.py", line 479, in _raise_if_needed
raise error_class(error_message)
databricks.sdk.errors.platform.BadRequest: org.apache.hadoop.hive.ql.metadata.HiveException: com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: To use assume role profiles the aws-java-sdk-sts module must be on the class path., com.amazonaws.auth.profile.ProfileCredentialsProvider@51650af3: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@4a4831f7: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/]
Error: installer: exit status 1
Is there an existing issue for this?
Current Behavior
The customer is having a credentials issue while trying to install UCX. They have tried a couple of things but still getting the same issue. They tried installing UCX, running the upgrade command and manually deleting the workflows in the workspace and then running the install command. Getting the same error in all scenarios. Even installing UCX in a new workspace failed with the same error.
Expected Behavior
UCX installs properly
Steps To Reproduce
No response
Cloud
AWS
Operating System
Linux
Version
latest via Databricks CLI
Relevant log output