Authenticating with azure clientID and secret does not work for databricks account api #721
Description
I'm trying to create a metastore with the databricks CLI on my azure environment.
Command databricks metastores create Dataplatform abfss://@.dfs.core.windows.net/unitycatalogs/
Works fine from my local terminal where i login using az login
I try to run it inside a container instance where i login using a clientID and Secret (https://learn.microsoft.com/en-us/azure/databricks/dev-tools/cli/databricks-cli-ref#--azure-service-principal-authentication)
This works well for all workspace related stuff, but creating a metastore on the accountlevel does not work. trace export of the command:
json
" time=2023-08-31T08:51:48.865Z level=TRACE source="config.go 193}"
msg="Loading config via environment" sdk=true time=2023-08-31T08:51:48.865Z level=TRACE source="config.go 193}"
msg="Loading config via config-file" sdk=true time=2023-08-31T08:51:48.865Z level=DEBUG source="config_file.go 96}"
msg="Loading profile profile from /root/.databrickscfg" sdk=true time=2023-08-31T08:51:48.865Z level=TRACE source="auth_default.go 52}"
msg="Attempting to configure auth: pat" sdk=true time=2023-08-31T08:51:48.866Z level=TRACE source="auth_default.go 52}"
msg="Attempting to configure auth: basic" sdk=true time=2023-08-31T08:51:48.866Z level=TRACE source="auth_default.go 52}"
msg="Attempting to configure auth: oauth-m2m" sdk=true time=2023-08-31T08:51:48.866Z level=TRACE source="auth_default.go 52}"
msg="Attempting to configure auth: databricks-cli" sdk=true time=2023-08-31T08:51:48.866Z level=TRACE source="auth_default.go 52}"
msg="Attempting to configure auth: metadata-service" sdk=true time=2023-08-31T08:51:48.866Z level=TRACE source="auth_default.go 52}"
msg="Attempting to configure auth: azure-msi" sdk=true time=2023-08-31T08:51:48.866Z level=TRACE source="auth_default.go 52}"
msg="Attempting to configure auth: azure-client-secret" sdk=true time=2023-08-31T08:51:48.866Z level=INFO source="auth_azure_client_secret.go 53}"
msg="Generating AAD token for Service Principal (fab09e2c-f5bd-44a5-8f4e-cfc9f667babe)" sdk=true time=2023-08-31T08:51:49.696Z level=DEBUG source="client.go 264}"
msg="GET /api/2.0/preview/scim/v2/Me\n< HTTP/2.0 200 OK\n[non-JSON document of 727 bytes]. <!doctype html><html lang=\"en\"><head><meta charset=\"utf-8\"/><meta name=\"viewport\" content=\"width... (631 more bytes)" sdk=true Error: invalid character '<' looking for beginning of value time=2023-08-31T08:51:49.698Z level=ERROR source="root.go 110}"
msg="failed execution" exit_code=1 error="invalid character '<' looking for beginning of value"
It seems the parsing the return of the token request is not correct.