Pass --force-refresh to CLI auth token command - #1628

## Release v0.130.0

### Breaking Changes

* Remove the `Experimental_IsUnifiedHost` field (and the `DATABRICKS_EXPERIMENTAL_IS_UNIFIED_HOST` environment variable) from `Config`. Unified host detection is now automatic via the `/.well-known/databricks-config` endpoint.
* Remove the unused `ErrWorkspaceIDInAccountClient` exported variable. It was never returned from any production path, and its message contradicted the unified host workflow where a single profile with both `AccountID` and `WorkspaceID` produces both clients.
* Remove the file-based OAuth token cache from `credentials/u2m/cache`. The removed symbols are `cache.NewFileTokenCache`, `cache.FileTokenCacheOption`, `cache.WithFileLocation`, and the private `tokenCacheFile` struct. The `TokenCache` interface, `ErrNotFound` sentinel, `HostCacheKeyProvider`, and `DiscoveryOAuthArgument` remain exported. `NewPersistentAuth` now defaults to a new in-memory cache (`cache.NewInMemoryTokenCache`) when no `WithTokenCache` option is passed; consumers that relied on the previous file-backed default must supply their own persistent cache. See databricks/cli#5056 for the companion CLI change that moves the file cache into the CLI.

### New Features and Improvements

* Add `u2m.WithDiscoveryHost` option to override the default `https://login.databricks.com` host used by the discovery login flow. Intended for testing and development against non-production environments.
* Add support for unified hosts. A single configuration profile can now be used for both account-level and workspace-level operations when the host supports it and both `AccountID` and `WorkspaceID` are available.

### Bug Fixes

 * Fix CLI token source `--profile` fallback: `--profile` is a global Cobra flag that old CLIs accept silently instead of reporting "unknown flag", making the previous error-based detection dead code. Now uses `databricks version` to detect CLI capabilities at init time ([#1605](#1605)).

### Internal Changes

 * Pass `--force-refresh` to Databricks CLI `auth token` command to bypass the CLI's internal token cache ([#1628](#1628)).

### API Changes
* Add [w.TemporaryVolumeCredentials](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#TemporaryVolumeCredentialsAPI) workspace-level service.
* Add `GetPermissionLevels`, `GetPermissions`, `SetPermissions` and `UpdatePermissions` methods for [w.KnowledgeAssistants](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/knowledgeassistants#KnowledgeAssistantsAPI) workspace-level service.
* Add `ThumbnailUrl` field for [apps.App](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#App).
* Add `JiraOptions`, `OutlookOptions` and `SmartsheetOptions` fields for [pipelines.ConnectorOptions](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#ConnectorOptions).
* Add `GoogleAdsConfig` field for [pipelines.SourceConfig](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#SourceConfig).
* Add `ReplaceExisting` field for [postgres.CreateBranchRequest](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/postgres#CreateBranchRequest).
* Add `ReplaceExisting` field for [postgres.CreateEndpointRequest](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/postgres#CreateEndpointRequest).

The SDK manages its own token caching via `CachedTokenSource`. When
the SDK shells out to `databricks auth token`, the CLI may return a
token from *its* own cache that is about to expire (or has already
expired from the SDK's perspective), producing unnecessary refresh
failures and retry loops.

The CLI added `--force-refresh` in v0.296.0
(databricks/cli#4767) to let callers bypass
its cache. With the version-detection infrastructure from the parent
PR already in place, opting in is a one-constant, one-branch change:

* Introduce `CLI_VERSION_FOR_FORCE_REFRESH = v0.296.0`.
* Split `buildCliCommand` into the existing profile/host decision
  (now `buildCoreCliCommand`) and a thin wrapper that appends
  `--force-refresh` when supported and otherwise logs a precise
  warning.

Future capability-gated flags slot into the same wrapper.

Mirrors:
* databricks/databricks-sdk-go#1628
* databricks/databricks-sdk-py#1378

Co-authored-by: Isaac

The SDK manages its own token caching via `CachedTokenSource`. When
the SDK shells out to `databricks auth token`, the CLI may return a
token from *its* own cache that is about to expire (or has already
expired from the SDK's perspective), producing unnecessary refresh
failures and retry loops.

The CLI added `--force-refresh` in v0.296.0
(databricks/cli#4767) to let callers bypass
its cache. With the version-detection infrastructure from the parent
PR already in place, opting in is a one-constant, one-branch change:

* Introduce `CLI_VERSION_FOR_FORCE_REFRESH = v0.296.0`.
* Split `buildCliCommand` into the existing profile/host decision
  (now `buildCoreCliCommand`) and a thin wrapper that appends
  `--force-refresh` when supported and otherwise logs a precise
  warning.

Future capability-gated flags slot into the same wrapper.

Mirrors:
* databricks/databricks-sdk-go#1628
* databricks/databricks-sdk-py#1378

Co-authored-by: Isaac

The SDK manages its own token caching via `CachedTokenSource`. When
the SDK shells out to `databricks auth token`, the CLI may return a
token from *its* own cache that is about to expire (or has already
expired from the SDK's perspective), producing unnecessary refresh
failures and retry loops.

The CLI added `--force-refresh` in v0.296.0
(databricks/cli#4767) to let callers bypass
its cache. With the version-detection infrastructure from the parent
PR already in place, opting in is a one-constant, one-branch change:

* Introduce `CLI_VERSION_FOR_FORCE_REFRESH = v0.296.0`.
* Split `buildCliCommand` into the existing profile/host decision
  (now `buildCoreCliCommand`) and a thin wrapper that appends
  `--force-refresh` when supported and otherwise logs a precise
  warning.

Future capability-gated flags slot into the same wrapper.

Mirrors:
* databricks/databricks-sdk-go#1628
* databricks/databricks-sdk-py#1378

Co-authored-by: Isaac

## 🥞 Stacked PR
Use this
[link](https://github.com/databricks/databricks-sdk-py/pull/1378/files)
to review incremental changes.
-
[stack/cli-force-refresh](databricks#1377)
[[Files
changed](https://github.com/databricks/databricks-sdk-py/pull/1377/files)]
[MERGED]
-
[**stack/cli-progressive-token-commands**](databricks#1378)
[[Files
changed](https://github.com/databricks/databricks-sdk-py/pull/1378/files)]

---------
## Summary

Append `--force-refresh` to the `databricks auth token` command when the
installed CLI is >= v0.296.0, so the CLI bypasses its internal token
cache and hands the SDK a freshly minted token every time.

Mirrors
[databricks/databricks-sdk-go#1628](databricks/databricks-sdk-go#1628).
Requires the version-detection infrastructure from the parent PR
[databricks#1377](databricks#1377).

## Why

The SDK already manages its own token caching via `oauth.Refreshable`.
When the SDK decides it needs a new token and shells out to `databricks
auth token`, the CLI may return a token from *its* own cache that is
about to expire (or that has already expired from the SDK's
perspective). That produces unnecessary refresh failures and retry loops
on top of a value that the SDK was confident was fresh.

The CLI added `--force-refresh` in
[databricks/cli#4767](databricks/cli#4767)
(motivated by
[databricks/cli#4564](databricks/cli#4564))
specifically to let callers bypass the CLI's cache. With the
version-detection infrastructure from the parent PR already in place,
opting in is a one-constant, one-branch change.

## What changed

### Interface changes

None. `CliTokenSource` is not part of the public API surface.

### Behavioral changes

- `databricks auth token` invocations now end with `--force-refresh`
whenever the detected CLI is >= v0.296.0. Callers on older CLIs see no
change.
- On older CLIs, a `WARNING` is logged: `"Databricks CLI <ver> does not
support --force-refresh (requires >= v0.296.0). The CLI's token cache
may provide stale tokens."`

`AzureCliTokenSource` is unaffected — it does not pass through
`DatabricksCliTokenSource` and does not opt into version-gated flag
selection.

### Internal changes

- New constant `DatabricksCliTokenSource._CLI_VERSION_FOR_FORCE_REFRESH
= CliVersion(0, 296, 0)`.
- `_build_cli_command` is split into two helpers, matching the shape the
Go SDK settled on after the same PR there:
- **`_build_core_cli_command(cli_path, cfg, version)`** — holds the
existing profile-vs-host decision (moved out of `_build_cli_command`).
- **`_build_cli_command(cli_path, cfg, version)`** — now a thin wrapper
that calls `_build_core_cli_command`, appends `--force-refresh` when
`version >= _CLI_VERSION_FOR_FORCE_REFRESH`, and otherwise logs the
unsupported-version `WARNING`.

Future version-gated flags slot into the same pattern: add a
`_CLI_VERSION_FOR_<flag>` constant and an `if version >= ...` block in
`_build_cli_command`.

## How is this tested?

Unit tests in `tests/test_credentials_provider.py`:

- Additional `test_build_cli_command` cases covering the full matrix:
  - `--host` + v0.296.0 → appends `--force-refresh`.
  - account `--host` + v0.296.0 → appends `--force-refresh`.
  - `--profile` + v0.296.0 → `--profile` + `--force-refresh`.
- `--profile` + v0.207.1 → `--profile` only (too old for
`--force-refresh`).
- `--profile`-only + v0.296.0 → `--profile` + `--force-refresh` even
without a host.
- zero version (detection failure) → `--host` only, no
`--force-refresh`.
- `test_build_cli_command_force_refresh_unsupported_logs_warning` —
asserts the `WARNING` log is emitted when `version <
_CLI_VERSION_FOR_FORCE_REFRESH`.

All parent-PR tests continue to pass unchanged.

The SDK manages its own token caching via `CachedTokenSource`. When
the SDK shells out to `databricks auth token`, the CLI may return a
token from *its* own cache that is about to expire (or has already
expired from the SDK's perspective), producing unnecessary refresh
failures and retry loops.

The CLI added `--force-refresh` in v0.296.0
(databricks/cli#4767) to let callers bypass
its cache. With the version-detection infrastructure from the parent
PR already in place, opting in is a one-constant, one-branch change:

* Introduce `CLI_VERSION_FOR_FORCE_REFRESH = v0.296.0`.
* Split `buildCliCommand` into the existing profile/host decision
  (now `buildCoreCliCommand`) and a thin wrapper that appends
  `--force-refresh` when supported and otherwise logs a precise
  warning.

Future capability-gated flags slot into the same wrapper.

Mirrors:
* databricks/databricks-sdk-go#1628
* databricks/databricks-sdk-py#1378

Co-authored-by: Isaac

The SDK manages its own token caching via `CachedTokenSource`. When
the SDK shells out to `databricks auth token`, the CLI may return a
token from *its* own cache that is about to expire (or has already
expired from the SDK's perspective), producing unnecessary refresh
failures and retry loops.

The CLI added `--force-refresh` in v0.296.0
(databricks/cli#4767) to let callers bypass
its cache. With the version-detection infrastructure from the parent
PR already in place, opting in is a one-constant, one-branch change:

* Introduce `CLI_VERSION_FOR_FORCE_REFRESH = v0.296.0`.
* Split `buildCliCommand` into the existing profile/host decision
  (now `buildCoreCliCommand`) and a thin wrapper that appends
  `--force-refresh` when supported and otherwise logs a precise
  warning.

Future capability-gated flags slot into the same wrapper.

Mirrors:
* databricks/databricks-sdk-go#1628
* databricks/databricks-sdk-py#1378

Co-authored-by: Isaac

The SDK manages its own token caching via `CachedTokenSource`. When
the SDK shells out to `databricks auth token`, the CLI may return a
token from *its* own cache that is about to expire (or has already
expired from the SDK's perspective), producing unnecessary refresh
failures and retry loops.

The CLI added `--force-refresh` in v0.296.0
(databricks/cli#4767) to let callers bypass
its cache. With the version-detection infrastructure from the parent
PR already in place, opting in is a one-constant, one-branch change:

* Introduce `CLI_VERSION_FOR_FORCE_REFRESH = v0.296.0`.
* Split `buildCliCommand` into the existing profile/host decision
  (now `buildCoreCliCommand`) and a thin wrapper that appends
  `--force-refresh` when supported and otherwise logs a precise
  warning.

Future capability-gated flags slot into the same wrapper.

Mirrors:
* databricks/databricks-sdk-go#1628
* databricks/databricks-sdk-py#1378

Co-authored-by: Isaac

The SDK manages its own token caching via `CachedTokenSource`. When
the SDK shells out to `databricks auth token`, the CLI may return a
token from *its* own cache that is about to expire (or has already
expired from the SDK's perspective), producing unnecessary refresh
failures and retry loops.

The CLI added `--force-refresh` in v0.296.0
(databricks/cli#4767) to let callers bypass
its cache. With the version-detection infrastructure from the parent
PR already in place, opting in is a one-constant, one-branch change:

* Introduce `CLI_VERSION_FOR_FORCE_REFRESH = v0.296.0`.
* Split `buildCliCommand` into the existing profile/host decision
  (now `buildCoreCliCommand`) and a thin wrapper that appends
  `--force-refresh` when supported and otherwise logs a precise
  warning.

Future capability-gated flags slot into the same wrapper.

Mirrors:
* databricks/databricks-sdk-go#1628
* databricks/databricks-sdk-py#1378

Co-authored-by: Isaac

The SDK manages its own token caching via `CachedTokenSource`. When
the SDK shells out to `databricks auth token`, the CLI may return a
token from *its* own cache that is about to expire (or has already
expired from the SDK's perspective), producing unnecessary refresh
failures and retry loops.

The CLI added `--force-refresh` in v0.296.0
(databricks/cli#4767) to let callers bypass
its cache. With the version-detection infrastructure from the parent
PR already in place, opting in is a one-constant, one-branch change:

* Introduce `CLI_VERSION_FOR_FORCE_REFRESH = v0.296.0`.
* Split `buildCliCommand` into the existing profile/host decision
  (now `buildCoreCliCommand`) and a thin wrapper that appends
  `--force-refresh` when supported and otherwise logs a precise
  warning.

Future capability-gated flags slot into the same wrapper.

Mirrors:
* databricks/databricks-sdk-go#1628
* databricks/databricks-sdk-py#1378

Co-authored-by: Isaac

## 🥞 Stacked PR
Use this
[link](https://github.com/databricks/databricks-sdk-java/pull/752/files)
to review incremental changes.
-
[stack/cli-force-refresh](databricks#751)
[[Files
changed](https://github.com/databricks/databricks-sdk-java/pull/751/files)]
-
[**stack/cli-attempt-chain**](databricks#752)
[[Files
changed](https://github.com/databricks/databricks-sdk-java/pull/752/files)]

---------
## Summary

Append `--force-refresh` to the `databricks auth token` command when the
installed CLI is >= v0.296.0, so the CLI bypasses its internal token
cache and hands the SDK a freshly minted token every time.

Mirrors
[databricks/databricks-sdk-go#1628](databricks/databricks-sdk-go#1628)
and
[databricks/databricks-sdk-py#1378](databricks/databricks-sdk-py#1378).
Requires the version-detection infrastructure from the parent PR
[databricks#751](databricks#751).

## Why

The SDK already manages its own token caching via `CachedTokenSource`.
When the SDK decides it needs a new token and shells out to `databricks
auth token`, the CLI may return a token from *its* own cache that is
about to expire (or that has already expired from the SDK's
perspective). That produces unnecessary refresh failures and retry loops
on top of a value that the SDK was confident was fresh.

The CLI added `--force-refresh` in
[databricks/cli#4767](databricks/cli#4767)
(motivated by
[databricks/cli#4564](databricks/cli#4564))
specifically to let callers bypass the CLI's cache. With the
version-detection infrastructure from the parent PR already in place,
opting in is a one-constant, one-branch change.

## What changed

### Interface changes

None. `CliTokenSource` is not part of the public API surface.

### Behavioral changes

- `databricks auth token` invocations now end with `--force-refresh`
whenever the detected CLI is >= v0.296.0. Callers on older CLIs see no
change.
- On older CLIs, a `WARNING` is logged: `"Databricks CLI <ver> does not
support --force-refresh (requires >= v0.296.0). The CLI's token cache
may provide stale tokens."`

`AzureCliCredentialsProvider` is unaffected — it does not pass through
`DatabricksCliCredentialsProvider` and does not opt into version-gated
flag selection.

### Internal changes

- New constant
`DatabricksCliCredentialsProvider.CLI_VERSION_FOR_FORCE_REFRESH = new
DatabricksCliVersion(0, 296, 0)`.
- `buildCliCommand` is split into two helpers, matching the shape the Go
and Python SDKs settled on after the same PR there:
- **`buildCoreCliCommand(cliPath, config, version)`** — holds the
existing profile-vs-host decision (moved out of `buildCliCommand`).
- **`buildCliCommand(cliPath, config, version)`** — now a thin wrapper
that calls `buildCoreCliCommand`, appends `--force-refresh` when
`version.atLeast(CLI_VERSION_FOR_FORCE_REFRESH)`, and otherwise logs the
unsupported-version `WARNING`.

Future version-gated flags slot into the same pattern: add a
`CLI_VERSION_FOR_<flag>` constant and an `if version.atLeast(...)` block
in `buildCliCommand`.

## How is this tested?

Additional `testBuildCliCommand` parameterized cases in
`DatabricksCliCredentialsProviderTest` cover the full matrix:

- `--host` + v0.296.0 → appends `--force-refresh`.
- account `--host` + v0.296.0 → appends `--force-refresh`.
- `--profile` + v0.296.0 → `--profile` + `--force-refresh`.
- `--profile` + v0.207.1 → `--profile` only (too old for
`--force-refresh`).
- `--host` + v0.295.0 → `--host` only (too old for `--force-refresh`).
- unknown version (detection failure) → `--host` only, no
`--force-refresh`.
- dev build → `--host` only, no `--force-refresh`.

All parent-PR tests continue to pass unchanged.