Add 5-minute token refresh buffer to pre-emptively refresh early - #1535

## Summary

This PR adds an explicit `PersistentAuth.ForceRefreshToken()` path for
cached U2M OAuth tokens so the CLI can implement `databricks auth token
--force-refresh` on top of the 5-minute proactive refresh buffer
introduced in #1535.

For maintainability reasons it also splits token loading from refresh
decisions and documents a cross-process refresh race for a follow-up
fix.

## Why

#1535 changed the default U2M token path to proactively refresh tokens
that are within 5 minutes of expiry so callers of `databricks auth
token` are less likely to receive a token that expires before it can be
used. That addresses the "give me a useful token" case, which came out
of the `auth token` discussion in [cli issue
#4564](databricks/cli#4564 (comment)).

That proactive refresh is intentionally best-effort, though.
`PersistentAuth.Token()` still returns the existing access token when it
is valid and a proactive refresh fails, because callers did not
explicitly ask to block on a fresh token. That behavior is appropriate
for the default path, but it is not strong enough for integrations that
want to treat the CLI as a token minter or that want to manage their own
cache/TTL policy. In those cases, "return a still-usable token" is
different from "refresh now and give me a newly minted token or fail."

This PR adds that stricter SDK primitive. It gives the CLI a clean way
to implement `--force-refresh` without changing the semantics of the
existing `Token()` flow.

The raw oauth2 error text for forced refresh failures can be misleading.
In particular, when a forced refresh is requested for a cached access
token that is still valid but does not include a refresh token, the
default oauth2 error suggests that the token is expired even though the
real problem is that the cached token is not refresh-capable.

This PR surfaces explicit refresh-token errors instead of relying on the
default oauth2 wording, so callers can distinguish "the current access
token is expired" from "a new token could not be minted because the
refresh token is missing or was rejected by the IdP".

## What changed

### Interface changes

- Added `(*PersistentAuth).ForceRefreshToken() (*oauth2.Token, error)`.
This always attempts to refresh the cached U2M token, regardless of its
current expiry, and redacts the refresh token from the returned value
just like `Token()` does.

### Behavioral changes

- For `Token()`, the only behavioral change is that expired tokens with
no refresh token now return `ErrMissingRefreshToken` instead of a less
specific oauth2 library error. Callers can use `errors.Is(err,
ErrMissingRefreshToken)` to detect this case.
- `PersistentAuth.ForceRefreshToken()` always tries to refresh, even
when the cached access token is still valid.
- If forced refresh fails, `ForceRefreshToken()` returns an error
instead of falling back to the existing token.
- If the cached token does not contain a refresh token,
`ForceRefreshToken()` fails explicitly.

### Internal changes

- Extracted cache lookup into `loadToken()` so token loading and refresh
policy are separated.
- Reused the existing refresh path so forced refresh still goes through
the same endpoint resolution, token exchange, cache update, and
refresh-token redaction logic as the normal path.
- Added dedicated tests for force-refresh success, missing refresh
token, transient refresh failure, invalid refresh token handling, and
the profile-to-host fallback path.
- Documented the remaining cross-process refresh race in `refresh()`
with a follow-up TODO rather than expanding the scope of this PR to add
inter-process coordination.
- Added `ErrMissingRefreshToken` for the "cached token has no refresh
token" case.
- Added missing-refresh-token check into the shared `refresh()` path so
both `Token()` and `ForceRefreshToken()` wrap the same semantic error.

## How is this tested?

- `go test ./credentials/u2m`
- Added unit tests covering:
  - refreshing a still-valid cached token via `ForceRefreshToken()`
- failing `ForceRefreshToken()` when the cached token has no refresh
token
  - surfacing the exact wrapped error on refresh failure
  - preserving `InvalidRefreshTokenError` behavior
- exercising forced refresh through the existing profile-key to host-key
fallback path

---------

Signed-off-by: Mihai Mitrea <[email protected]>

Wire a new `--force-refresh` CLI flag that delegates to the SDK's
`ForceRefreshToken()` method, bypassing the cached token validity check.
The default path through `Token()` is unchanged.

> **Note:** this will not compile until the SDK ships
`ForceRefreshToken()`
([databricks/databricks-sdk-go#1552](databricks/databricks-sdk-go#1552))
and the CLI''s Go SDK dependency is bumped to v0.126.

## Changes

- Add a `--force-refresh` boolean flag to `databricks auth token`. When
set, the command calls `PersistentAuth.ForceRefreshToken()` instead of
`PersistentAuth.Token()`, which always performs a token refresh against
the IdP regardless of the cached token's remaining TTL.
- The error handling is shared between both paths: invalid refresh
tokens produce the same actionable "To reauthenticate, run..." message,
and the backward-compat `cache.ErrNotFound` rewrite is preserved.
- Add three acceptance tests covering the force-refresh flag end-to-end:
- **`force-refresh-success`**: pre-populated cache with a valid token,
`--force-refresh` returns the server's new token (not the cached one).
- **`force-refresh-invalid-refresh-token`**: server returns 401 with
invalid refresh token; asserts the actionable re-login error message.
- **`force-refresh-no-cache`**: no cached token exists; asserts the
backward-compat error message is preserved for `--force-refresh`.
- Add unit tests verifying the default path still returns a cached valid
token without refreshing, and that `--force-refresh` correctly delegates
to `ForceRefreshToken()` for both success and failure cases.

## Why

[#4564](#4564) reports that
external consumers using `databricks auth token` as a credential helper
(e.g. Claude Code via `apiKeyHelper`) can receive near-expired tokens
that expire before they can be used.


[databricks/databricks-sdk-go#1535](databricks/databricks-sdk-go#1535)
addressed the common case by adding a 5-minute proactive refresh buffer
to `Token()`. However, that proactive refresh is intentionally
best-effort: `Token()` still returns the existing access token when it
is valid and a proactive refresh fails, because callers did not
explicitly ask for a fresh token.

For integrations that treat the CLI as a token minter or want to manage
their own cache/TTL policy, "return a still-usable token" is different
from "refresh now and give me a newly minted token or fail." The
`--force-refresh` flag gives those integrations an explicit way to
guarantee a fresh token.

## Tests

- Unit tests in `cmd/auth/token_test.go`: table-driven cases covering
default-path cache reuse, force-refresh success, and force-refresh error
preservation.
- Acceptance tests in
`acceptance/cmd/auth/token/force-refresh-{success,invalid-refresh-token,no-cache}/`:
end-to-end tests against the mock OIDC server, run for both `terraform`
and `direct` engine variants.

---------

Co-authored-by: simon <[email protected]>

Wire a new `--force-refresh` CLI flag that delegates to the SDK's
`ForceRefreshToken()` method, bypassing the cached token validity check.
The default path through `Token()` is unchanged.

> **Note:** this will not compile until the SDK ships
`ForceRefreshToken()`
([databricks/databricks-sdk-go#1552](databricks/databricks-sdk-go#1552))
and the CLI''s Go SDK dependency is bumped to v0.126.

## Changes

- Add a `--force-refresh` boolean flag to `databricks auth token`. When
set, the command calls `PersistentAuth.ForceRefreshToken()` instead of
`PersistentAuth.Token()`, which always performs a token refresh against
the IdP regardless of the cached token's remaining TTL.
- The error handling is shared between both paths: invalid refresh
tokens produce the same actionable "To reauthenticate, run..." message,
and the backward-compat `cache.ErrNotFound` rewrite is preserved.
- Add three acceptance tests covering the force-refresh flag end-to-end:
- **`force-refresh-success`**: pre-populated cache with a valid token,
`--force-refresh` returns the server's new token (not the cached one).
- **`force-refresh-invalid-refresh-token`**: server returns 401 with
invalid refresh token; asserts the actionable re-login error message.
- **`force-refresh-no-cache`**: no cached token exists; asserts the
backward-compat error message is preserved for `--force-refresh`.
- Add unit tests verifying the default path still returns a cached valid
token without refreshing, and that `--force-refresh` correctly delegates
to `ForceRefreshToken()` for both success and failure cases.

## Why

[#4564](#4564) reports that
external consumers using `databricks auth token` as a credential helper
(e.g. Claude Code via `apiKeyHelper`) can receive near-expired tokens
that expire before they can be used.


[databricks/databricks-sdk-go#1535](databricks/databricks-sdk-go#1535)
addressed the common case by adding a 5-minute proactive refresh buffer
to `Token()`. However, that proactive refresh is intentionally
best-effort: `Token()` still returns the existing access token when it
is valid and a proactive refresh fails, because callers did not
explicitly ask for a fresh token.

For integrations that treat the CLI as a token minter or want to manage
their own cache/TTL policy, "return a still-usable token" is different
from "refresh now and give me a newly minted token or fail." The
`--force-refresh` flag gives those integrations an explicit way to
guarantee a fresh token.

## Tests

- Unit tests in `cmd/auth/token_test.go`: table-driven cases covering
default-path cache reuse, force-refresh success, and force-refresh error
preservation.
- Acceptance tests in
`acceptance/cmd/auth/token/force-refresh-{success,invalid-refresh-token,no-cache}/`:
end-to-end tests against the mock OIDC server, run for both `terraform`
and `direct` engine variants.

---------

Co-authored-by: simon <[email protected]>