Skip to content

Commit e1ef265

Browse files
Pass --force-refresh to CLI auth token command (databricks#1378)
## 🥞 Stacked PR Use this [link](https://github.com/databricks/databricks-sdk-py/pull/1378/files) to review incremental changes. - [stack/cli-force-refresh](databricks#1377) [[Files changed](https://github.com/databricks/databricks-sdk-py/pull/1377/files)] [MERGED] - [**stack/cli-progressive-token-commands**](databricks#1378) [[Files changed](https://github.com/databricks/databricks-sdk-py/pull/1378/files)] --------- ## Summary Append `--force-refresh` to the `databricks auth token` command when the installed CLI is >= v0.296.0, so the CLI bypasses its internal token cache and hands the SDK a freshly minted token every time. Mirrors [databricks/databricks-sdk-go#1628](databricks/databricks-sdk-go#1628). Requires the version-detection infrastructure from the parent PR [databricks#1377](databricks#1377). ## Why The SDK already manages its own token caching via `oauth.Refreshable`. When the SDK decides it needs a new token and shells out to `databricks auth token`, the CLI may return a token from *its* own cache that is about to expire (or that has already expired from the SDK's perspective). That produces unnecessary refresh failures and retry loops on top of a value that the SDK was confident was fresh. The CLI added `--force-refresh` in [databricks/cli#4767](databricks/cli#4767) (motivated by [databricks/cli#4564](databricks/cli#4564)) specifically to let callers bypass the CLI's cache. With the version-detection infrastructure from the parent PR already in place, opting in is a one-constant, one-branch change. ## What changed ### Interface changes None. `CliTokenSource` is not part of the public API surface. ### Behavioral changes - `databricks auth token` invocations now end with `--force-refresh` whenever the detected CLI is >= v0.296.0. Callers on older CLIs see no change. - On older CLIs, a `WARNING` is logged: `"Databricks CLI <ver> does not support --force-refresh (requires >= v0.296.0). The CLI's token cache may provide stale tokens."` `AzureCliTokenSource` is unaffected — it does not pass through `DatabricksCliTokenSource` and does not opt into version-gated flag selection. ### Internal changes - New constant `DatabricksCliTokenSource._CLI_VERSION_FOR_FORCE_REFRESH = CliVersion(0, 296, 0)`. - `_build_cli_command` is split into two helpers, matching the shape the Go SDK settled on after the same PR there: - **`_build_core_cli_command(cli_path, cfg, version)`** — holds the existing profile-vs-host decision (moved out of `_build_cli_command`). - **`_build_cli_command(cli_path, cfg, version)`** — now a thin wrapper that calls `_build_core_cli_command`, appends `--force-refresh` when `version >= _CLI_VERSION_FOR_FORCE_REFRESH`, and otherwise logs the unsupported-version `WARNING`. Future version-gated flags slot into the same pattern: add a `_CLI_VERSION_FOR_<flag>` constant and an `if version >= ...` block in `_build_cli_command`. ## How is this tested? Unit tests in `tests/test_credentials_provider.py`: - Additional `test_build_cli_command` cases covering the full matrix: - `--host` + v0.296.0 → appends `--force-refresh`. - account `--host` + v0.296.0 → appends `--force-refresh`. - `--profile` + v0.296.0 → `--profile` + `--force-refresh`. - `--profile` + v0.207.1 → `--profile` only (too old for `--force-refresh`). - `--profile`-only + v0.296.0 → `--profile` + `--force-refresh` even without a host. - zero version (detection failure) → `--host` only, no `--force-refresh`. - `test_build_cli_command_force_refresh_unsupported_logs_warning` — asserts the `WARNING` log is emitted when `version < _CLI_VERSION_FOR_FORCE_REFRESH`. All parent-PR tests continue to pass unchanged.
1 parent 650b178 commit e1ef265

3 files changed

Lines changed: 136 additions & 1 deletion

File tree

‎NEXT_CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,5 +13,6 @@
1313
### Breaking Changes
1414

1515
### Internal Changes
16+
* Pass `--force-refresh` to Databricks CLI `auth token` command so the SDK always receives a freshly minted token instead of a potentially stale one from the CLI's internal cache.
1617

1718
### API Changes

‎databricks/sdk/credentials_provider.py‎

Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1012,6 +1012,9 @@ def _validate_token_scopes(self, token: oauth.Token):
10121012
# --profile support added in CLI v0.207.1: https://github.com/databricks/cli/pull/855
10131013
_CLI_VERSION_FOR_PROFILE = CliVersion(0, 207, 1)
10141014

1015+
# --force-refresh support added in CLI v0.296.0: https://github.com/databricks/cli/pull/4767
1016+
_CLI_VERSION_FOR_FORCE_REFRESH = CliVersion(0, 296, 0)
1017+
10151018
@staticmethod
10161019
def _parse_cli_version(output: str) -> CliVersion:
10171020
"""Parse the JSON output of `databricks version --output json`.
@@ -1101,7 +1104,34 @@ def _resolve_cli_command(cli_path: str, cfg: "Config") -> List[str]:
11011104

11021105
@staticmethod
11031106
def _build_cli_command(cli_path: str, cfg: "Config", version: CliVersion) -> List[str]:
1104-
"""Build the `auth token` command.
1107+
"""Build the full CLI command, including capability-gated flags.
1108+
1109+
Delegates the profile/host decision to _build_core_cli_command and
1110+
appends --force-refresh when supported.
1111+
"""
1112+
cmd = DatabricksCliTokenSource._build_core_cli_command(cli_path, cfg, version)
1113+
if version >= DatabricksCliTokenSource._CLI_VERSION_FOR_FORCE_REFRESH:
1114+
cmd.append("--force-refresh")
1115+
elif version == CliVersion() or version.is_default_dev_build:
1116+
# Detection failed or no version metadata — we can't prove the CLI
1117+
# lacks --force-refresh, just failed to confirm it. Upstream has
1118+
# already logged the underlying cause, so use softer wording.
1119+
logger.warning(
1120+
f"Could not confirm --force-refresh support for Databricks CLI {version} "
1121+
f"(requires >= {DatabricksCliTokenSource._CLI_VERSION_FOR_FORCE_REFRESH}). "
1122+
"The CLI's token cache may provide stale tokens."
1123+
)
1124+
else:
1125+
logger.warning(
1126+
f"Databricks CLI {version} does not support --force-refresh "
1127+
f"(requires >= {DatabricksCliTokenSource._CLI_VERSION_FOR_FORCE_REFRESH}). "
1128+
"The CLI's token cache may provide stale tokens."
1129+
)
1130+
return cmd
1131+
1132+
@staticmethod
1133+
def _build_core_cli_command(cli_path: str, cfg: "Config", version: CliVersion) -> List[str]:
1134+
"""Build the base `auth token` command without capability-gated flags.
11051135
11061136
Falls back to --host when --profile is either not configured or not
11071137
supported by the installed CLI. Raises IOError describing which

‎tests/test_credentials_provider.py‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -451,6 +451,48 @@ def _make_cfg(*, profile=None, host=None, account_id=None):
451451
_CV(0, 0, 0),
452452
[_CLI, "auth", "token", "--host", _HOST],
453453
),
454+
(
455+
"host with force-refresh",
456+
_make_cfg(host=_HOST),
457+
_CV(0, 296, 0),
458+
[_CLI, "auth", "token", "--host", _HOST, "--force-refresh"],
459+
),
460+
(
461+
"account host with force-refresh",
462+
_make_cfg(host=_ACCT_HOST, account_id="acct-123"),
463+
_CV(0, 296, 0),
464+
[_CLI, "auth", "token", "--host", _ACCT_HOST, "--account-id", "acct-123", "--force-refresh"],
465+
),
466+
(
467+
"profile with force-refresh",
468+
_make_cfg(profile="my-profile", host=_HOST),
469+
_CV(0, 296, 0),
470+
[_CLI, "auth", "token", "--profile", "my-profile", "--force-refresh"],
471+
),
472+
(
473+
"profile supports profile but not force-refresh",
474+
_make_cfg(profile="my-profile", host=_HOST),
475+
_CV(0, 207, 1),
476+
[_CLI, "auth", "token", "--profile", "my-profile"],
477+
),
478+
(
479+
"profile-only with force-refresh",
480+
_make_cfg(profile="my-profile"),
481+
_CV(0, 296, 0),
482+
[_CLI, "auth", "token", "--profile", "my-profile", "--force-refresh"],
483+
),
484+
(
485+
"unknown version, host only, no force-refresh",
486+
_make_cfg(host=_HOST),
487+
_CV(),
488+
[_CLI, "auth", "token", "--host", _HOST],
489+
),
490+
(
491+
"dev-build version, host only, no force-refresh",
492+
_make_cfg(host=_HOST),
493+
_CV(0, 0, 0),
494+
[_CLI, "auth", "token", "--host", _HOST],
495+
),
454496
],
455497
)
456498
def test_build_cli_command(name, cfg, version, expected):
@@ -592,6 +634,68 @@ def test_resolve_cli_command_new_cli_uses_profile(mocker):
592634
assert cmd == [_CLI, "auth", "token", "--profile", "my-profile"]
593635

594636

637+
def test_build_cli_command_force_refresh_unsupported_logs_warning(caplog):
638+
import logging
639+
640+
cfg = _make_cfg(host=_HOST)
641+
with caplog.at_level(logging.WARNING, logger="databricks.sdk.credentials_provider"):
642+
credentials_provider.DatabricksCliTokenSource._build_cli_command(_CLI, cfg, _CV(0, 295, 0))
643+
assert any(
644+
"does not support --force-refresh" in rec.message and rec.levelname == "WARNING" for rec in caplog.records
645+
)
646+
647+
648+
@pytest.mark.parametrize(
649+
"version",
650+
[
651+
# Detection failed: we don't actually know the CLI lacks --force-refresh.
652+
_CV(),
653+
# Default dev build: no version metadata injected, same story.
654+
_CV(0, 0, 0),
655+
],
656+
)
657+
def test_build_cli_command_unconfirmed_force_refresh_softens_warning(caplog, version):
658+
import logging
659+
660+
cfg = _make_cfg(host=_HOST)
661+
with caplog.at_level(logging.WARNING, logger="databricks.sdk.credentials_provider"):
662+
credentials_provider.DatabricksCliTokenSource._build_cli_command(_CLI, cfg, version)
663+
# Softer phrasing for states where --force-refresh support wasn't proven absent.
664+
assert any(
665+
"Could not confirm --force-refresh support" in rec.message and rec.levelname == "WARNING"
666+
for rec in caplog.records
667+
)
668+
assert not any("does not support --force-refresh" in rec.message for rec in caplog.records)
669+
670+
671+
def test_build_cli_command_force_refresh_supported_no_warning(caplog):
672+
import logging
673+
674+
cfg = _make_cfg(host=_HOST)
675+
with caplog.at_level(logging.WARNING, logger="databricks.sdk.credentials_provider"):
676+
credentials_provider.DatabricksCliTokenSource._build_cli_command(_CLI, cfg, _CV(0, 296, 0))
677+
# No --force-refresh-related warning when the flag is supported.
678+
assert not any("--force-refresh" in rec.message for rec in caplog.records)
679+
680+
681+
def test_resolve_cli_command_malformed_version_json_falls_back(mocker, caplog):
682+
# Pin the integrated path: `databricks version --output json` succeeds but
683+
# emits unparseable JSON. _parse_cli_version returns CliVersion() with only
684+
# DEBUG logging, _resolve_cli_command falls back to --host, and no WARNING
685+
# about --profile fires because the unknown sentinel takes the softened branch.
686+
import logging
687+
688+
_stub_version_output(mocker, "{not valid json")
689+
cfg = _make_cfg(profile="my-profile", host=_HOST)
690+
with caplog.at_level(logging.DEBUG, logger="databricks.sdk.credentials_provider"):
691+
cmd = credentials_provider.DatabricksCliTokenSource._resolve_cli_command(_CLI, cfg)
692+
assert cmd == [_CLI, "auth", "token", "--host", _HOST]
693+
assert any(
694+
"Failed to parse Databricks CLI version" in rec.message and rec.levelname == "DEBUG" for rec in caplog.records
695+
)
696+
assert not any("does not support --profile" in rec.message for rec in caplog.records)
697+
698+
595699
# Tests for cloud-agnostic hosts and removed cloud checks
596700
class TestCloudAgnosticHosts:
597701
"""Tests that credential providers work with cloud-agnostic hosts after removing is_azure/is_gcp checks."""

0 commit comments

Comments
 (0)