Skip to content

Commit cc3bcd3

Browse files
denikIsaac
denik
and
Isaac
authored
Treat backend-populated immutable fields as backend defaults by default (#6790)
An immutable (`recreate_on_changes`) field the bundle config never set but the backend populated — `old` and `new` nil, `remote` set — is now left in place instead of triggering delete + create, reported as `immutable_backend_value`. This generalizes the `trace_location` fix ([#6787](#6787)) to **every** optional immutable field: the user isn't managing the field, and recreating to drop a backend-populated immutable value is destructive. The check is shape-based (config-absent + backend-present), so it only fires when the user omitted the field; a genuine change to an immutable field still recreates. ### Fields that already behaved this way (explicit `backend_defaults`, now removed — implicit) These had a hand-written `backend_defaults` entry that duplicated the new implicit rule; removed here: | Field | Prior link | |---|---| | `secret_scopes.scope_backend_type` | [#4834](#4834); note this field is also `missing_in_remote`, so the entry was already inert | | `volumes.storage_location` | [#5145](#5145) (external volumes) | | `experiments.artifact_location` | initial direct-engine field classification | | `experiments.trace_location` | [#6787](#6787) | | `registered_models.storage_location` | initial direct-engine field classification | | `pipelines.storage` | initial direct-engine field classification | | `model_serving_endpoints.route_optimized` | was `values:[false]`-restricted; now unconditional | | `vector_search_indexes.index_subtype` | initial direct-engine field classification | `backend_defaults` entries on **non**-recreate fields are kept (e.g. `pipelines.clusters[*].node_type_id`, `model_serving_endpoints.config.traffic_config`, `registered_models.owner/full_name/metastore_id`). ### Fields newly covered by the implicit default Previously these would recreate on a config-absent, backend-populated value; now left in place. (Impact is real only for the subset the backend actually populates when omitted — flagged **⚠**.) - **catalogs**: `storage_root` ⚠, `connection_name`, `provider_name`, `share_name` - **schemas**: `storage_root` ⚠ - **dashboards**: `parent_path` ⚠ - **model_serving_endpoints**: `config.auto_capture_config.catalog_name` ⚠, `.schema_name` ⚠, `.table_name_prefix` ⚠, `description` - **quality_monitors**: `assets_dir` ⚠ - **volumes**: `volume_type` ⚠ - **vector_search_indexes**: `index_type`, `primary_key`, `endpoint_name`, `delta_sync_index_spec`, `direct_access_index_spec` (non-ignored leaves) - **vector_search_endpoints**: `endpoint_type` ⚠ - **external_locations**: `credential_name`, `encryption_details`, `file_event_queue` - **database_instances**: `parent_instance_ref` (non-ignored leaves) - **instance_pools**: `disk_spec`, `node_type_flexibility`, `preloaded_spark_versions`, `preloaded_docker_images` - **pipelines**: `gateway_definition.*`, `ingestion_definition.*` - **model_provider_services**: `config.provider_type` - **postgres_roles**: `parent`, `role_id` - **secrets**: `catalog_name`, `name`, `schema_name` (identity fields; inert in practice) ### Whole-struct (field-less) recreate resources These declare the entire resource immutable (`recreate_on_changes: - reason: immutable`). Any config-absent, backend-populated field is now left in place rather than recreating the whole resource: `database_catalogs`, `internal_immutable_snapshots`, `job_runs`, `postgres_synced_tables`, `synced_database_tables`, `postgres_catalogs`. (`job_runs`' re-run-on-result-state logic is driven by `OverrideChangeDesc`, which runs after classification and is unaffected.) ### Not affected - Recreate fields already covered by `ignore_remote_changes` (already skipped when config-absent): `postgres_branches.*`, `postgres_endpoints.endpoint_type`, `postgres_projects.pg_version`, `postgres_roles.{postgres_role,auth_method,identity_type}`, `postgres_catalogs.postgres_database`, `experiments.trace_location.uc_trace_location.effective_table_prefix`, `vector_search_indexes.direct_access_index_spec.schema_json`, `database_instances.parent_instance_ref.{uid,lsn,effective_lsn}`. - `missing_in_remote` recreate fields (never echoed, so the shape can't occur): `secret_scopes.{scope_backend_type,backend_azure_keyvault,initial_manage_principal}`. ### Testing `TestRecreateBackendDefault` covers the new classifier path directly on real adapters; plan goldens that show the skip are updated from `backend_default` to `immutable_backend_value`. This pull request and its description were written by Isaac. Co-authored-by: Isaac <[email protected]>
1 parent bac9be9 commit cc3bcd3

29 files changed

Lines changed: 105 additions & 60 deletions

File tree

‎.nextchanges/bundles/recreate-backend-default.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
* Direct engine no longer recreates a resource when an immutable field the config omits was populated by the backend. ([#6790](https://github.com/databricks/cli/pull/6790))

‎acceptance/bundle/migrate/basic/out.plan_update.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -203,7 +203,7 @@
203203
},
204204
"storage": {
205205
"action": "skip",
206-
"reason": "backend_default",
206+
"reason": "immutable_backend_value",
207207
"remote": "dbfs:/pipelines/[UUID]"
208208
},
209209
"tags['myjob_name']": {
@@ -238,7 +238,7 @@
238238
"changes": {
239239
"storage_location": {
240240
"action": "skip",
241-
"reason": "backend_default",
241+
"reason": "immutable_backend_value",
242242
"remote": "s3://[METASTORE_NAME]/metastore/[UUID]/volumes/[UUID]"
243243
}
244244
}

‎acceptance/bundle/migrate/default-python/out.plan_after_deploy.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -343,7 +343,7 @@
343343
"changes": {
344344
"storage": {
345345
"action": "skip",
346-
"reason": "backend_default",
346+
"reason": "immutable_backend_value",
347347
"remote": "dbfs:/pipelines/[UUID]"
348348
}
349349
}

‎acceptance/bundle/migrate/default-python/out.plan_after_migration.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -343,7 +343,7 @@
343343
"changes": {
344344
"storage": {
345345
"action": "skip",
346-
"reason": "backend_default",
346+
"reason": "immutable_backend_value",
347347
"remote": "dbfs:/pipelines/[UUID]"
348348
}
349349
}

‎acceptance/bundle/migrate/default-python/output.txt‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -147,7 +147,7 @@ Building python_artifact...
147147
{
148148
"storage": {
149149
"action": "skip",
150-
"reason": "backend_default",
150+
"reason": "immutable_backend_value",
151151
"remote": "dbfs:/pipelines/[UUID]"
152152
}
153153
}
@@ -266,7 +266,7 @@ Building python_artifact...
266266
{
267267
"storage": {
268268
"action": "skip",
269-
"reason": "backend_default",
269+
"reason": "immutable_backend_value",
270270
"remote": "dbfs:/pipelines/[UUID]"
271271
}
272272
}

‎acceptance/bundle/resource_deps/remote_pipeline/out.plan_skip.direct.json‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626
"changes": {
2727
"storage": {
2828
"action": "skip",
29-
"reason": "backend_default",
29+
"reason": "immutable_backend_value",
3030
"remote": "dbfs:/pipelines/[UUID]"
3131
}
3232
}
@@ -59,7 +59,7 @@
5959
"changes": {
6060
"storage": {
6161
"action": "skip",
62-
"reason": "backend_default",
62+
"reason": "immutable_backend_value",
6363
"remote": "dbfs:/pipelines/[UUID]"
6464
}
6565
}
@@ -92,7 +92,7 @@
9292
"changes": {
9393
"storage": {
9494
"action": "skip",
95-
"reason": "backend_default",
95+
"reason": "immutable_backend_value",
9696
"remote": "dbfs:/pipelines/[UUID]"
9797
}
9898
}
@@ -125,7 +125,7 @@
125125
"changes": {
126126
"storage": {
127127
"action": "skip",
128-
"reason": "backend_default",
128+
"reason": "immutable_backend_value",
129129
"remote": "dbfs:/pipelines/[UUID]"
130130
}
131131
}

‎acceptance/bundle/resources/grants/volumes/out.plan2.direct.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@
3939
"changes": {
4040
"storage_location": {
4141
"action": "skip",
42-
"reason": "backend_default",
42+
"reason": "immutable_backend_value",
4343
"remote": "s3://[METASTORE_NAME]/metastore/[UUID]/volumes/[UUID]"
4444
}
4545
}

‎acceptance/bundle/resources/permissions/pipelines/update/out.plan_delete_all.direct.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626
"changes": {
2727
"storage": {
2828
"action": "skip",
29-
"reason": "backend_default",
29+
"reason": "immutable_backend_value",
3030
"remote": "dbfs:/pipelines/[FOO_ID]"
3131
}
3232
}

‎acceptance/bundle/resources/permissions/pipelines/update/out.plan_delete_one.direct.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626
"changes": {
2727
"storage": {
2828
"action": "skip",
29-
"reason": "backend_default",
29+
"reason": "immutable_backend_value",
3030
"remote": "dbfs:/pipelines/[FOO_ID]"
3131
}
3232
}

‎acceptance/bundle/resources/permissions/pipelines/update/out.plan_restore.direct.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626
"changes": {
2727
"storage": {
2828
"action": "skip",
29-
"reason": "backend_default",
29+
"reason": "immutable_backend_value",
3030
"remote": "dbfs:/pipelines/[FOO_ID]"
3131
}
3232
}

0 commit comments

Comments
 (0)