Skip to content

Commit 46c6293

Browse files
denikIsaac
andauthored
direct: Fully revoke grant privileges ALL_PRIVILEGES does not imply (#6743)
## Changes - `direct`: route all grant-revocation paths in `buildGrantChanges` through `grantRemovals`. When `ALL_PRIVILEGES` stays granted, keep it out of `Remove` and revoke only the no-longer-wanted excluded privileges (as #6733 already did); otherwise remove `ALL_PRIVILEGES` (clearing everything it implies) plus the excluded privileges (`MANAGE`, `READ_METADATA`, `EXTERNAL_USE_SCHEMA`, `EXTERNAL_USE_LOCATION`) by name. - `testserver`: model `Remove: [ALL_PRIVILEGES]` the way the real backend does — it clears implied privileges but leaves the four excluded ones — instead of wiping the principal. - New acceptance test `grants/schemas/remove_principal_manage`. ## Why Follow-up to #6733. Revoking a principal (or downgrading it off `ALL_PRIVILEGES`) sent `Remove: [ALL_PRIVILEGES]`, which the backend does not apply to the privileges `ALL_PRIVILEGES` excludes. Those lingered, so the principal never went away and every plan/deploy reported a change — a perpetual update. The fake server's full-wipe model hid this; fixing it surfaced the bug. It is pre-existing (a principal granted only `[MANAGE]` reproduces it), and #6733 made it more reachable by letting `MANAGE` land alongside `ALL_PRIVILEGES`. ## Tests `remove_principal_manage` (direct engine, local + cloud) plus `buildGrantChanges` unit cases. Verified on aws-cli: reproduced the perpetual update before the fix and confirmed convergence after. Terraform is excluded because it sends a precise `Remove: [MANAGE]` and already converges. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <[email protected]>
1 parent ac5988e commit 46c6293

11 files changed

Lines changed: 220 additions & 19 deletions

File tree

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
* Fixed the direct engine dropping `MANAGE`, `READ_METADATA`, `EXTERNAL_USE_SCHEMA` and `EXTERNAL_USE_LOCATION` grants when they were combined with `ALL_PRIVILEGES`. ([#6733](https://github.com/databricks/cli/pull/6733))
1+
* Fixed the direct engine mishandling UC grants that combine `ALL_PRIVILEGES` with a privilege it does not imply (`MANAGE`, `READ_METADATA`, `EXTERNAL_USE_SCHEMA`, `EXTERNAL_USE_LOCATION`): such privileges were dropped when granted and left behind when revoked, so the deployment never converged. ([#6733](https://github.com/databricks/cli/pull/6733), [#6743](https://github.com/databricks/cli/pull/6743))
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
bundle:
2+
name: schema-grants-remove-principal-manage-$UNIQUE_NAME
3+
4+
resources:
5+
schemas:
6+
grants_schema:
7+
name: schema_remove_principal_manage_$UNIQUE_NAME
8+
catalog_name: main
9+
grants:
10+
- principal: $CURRENT_USER_NAME
11+
privileges:
12+
- CREATE_TABLE
13+
# This principal holds MANAGE, which ALL_PRIVILEGES does not imply. Removing
14+
# it exercises the revocation path: buildGrantChanges must revoke MANAGE by
15+
# name (not just via Remove: [ALL_PRIVILEGES], which the backend does not
16+
# apply to MANAGE), otherwise the principal lingers and the plan never
17+
# converges.
18+
- { principal: deco-test-user@databricks.com, privileges: [MANAGE] } # TO_REMOVE
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"[principal='[email protected]']": {
3+
"action": "update",
4+
"old": {
5+
"principal": "[email protected]",
6+
"privileges": [
7+
"MANAGE"
8+
]
9+
},
10+
"remote": {
11+
"principal": "[email protected]",
12+
"privileges": [
13+
"MANAGE"
14+
]
15+
}
16+
}
17+
}
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
{
2+
"method": "PATCH",
3+
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_remove_principal_manage_[UNIQUE_NAME]",
4+
"body": {
5+
"changes": [
6+
{
7+
"add": [
8+
"CREATE_TABLE"
9+
],
10+
"principal": "[USERNAME]",
11+
"remove": [
12+
"ALL_PRIVILEGES"
13+
]
14+
},
15+
{
16+
"principal": "[email protected]",
17+
"remove": [
18+
"ALL_PRIVILEGES",
19+
"MANAGE"
20+
]
21+
}
22+
]
23+
}
24+
}
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"method": "PATCH",
3+
"path": "/api/2.1/unity-catalog/permissions/schema/main.schema_remove_principal_manage_[UNIQUE_NAME]",
4+
"body": {
5+
"changes": [
6+
{
7+
"principal": "[email protected]",
8+
"remove": [
9+
"MANAGE"
10+
]
11+
}
12+
]
13+
}
14+
}

‎acceptance/bundle/resources/grants/schemas/remove_principal_manage/out.test.toml‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
2+
>>> [CLI] bundle deploy
3+
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/schema-grants-remove-principal-manage-[UNIQUE_NAME]/default/files...
4+
Created schemas.grants_schema
5+
Created schemas.grants_schema.grants
6+
Files: 3 uploaded, 0 deleted
7+
Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
8+
9+
>>> [CLI] grants get schema main.schema_remove_principal_manage_[UNIQUE_NAME]
10+
json.privilege_assignments[].principal = "[USERNAME]";
11+
json.privilege_assignments[].principal = "[email protected]";
12+
json.privilege_assignments[].privileges[] = "CREATE_TABLE";
13+
json.privilege_assignments[].privileges[] = "MANAGE";
14+
15+
>>> [CLI] bundle deploy -qq
16+
17+
>>> [CLI] grants get schema main.schema_remove_principal_manage_[UNIQUE_NAME]
18+
json.privilege_assignments[].principal = "[USERNAME]";
19+
json.privilege_assignments[].privileges[] = "CREATE_TABLE";
20+
21+
>>> [CLI] bundle plan
22+
Plan: 0 to add, 0 to change, 0 to delete, 2 unchanged
23+
24+
>>> errcode [CLI] bundle destroy --auto-approve
25+
The following resources will be deleted:
26+
delete resources.schemas.grants_schema
27+
28+
This action will result in the deletion of the following UC schemas. Any underlying data may be lost:
29+
delete resources.schemas.grants_schema
30+
31+
All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/schema-grants-remove-principal-manage-[UNIQUE_NAME]/default
32+
33+
Destroy: 1 deleted
Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
envsubst < databricks.yml.tmpl > databricks.yml
2+
3+
cleanup() {
4+
trace errcode $CLI bundle destroy --auto-approve
5+
rm -f "$OUT_REQUESTS"
6+
}
7+
trap cleanup EXIT
8+
9+
trace $CLI bundle deploy
10+
trace $CLI grants get schema main.schema_remove_principal_manage_$UNIQUE_NAME | gron.py --noindex | sort_lines.py --repl | contains.py "$CURRENT_USER_NAME" '[email protected]'
11+
12+
# Flush the grant requests; we only want to record the revoke request below.
13+
print_requests.py --method PATCH //permissions > LOG.grant_requests
14+
15+
# Remove the MANAGE principal from config.
16+
grep -v 'TO_REMOVE' databricks.yml > tmp.yml && mv tmp.yml databricks.yml
17+
18+
# The plan revokes the removed principal. The per-resource change diff is a direct
19+
# engine concept (terraform's plan JSON does not expose it), so record it for direct.
20+
if [ "$DATABRICKS_BUNDLE_ENGINE" = "direct" ]; then
21+
$CLI bundle plan -o json | jq '.plan["resources.schemas.grants_schema.grants"].changes' > out.plan.changes.direct.json
22+
fi
23+
24+
# Deploy with -qq: the direct variant writes out.plan.changes.direct.json into the
25+
# bundle root above, so the "Files: N uploaded" count would differ between engines.
26+
trace $CLI bundle deploy -qq
27+
28+
# The revoke request (engine-specific): the direct engine removes MANAGE by name in
29+
# addition to ALL_PRIVILEGES, terraform removes MANAGE directly. Removing only
30+
# ALL_PRIVILEGES would leave MANAGE behind on the backend.
31+
print_requests.py --method PATCH //permissions > out.requests.$DATABRICKS_BUNDLE_ENGINE.txt
32+
33+
# The removed principal is gone, and the deploy converges (no perpetual update) on
34+
# both engines.
35+
trace $CLI grants get schema main.schema_remove_principal_manage_$UNIQUE_NAME | gron.py --noindex | sort_lines.py --repl
36+
trace $CLI bundle plan

‎bundle/direct/dresources/grants.go‎

Lines changed: 19 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -153,24 +153,12 @@ func buildGrantChanges(desiredAssignments []catalog.PrivilegeAssignment, remote
153153
desiredPrincipals := make(map[string]struct{}, len(desiredAssignments))
154154
for _, ga := range desiredAssignments {
155155
desiredPrincipals[ga.Principal] = struct{}{}
156-
change := catalog.PermissionsChange{
156+
changes = append(changes, catalog.PermissionsChange{
157157
Principal: ga.Principal,
158158
Add: ga.Privileges,
159-
Remove: nil,
159+
Remove: grantRemovals(ga.Privileges, remote[ga.Principal]),
160160
ForceSendFields: nil,
161-
}
162-
if slices.Contains(ga.Privileges, catalog.PrivilegeAllPrivileges) {
163-
// ALL_PRIVILEGES cannot also appear in Remove (the backend rejects the
164-
// duplicate), and removing it would not clear the privileges it does not
165-
// imply anyway, so revoke by name the excluded privileges the principal
166-
// still holds remotely but no longer wants.
167-
change.Remove = revokedExcludedPrivileges(ga.Privileges, remote[ga.Principal])
168-
} else {
169-
// Removing ALL_PRIVILEGES clears every privilege the principal holds, so
170-
// re-adding the desired set above reconciles it to exactly the config.
171-
change.Remove = []catalog.Privilege{catalog.PrivilegeAllPrivileges}
172-
}
173-
changes = append(changes, change)
161+
})
174162
}
175163

176164
// Principals present remotely but no longer desired: revoke everything.
@@ -185,16 +173,30 @@ func buildGrantChanges(desiredAssignments []catalog.PrivilegeAssignment, remote
185173
changes = append(changes, catalog.PermissionsChange{
186174
Principal: principal,
187175
Add: nil,
188-
Remove: []catalog.Privilege{catalog.PrivilegeAllPrivileges},
176+
Remove: grantRemovals(nil, remote[principal]),
189177
ForceSendFields: nil,
190178
})
191179
}
192180
return changes
193181
}
194182

183+
// grantRemovals returns the privileges to revoke for a principal so the backend
184+
// converges to desired. When ALL_PRIVILEGES stays desired it must not appear in
185+
// Remove (the backend rejects the same privilege in Add and Remove), so only the
186+
// no-longer-wanted excluded privileges are revoked. Otherwise ALL_PRIVILEGES is
187+
// removed to clear everything it implies, plus the excluded privileges by name
188+
// because ALL_PRIVILEGES does not imply them and its removal would leave them behind.
189+
func grantRemovals(desired, remotePrivileges []catalog.Privilege) []catalog.Privilege {
190+
remove := revokedExcludedPrivileges(desired, remotePrivileges)
191+
if slices.Contains(desired, catalog.PrivilegeAllPrivileges) {
192+
return remove
193+
}
194+
return append([]catalog.Privilege{catalog.PrivilegeAllPrivileges}, remove...)
195+
}
196+
195197
// revokedExcludedPrivileges returns the privileges not implied by ALL_PRIVILEGES
196198
// (see allPrivilegesExcludes) that the principal holds remotely but no longer
197-
// desires, so they can be revoked by name alongside a (re)granted ALL_PRIVILEGES.
199+
// desires, so they can be revoked by name.
198200
func revokedExcludedPrivileges(desired, remotePrivileges []catalog.Privilege) []catalog.Privilege {
199201
var remove []catalog.Privilege
200202
for _, p := range allPrivilegesExcludes {

‎bundle/direct/dresources/grants_test.go‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,42 @@ func TestBuildGrantChanges(t *testing.T) {
8888
},
8989
},
9090
},
91+
{
92+
// Removing a principal that holds MANAGE: ALL_PRIVILEGES removal clears
93+
// everything it implies, and MANAGE (which it does not imply) is revoked
94+
// by name so the principal is fully removed instead of lingering.
95+
name: "revokes excluded privilege of a removed principal",
96+
remote: map[string][]catalog.Privilege{
97+
"alice": {catalog.PrivilegeAllPrivileges, catalog.PrivilegeManage},
98+
},
99+
expected: []catalog.PermissionsChange{
100+
{
101+
Principal: "alice",
102+
Remove: []catalog.Privilege{catalog.PrivilegeAllPrivileges, catalog.PrivilegeManage},
103+
},
104+
},
105+
},
106+
{
107+
// Downgrading a principal off ALL_PRIVILEGES: MANAGE must be revoked by
108+
// name too, since removing ALL_PRIVILEGES does not clear it.
109+
name: "revokes excluded privilege when downgrading off ALL_PRIVILEGES",
110+
desired: []catalog.PrivilegeAssignment{
111+
{
112+
Principal: "alice",
113+
Privileges: []catalog.Privilege{catalog.PrivilegeSelect},
114+
},
115+
},
116+
remote: map[string][]catalog.Privilege{
117+
"alice": {catalog.PrivilegeAllPrivileges, catalog.PrivilegeManage},
118+
},
119+
expected: []catalog.PermissionsChange{
120+
{
121+
Principal: "alice",
122+
Add: []catalog.Privilege{catalog.PrivilegeSelect},
123+
Remove: []catalog.Privilege{catalog.PrivilegeAllPrivileges, catalog.PrivilegeManage},
124+
},
125+
},
126+
},
91127
}
92128

93129
for _, tt := range tests {

0 commit comments

Comments
 (0)