Skip to content

Commit 15e1747

Browse files
morningmanclaude
andcommitted
[fix](ranger) Build a Ranger plugin with its service's policies or not at all, instead of loading it in the background
RangerBasePlugin.init() survives a Ranger admin it cannot reach: it answers out of the policy cache when there is one and out of nothing when there is not, refusing every check until a later poll succeeds. That was the plugin's behaviour before this PR as well, and it is the wrong state for Doris to start in: with access_controller_type=ranger-doris an FE that has no policies is one nobody can use, root included, and a catalog bound to ranger-hive in that state refuses every statement - in both cases with nothing but a line in fe.log to say why. The background loading added earlier in this PR optimized exactly that state (the FE started while the admin was unreachable), and everything built around it - the preflight, the three-way cleanup, the failed-plugin replacement in both factories, the wait that gives up when the controller closes - existed only to make that safe. LoadedRangerPlugin, now the base of both plugins, runs RangerBasePlugin.init() on the constructing thread as before and refuses to leave the plugin without policies: a load that ended with none from the admin or the cache - an admin that cannot be reached and no cache, or a service the admin does not know - stops what it started and throws with the cause, which fails the FE start, the CREATE CATALOG dry run, or the statement binding a catalog to its source again. Once built, an outage of the admin is Ranger's to survive, as it always was. A missing user store is still logged and accepted: an admin that serves none cannot be told from one that has not answered yet, and policy items written against a group simply do not apply until it arrives. The factories, the audit stack, the controllers and the fe-core comments go back to what master has, corrected for the user store the load downloads now. The plugin's test runs Ranger's real init() against an admin client the test stands in for - policies served, policies cached, nothing at all, service unknown, unusable configuration - and reads the groups back through Ranger's own enricher. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_0191LreyBauJV7K7sZAbrB27
1 parent 89cb573 commit 15e1747

19 files changed

Lines changed: 543 additions & 1467 deletions

File tree

‎fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/BackgroundLoadedRangerPlugin.java‎

Lines changed: 0 additions & 408 deletions
This file was deleted.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
// Licensed to the Apache Software Foundation (ASF) under one
2+
// or more contributor license agreements. See the NOTICE file
3+
// distributed with this work for additional information
4+
// regarding copyright ownership. The ASF licenses this file
5+
// to you under the Apache License, Version 2.0 (the
6+
// "License"); you may not use this file except in compliance
7+
// with the License. You may obtain a copy of the License at
8+
//
9+
// http://www.apache.org/licenses/LICENSE-2.0
10+
//
11+
// Unless required by applicable law or agreed to in writing,
12+
// software distributed under the License is distributed on an
13+
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
// KIND, either express or implied. See the License for the
15+
// specific language governing permissions and limitations
16+
// under the License.
17+
18+
package org.apache.doris.catalog.authorizer.ranger;
19+
20+
import org.apache.logging.log4j.LogManager;
21+
import org.apache.logging.log4j.Logger;
22+
import org.apache.ranger.plugin.service.RangerBasePlugin;
23+
import org.apache.ranger.plugin.util.ServicePolicies;
24+
25+
import java.util.concurrent.TimeUnit;
26+
27+
/**
28+
* What the two Ranger plugins Doris embeds have in common: they are built with their service's policies or
29+
* not at all, and the requests built over them carry the groups Ranger keeps for a user.
30+
*
31+
* <p><b>Built with the policies, or not at all.</b> {@code RangerBasePlugin.init()} downloads the service's
32+
* roles and policies - and, with the enricher below on them, its user store - before it returns, and it
33+
* survives a Ranger admin it cannot reach: it answers out of the local policy cache
34+
* ({@code ranger.plugin.<type>.policy.cache.dir}) when there is one, and out of no policies at all when there
35+
* is not, refusing every check until a later poll succeeds. That is the right thing for a plugin embedded in
36+
* a service that has to keep running, and the wrong thing at the moment Doris binds one: an FE whose instance
37+
* scope {@code access_controller_type=ranger-doris} governs, and which has no policies, is an FE nobody can
38+
* use - no account bypasses the source - and a catalog bound to a {@code ranger-hive} source in that state
39+
* refuses every statement against it, in both cases with nothing but a line in fe.log to say why. So
40+
* {@link #init()} refuses that state instead: a load that ended with no policies, from the admin or the
41+
* cache, stops what it started and throws, with the cause. That fails the FE start, the {@code CREATE
42+
* CATALOG} (whose dry run builds the plugin), or the statement binding a catalog to its source again, which
43+
* is where an operator sees it. Once built, an outage of the admin is Ranger's to survive, as it always
44+
* was: the refresher keeps the last policies it downloaded and keeps polling.
45+
*
46+
* <p>The user store is not part of that. A load that found the policies but no user store is logged and
47+
* accepted - policy items written against a group do not apply until the store arrives, which the enricher
48+
* keeps asking for - because "no store has arrived" cannot be told apart from "this admin serves none", and
49+
* an admin from before the user store download existed, or one that fails that download while serving the
50+
* policies, ran every deployment before groups were attached at all; see {@link RangerUserStoreGroups}.
51+
*
52+
* <p><b>Requests carry Ranger's own groups.</b> Doris has none to offer, and a policy item written against a
53+
* group matches nothing without them; the store they are read from is asked for with the policies, see
54+
* {@link #setPolicies} and {@link RangerUserStoreGroups}.
55+
*/
56+
public abstract class LoadedRangerPlugin extends RangerBasePlugin {
57+
private static final Logger LOG = LogManager.getLogger(LoadedRangerPlugin.class);
58+
59+
protected LoadedRangerPlugin(String serviceType, String serviceName, String appId) {
60+
super(serviceType, serviceName, appId);
61+
}
62+
63+
/**
64+
* Loads the plugin - {@code RangerBasePlugin.init()}: roles, policies and user store, on this thread -
65+
* and refuses to leave it without policies; see the class comment.
66+
*
67+
* @throws IllegalStateException when the load ended with no policies from either the admin or the cache
68+
*/
69+
@Override
70+
public void init() {
71+
LOG.info(RangerUserStoreGroups.describe(getConfig()));
72+
long startedAtNanos = System.nanoTime();
73+
try {
74+
super.init();
75+
if (getPoliciesVersion() < 0) {
76+
throw new IllegalStateException(describeNoPolicies());
77+
}
78+
} catch (RuntimeException | Error e) {
79+
// Stops what the load had started before it failed - the policy refresher and its download
80+
// timer, and the engine if one was built - so that a plugin nobody will hold does not go on
81+
// polling the admin. A configuration the load could not use fails inside RangerBasePlugin.init()
82+
// before any of that is started, and stopping nothing is a no-op.
83+
try {
84+
cleanup();
85+
} catch (RuntimeException | Error stopFailure) {
86+
e.addSuppressed(stopFailure);
87+
}
88+
throw e;
89+
}
90+
LOG.info("Ranger service {} loaded in {} ms: policies version {}, roles version {}, user store"
91+
+ " version {}", getServiceName(),
92+
TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startedAtNanos), getPoliciesVersion(),
93+
getRolesVersion(), getUserStoreVersion());
94+
if (RangerUserStoreGroups.enabledFor(getConfig()) && getUserStoreVersion() < 0) {
95+
// Policies to answer out of, but no user store to read groups from: the admin could not be reached
96+
// for it and none was cached. Requests carry no groups until one arrives - the enricher keeps
97+
// asking - which is what this source sent before groups were attached at all, and which a policy
98+
// item written against a group, a deny included, does not match.
99+
LOG.warn("Ranger service {} loaded its policies but no user store: until one arrives, requests"
100+
+ " carry no groups and policy items written against a group do not apply", getServiceName());
101+
}
102+
}
103+
104+
/** Why the plugin was refused: what was asked for the policies, and where they could have been cached. */
105+
private String describeNoPolicies() {
106+
String prefix = getConfig().getPropertyPrefix();
107+
String adminUrl = getConfig().get(prefix + ".policy.rest.url");
108+
String cacheDir = getConfig().get(prefix + ".policy.cache.dir");
109+
return "Ranger service " + getServiceName() + " (type " + getServiceType() + ") has no policies to"
110+
+ " authorize against: Ranger Admin at " + adminUrl + " could not be reached or does not know"
111+
+ " the service, and "
112+
+ (cacheDir == null
113+
? "no policy cache directory is configured (" + prefix + ".policy.cache.dir)"
114+
: "no policy cache was found under " + cacheDir)
115+
+ ". Fix ranger-" + getServiceType() + "-security.xml in fe/conf, or bring Ranger Admin back,"
116+
+ " and try again";
117+
}
118+
119+
/**
120+
* Takes the policies Ranger downloaded, and asks for the user store with them, so that the requests the
121+
* source over this plugin builds can carry the groups Ranger keeps for a user; see
122+
* {@link RangerUserStoreGroups}. Done on the way in rather than left to Ranger, because Ranger only does
123+
* it on its own from 2.5 on, and behind a property.
124+
*/
125+
@Override
126+
public void setPolicies(ServicePolicies policies) {
127+
RangerUserStoreGroups.addUserStoreEnricher(getConfig(), policies);
128+
super.setPolicies(policies);
129+
}
130+
}

‎fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java‎

Lines changed: 4 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@
3535
import org.apache.ranger.plugin.policyengine.RangerAccessResourceImpl;
3636
import org.apache.ranger.plugin.policyengine.RangerAccessResult;
3737
import org.apache.ranger.plugin.policyengine.RangerAccessResultProcessor;
38+
import org.apache.ranger.plugin.service.RangerBasePlugin;
3839

3940
import java.util.Collection;
4041
import java.util.HashMap;
@@ -57,13 +58,6 @@
5758
* give - its policy engine not yet initialized, or being cleaned up - is a refusal too, not an empty policy
5859
* set. Getting the second one wrong is silent: the engine reads "no row filter, no column mask" as licence to
5960
* read the table whole and in the clear.
60-
*
61-
* <p>A plugin still on its first load is neither: it is about to have an answer, and every question here
62-
* waits for it ({@link BackgroundLoadedRangerPlugin#awaitLoaded(java.util.function.BooleanSupplier)})
63-
* before anything else - before the lifecycle fence above all, so that a load the Ranger admin is slow to
64-
* serve is waited for with no lock held, and closing this controller meanwhile is not queued behind it. Nor
65-
* is the wait queued behind the close: a controller closed while its check waits stops waiting and refuses,
66-
* as it would have after the load.
6761
*/
6862
public abstract class RangerAccessController implements AuthorizationPlugin {
6963
private static final Logger LOG = LogManager.getLogger(RangerAccessController.class);
@@ -187,22 +181,9 @@ protected interface PluginCheck {
187181
@Override
188182
public final void checkPrivilege(AuthorizedSubject subject, AuthorizedResource resource,
189183
AccessRequirement requirement, AccessContext context) throws AccessDeniedException {
190-
awaitPluginUnlessClosed();
191184
checkWhileOpen(resource, () -> checkPrivilegeInternal(subject, resource, requirement, context));
192185
}
193186

194-
/**
195-
* Waits for the Ranger plugin's first load to end, unless this controller is closed first.
196-
*
197-
* <p>Before the lifecycle fence and with no lock held, see the class comment. Closed, this controller
198-
* refuses whatever the load brings, so there is nothing to wait for: a query holding a controller its
199-
* catalog has since let go of is refused at once, not after the REST timeouts of a Ranger admin that is
200-
* not answering - the plugin's own {@code cleanup()} deliberately does not wait for that load either.
201-
*/
202-
private void awaitPluginUnlessClosed() {
203-
getPlugin().awaitLoaded(() -> closed);
204-
}
205-
206187
/** What this Ranger service decides about {@code resource}, asked only while this source is open. */
207188
protected abstract void checkPrivilegeInternal(AuthorizedSubject subject, AuthorizedResource resource,
208189
AccessRequirement requirement, AccessContext context) throws AccessDeniedException;
@@ -259,8 +240,8 @@ protected static String clientAddressOf(AuthorizedSubject subject, AccessContext
259240

260241
/**
261242
* The groups to put on a request about {@code subject}: the ones Ranger's own user store puts the user
262-
* in, see {@link RangerUserStoreGroups}. Empty until the plugin has downloaded the store, for a user the
263-
* store does not know, and when the deployment has switched this off.
243+
* in, see {@link RangerUserStoreGroups}. Empty when the plugin has no user store, for a user the store does
244+
* not know, and when the deployment has switched this off.
264245
*/
265246
protected Set<String> groupsOf(AuthorizedSubject subject) {
266247
return RangerUserStoreGroups.groupsOf(getPlugin(), subject.getUser());
@@ -353,7 +334,6 @@ private static String policyIdent(RangerAccessResult policy) {
353334
@Override
354335
public List<RowFilterSpec> getRowFilters(AuthorizedSubject subject, AuthorizedResource.Table table,
355336
AccessContext context) {
356-
awaitPluginUnlessClosed();
357337
return whileOpen(() -> evalRowFilterPolicies(subject, table, context));
358338
}
359339

@@ -403,7 +383,6 @@ private List<RowFilterSpec> evalRowFilterPolicies(AuthorizedSubject subject,
403383
@Override
404384
public Map<String, DataMaskSpec> getDataMasks(AuthorizedSubject subject, AuthorizedResource.Table table,
405385
Set<String> columns, AccessContext context) {
406-
awaitPluginUnlessClosed();
407386
return whileOpen(() -> {
408387
Map<String, DataMaskSpec> masks = new HashMap<>();
409388
for (String column : columns) {
@@ -541,7 +520,7 @@ protected String dataMaskExpressionOf(RangerAccessResult policy, String maskType
541520

542521
protected abstract RangerAccessResourceImpl createResource(String ctl, String db, String tbl, String col);
543522

544-
protected abstract BackgroundLoadedRangerPlugin getPlugin();
523+
protected abstract RangerBasePlugin getPlugin();
545524

546525
protected abstract RangerAccessResultProcessor getAccessResultProcessor();
547526
}

‎fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerUserStoreGroups.java‎

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
import org.apache.ranger.plugin.contextenricher.RangerUserStoreEnricher;
2525
import org.apache.ranger.plugin.policyengine.RangerPluginContext;
2626
import org.apache.ranger.plugin.service.RangerAuthContext;
27+
import org.apache.ranger.plugin.service.RangerBasePlugin;
2728
import org.apache.ranger.plugin.util.RangerUserStoreUtil;
2829
import org.apache.ranger.plugin.util.ServiceDefUtil;
2930
import org.apache.ranger.plugin.util.ServicePolicies;
@@ -80,7 +81,7 @@ public static boolean enabledFor(RangerPluginConfig config) {
8081
* Puts a user store enricher on the service definition Ranger just downloaded, unless one is there or
8182
* this is switched off, so that the plugin fetches and refreshes the user store {@link #groupsOf} reads.
8283
*
83-
* <p>Called by {@link BackgroundLoadedRangerPlugin#setPolicies} before handing the policies on, on every
84+
* <p>Called by {@link LoadedRangerPlugin#setPolicies} before handing the policies on, on every
8485
* call and not only the first: a download of policy deltas comes with its own copy of the service
8586
* definition, which is why {@code RangerBasePlugin} re-adds the enricher on deltas too. The retriever
8687
* class and the refresh interval are read under the option names Ranger itself reads them under, so that
@@ -116,18 +117,16 @@ public static String describe(RangerPluginConfig config) {
116117
/**
117118
* The groups the user store {@code plugin} has downloaded puts {@code user} in.
118119
*
119-
* <p>Read once the plugin's first load has ended, since the store arrives with it: asked earlier, this
120-
* would say "no groups" about a user the store is about to put in several. Empty when this is switched
121-
* off, when no store arrived - Ranger Admin could not be reached, and nothing was cached - and when the
120+
* <p>Empty when this is switched off, when no store has arrived - Ranger Admin could not be reached for
121+
* it and nothing was cached, which {@link LoadedRangerPlugin} says why it does not refuse - and when the
122122
* store does not know the user, which is the case for every account that exists in Doris only. Empty
123123
* and not null on purpose: a request with an empty group set matches items written against users and
124124
* roles exactly as it did before.
125125
*/
126-
public static Set<String> groupsOf(BackgroundLoadedRangerPlugin plugin, String user) {
126+
public static Set<String> groupsOf(RangerBasePlugin plugin, String user) {
127127
if (user == null || !enabledFor(plugin.getConfig())) {
128128
return Collections.emptySet();
129129
}
130-
plugin.awaitLoaded();
131130
// The auth context the policy engine publishes is where Ranger's own request processing reads the
132131
// store from; it is replaced together with the engine, and the store is carried over when it is.
133132
RangerPluginContext pluginContext = plugin.getPluginContext();

‎fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,6 @@
2525
import org.apache.doris.authorization.AuthorizedResource;
2626
import org.apache.doris.authorization.AuthorizedSubject;
2727
import org.apache.doris.authorization.spi.AuthorizationContext;
28-
import org.apache.doris.catalog.authorizer.ranger.BackgroundLoadedRangerPlugin;
2928
import org.apache.doris.catalog.authorizer.ranger.RangerAccessController;
3029

3130
import org.apache.logging.log4j.LogManager;
@@ -35,6 +34,7 @@
3534
import org.apache.ranger.plugin.policyengine.RangerAccessResult;
3635
import org.apache.ranger.plugin.policyengine.RangerAccessResultProcessor;
3736
import org.apache.ranger.plugin.service.RangerAuthContextListener;
37+
import org.apache.ranger.plugin.service.RangerBasePlugin;
3838

3939
import java.util.Collections;
4040
import java.util.Date;
@@ -71,7 +71,7 @@ public class RangerDorisAccessController extends RangerAccessController {
7171
* and the lifecycle fence in {@link RangerAccessController} - not a null field - is what stops it from
7272
* reaching a plugin that has been cleaned up.
7373
*/
74-
private final BackgroundLoadedRangerPlugin dorisPlugin;
74+
private final RangerBasePlugin dorisPlugin;
7575
// private static ScheduledThreadPoolExecutor logFlushTimer = ThreadPoolManager.newDaemonScheduledThreadPool(1,
7676
// "ranger-doris-audit-log-flusher-timer", true);
7777
// private RangerHiveAuditHandler auditHandler;
@@ -89,7 +89,7 @@ public RangerDorisAccessController(String serviceName, RangerAuthContextListener
8989
// logFlushTimer.scheduleAtFixedRate(new RangerHiveAuditLogFlusher(auditHandler), 10, 20L, TimeUnit.SECONDS);
9090
}
9191

92-
public RangerDorisAccessController(BackgroundLoadedRangerPlugin plugin, AuthorizationContext context) {
92+
public RangerDorisAccessController(RangerBasePlugin plugin, AuthorizationContext context) {
9393
this(plugin, Collections.emptyMap(), context);
9494
}
9595

@@ -98,7 +98,7 @@ public RangerDorisAccessController(BackgroundLoadedRangerPlugin plugin, Authoriz
9898
* out: the plugin is what polling the Ranger service costs, so it is shared, while everything a binding
9999
* configures - {@link #DEFER_TO_GLOBAL_SCOPE_AUTHORITY} - belongs to the controller and so to the binding.
100100
*/
101-
public RangerDorisAccessController(BackgroundLoadedRangerPlugin plugin, Map<String, String> properties,
101+
public RangerDorisAccessController(RangerBasePlugin plugin, Map<String, String> properties,
102102
AuthorizationContext context) {
103103
super(properties, context);
104104
dorisPlugin = Objects.requireNonNull(plugin, "ranger plugin is required");
@@ -455,7 +455,7 @@ protected RangerDorisResource createResource(String ctl, String db, String tbl,
455455
}
456456

457457
@Override
458-
protected BackgroundLoadedRangerPlugin getPlugin() {
458+
protected RangerBasePlugin getPlugin() {
459459
return dorisPlugin;
460460
}
461461

0 commit comments

Comments
 (0)