Skip to content

Instantly share code, notes, and snippets.

@dmartinol
Last active May 16, 2024 15:47
Show Gist options
  • Select an option

  • Save dmartinol/dc011a7ad538a02e049ee361d06bb8d3 to your computer and use it in GitHub Desktop.

Select an option

Save dmartinol/dc011a7ad538a02e049ee361d06bb8d3 to your computer and use it in GitHub Desktop.
Feast Permission Model

See feast-dev/feast#4198

Feast permissions model

Goals:

  • Protect individual Feast instances like FeatureView, OnDemandFeatureView, FeatureService (and others)
  • Decouple permission configuration from the instance definition
  • Allow to plug-in custom authentication modules (Feast server use case)
  • Allow to protect the unauthorized use of the protected resources from the python SDK

Defining permissions

The proposal assumes to use the RBAC security model to define Feast permissions.

Permissions are defined in the feature definition as instances of FeastPermission type. Each instance identifies:

  • The permission name
  • The protected resource, one of Entity, DataSource, FeatureView, StreamFeatureView, OnDemandFeatureView, FeatureService, SavedDataset, ValidationReference
    • Optionally a subset of the target resources can be selected by applying a regex filter on the resource name or tags
  • The permitted actions, one of read, edit, query_offline, query_online, write_online and write_offline
  • The policies that must be satisfied to access or perform these actions on the protected resource
    • The default policy validates the configured permission roles against the user/client roles
  • The decision strategy to evaluate multiple policies, one of unanimous, affirmative, consensus

Note we assume to model only the permissions to allow executions, not to deny them, at least as a first step.

The following example specifies that only users with risk_reader role can query from resources tagged with team: risk and named like very_important_*:

risk_role = FeastPermission(
    name="query_risk_stores",
    resources=FeastResource(
        type=AuthzedResource.ALL,
        conditions=[
            {
                "name": "very_important_*",  # Resource name must match the pattern AND
                "team": "risk",  # Resource must include this tag
            }
        ],
    ),
    actions=[AuthzedAction.QUERY],
    policies=[RoleBasedPolicy(["risk_reader"])],
    decision_strategy=DecisionStrategy.UNANIMOUS,  # Could be omitted, it's the default value BTW
)

Permission classes to be introduced:

# In alternative, consider using the FeastObjectType enum (ATM it misses some types and ALL)
class AuthzedResource(enum.Enum):
    ALL = "all"
    DATA_SOURCE = "data source"
    ENTITY = "entity"
    FEATURE_VIEW = "feature view"
    ON_DEMAND_FEATURE_VIEW = "on demand feature view"
    STREAM_FEATURE_VIEW = "stream feature view"
    FEATURE_SERVICE = "feature service"
    SAVED_DATASET = "saved dataset"
    VALIDATION_REFERENCE = "validation reference"


class AuthzedAction(enum.Enum):
    ALL = "all"
    READ = "read"
    EDIT = "edit"  # CREATE + UPDATE + DELETE
    QUERY = "query"  # ONLINE + OFFLINE
    QUERY_ONLINE = "query online"
    QUERY_OFFLINE = "query offline"
    WRITE = ("write",)  # ONLINE + OFFLINE
    WRITE_ONLINE = "write online"
    WRITE_OFFLINE = "write offline"


class DecisionStrategy(enum.Enum):
    # all policies must evaluate to a positive decision
    UNANIMOUS = 1
    # at least one policy must evaluate to a positive decision
    AFFIRMATIVE = 2
    # the no. of positive decisions must be greater than the no. of negative decisions
    CONSENSUS = 3


class FeastResource:
    def __init__(
        self,
        type: AuthzedResource,
        # Optional filter to match only resources with a given name or tag
        # ALL the conditions must match to select any resource of the given type
        # Defaults to None, equivalent to not apply any additional filter
        conditions: Optional[Dict[str, str]] = {},
        # Note: here we could probably be more explicit with parameters:
        # "name_patterns: Optional[List[str]] = []"
        # "required_tags: Optional[Dict[str, str]] = {}"
    ):
        pass


# We can instroduce the FeastPolicy interface whenever other classes are added to support custom policies
class RoleBasedPolicy:
    def __init__(
        self,
        roles: List[StrictStr],
    ):
        pass

    def get_roles(self):
        pass


class FeastPermission:
    def __init__(
        self,
        name: StrictStr,
        resources: Optional[List[FeastResource]] = [FeastResource(AuthzedResource.ALL)],
        actions: Optional[List[AuthzedAction]] = [AuthzedAction.ALL],
        policies: Optional[List[RoleBasedPolicy]] = [],  # Equivalent to allow-all
        decision_strategy: Optional[DecisionStrategy] = DecisionStrategy.UNANIMOUS,
    ):
        pass

Class methods-static builders

We can add helpers to simplify the creation of permissions with pre-built defaults:

    @classmethod
    def with_permission_to_query_online(
        cls,
        name: StrictStr,
        roles: List[StrictStr],
        conditions: Dict[str, str],
        decision_strategy: Optional[DecisionStrategy] = DecisionStrategy.AFFIRMATIVE,
    ):
        return cls(
            name=name,
            resources=[
                FeastResource(
                    type=AuthzedResource.ALL,
                    conditions=conditions,
                )
            ],
            actions=[AuthzedAction.QUERY_ONLINE],
            policies=[RoleBasedPolicy(roles)],
            decision_strategy=decision_strategy,
        )

Example of feature store definition using builders

FeatureView(tags={"team": "risk"})

FeatureService(tags={"team": "general_purpose"})

# Only users with 'restricted-team' role can read online store from resources with tag 'team: risk'
restricted_access_to_online_store = FeastPermission.with_permission_to_query_online(
    name="restricted_access_to_online_store",
    roles=["restricted-team"],
    conditions={"team": "risk"},
)

# Users with 'general-team' role can read online store from any resources
general_access_to_online_stores = FeastPermission.with_permission_to_query_online(
    name="general_access_to_online_stores",
    roles=["general-team"],
)

RBAC Security Architecture

High level design:

  • Permissions are defined in the feature configuration and stored in the registry
    • Registry class is extended with list_feast_permissions to return the configured permissions
  • RoleManager class is used to manage the user roles
  • PolicyEnforcer class is used to validate the execution of a given action on a protected resource against the configured policies
  • A SecurityContext class defines the current security context, holding information about the actual RoleManager, PolicyEnforcer and the current user
    • In the server use case, the server must properly configure the SecurityContext instance at startup time to set the current user and roles at each before handling any requests from the client
    • A K8sSecurityContext implementation can be provided to handle the case where the server runs as a K8s service:
      • The auth token of the requests contains the client ServiceAccount (e.g. the current user)
      • K8s API can be used to calculate the associated roles (caching can reduce the network overhead)
    • A K8sSecurityContext is instead created (__init__.py) to handle the case of using the Feast SDK in K8s runtimes, to pre-populate the RoleManager roles with the roles assigned to the current ServieAccount (which also acts as the current user)
    • Whatever the implementation, if no permissions have been defined the behavior is to grant unrestricted access to all users for all resources.

RoleManager class

class RoleManager:
    """Roles management"""

    def add_roles_for_user(self, user: str, roles: List[str]):
        """
        Adds the given roles to the user.
        """
        return False

    def has_roles_for_user(self, user: str, roles: List[str]) -> bool:
        """Whether the given user has all of the requested roles"""
        return False

PolicyEnforcer class

class PolicyEnforcer:
    """Enforce permission policies"""

    def enforce_policy(
        self,
        user: StrictStr,
        resource: Union[
            Entity,
            DataSource,
            FeatureView,
            StreamFeatureView,
            OnDemandFeatureView,
            FeatureService,
            SavedDataset,
            ValidationReference,
        ],
        actions: List[AuthzedAction],
        permissions: List[FeastPermission],
    ) -> Tuple[bool, str]:
        """Decides whether the given user can execute all the actions on the provided resource, with the
        configured permissions.
        Return boolean result with reason, like:
        'No permissions to read online from FeatureView:foo. Requires [role1, role2]'
        """
        return (True, "")

Implementation hintsfor enforce_policy:

  • Filter the permissions to select only those that apply to the given resource by type and conditions
  • Filter further to select only the permissions matching the given actions, or that include them (e.g. QUERY action includes both QUERY_ONLINE and QUERY-OFFLINE)
  • Extract from the RoleManager the roles of the current user as user_roles
  • Extract from each the filtered permissions the roles of the RoleBasedPolicy as required_roles
  • Finally, the execution is allowed only if the user_roles include all the required_roles (for each permission)

SecurityContext class

class SecurityContext:
    """The security context"""

    def __init__(self):
        self._role_manager = RoleManager()
        self._policy_enforcer = PolicyEnforcer(self._role_manager)
        self._current_user = ""
        self._permissions: List[FeastPermission] = []

    @property
    def role_manager(self) -> RoleManager:
        """The role manager"""
        return self._role_manager

    @property
    def policy_enforcer(self):
        """The policy enforcer"""
        return self._policy_enforcer

    @property
    def current_user(self) -> str:
        """The current user"""
        return self._current_user

    @property
    def permissions(self) -> List[FeastPermission]:
        """The configured permissions"""
        return self._permissions

    # Example of method to implement the enforcement on multiple resources and raise a PermissionError 
    def assert_permissions(
        self,
        resource: List[
            Union[
                Entity,
                DataSource,
                FeatureView,
                StreamFeatureView,
                OnDemandFeatureView,
                FeatureService,
                SavedDataset,
                ValidationReference,
            ]
        ],
        actions: List[AuthzedAction],
    ):
        """Raise a PermissionError if the current user cannot execute all the given actions
        on all the provided resources"""
        for r in resource:
            result, explain = self._policy_enforcer.enforce_policy(
                self._current_user, r, actions, self._permissions
            )
            if not result:
                raise PermissionError(explain)

Adding permission validation in Feast code

TODO: Define the methods and functions that require permission policy enforcement. TODO: Select b/w programmatic or decoration options

Some examples on how to apply the permission validation using either programmatic approach or using decorators

Retrieve the FeatureView instance (programmatic option)

Ensure the current user can access a given FeatureView:

    """ feature_store.py """
    
    def _get_feature_view(
        self,
        name: str,
        hide_dummy_entity: bool = True,
        allow_registry_cache: bool = False,
    ) -> FeatureView:
        feature_view: FeatureView = FeatureView()

        # Using enforce_policy
        enforcer = self.security_context.policy_enforcer
        result, explain = enforcer.enforce_policy(feature_view, AuthzedAction.READ)
        if not result:
            raise PermissionError(explain)

        return feature_view

Fetch historical features (programmatic option)

Ensure the current user can access offline data from the related FeatureView's:

    """ feature_store.py """
    
    def get_historical_features(
        self,
        entity_df,
        features: Union[List[str], FeatureService],
        full_feature_names: bool = False,
    ):
        feature_views = list()

        # Using assert_permissions
        self.security_context.assert_permissions(
            feature_views, [AuthzedAction.QUERY_OFFLINE]
        )

        pass

Access the join keys (decorator option)

    """ feature_view.py """

    @ensure_policy(AuthzedAction.READ)
    @property
    def join_keys(self) -> List[str]:
        """Returns a list of all the join keys."""
        return [entity.name for entity in self.entity_columns]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment