RBAC

Note

RBAC feature applies only to Determined Enterprise Edition.

Getting Started

Warning

If you are looking to enable RBAC on an existing Determined installation, please see the migration guide below.

By default, a Determined installation comes with Role Based Access Control disabled. To enable RBAC, set the following option in the master config:

security:
  authz:
    type: rbac
Brand new Determined installations include two user accounts:
  • admin user has full cluster access by default through pre-canned ClusterAdmin role.

  • determined user has no permissions.

Both accounts have empty passwords. You are encouraged to set strong passwords or deactivate these accounts for security reasons.

Example setup - CLI

In this section, we will configure a Determined instance to support a cluster administrator account, and a few engineers with varying level of access.

First, create a new user alice, set a password, make it an admin, and deactivate the default accounts:

det -u admin user create alice
det user change-password alice
det rbac assign-role -u alice ClusterAdmin
det user login alice
det user deactivate admin
det user deactivate determined

We want to enable two teams, one working on a traffic light detection model, and another one working on a stop signs detection model. We’d like these two teams to have their separate, compartmentalized workspaces. Start with creating non-priviledged users and workspaces:

det user create mle-traffic-00
det user create mle-traffic-01
det user create mle-traffic-02
det user create mle-stop-00
det workspace create "Traffic Lights"
det workspace create "Stop Signs"

We have several users on the traffic lights team, so we will create a user group to simplify the permission management:

det user-group create "Traffic Lights Team"
det user-group add-user "Traffic Lights Team" mle-traffic-00,mle-traffic-01,mle-traffic-02

Give Editor role on traffic lights workspace to entire traffic lights team, and WorkspaceAdmin role to the mle-traffic-00 and mle-stop-00 users on their respective workspace.

det rbac assign-role -g "Traffic Lights Team" -w "Traffic Lights" Editor
det rbac assign-role -u mle-traffic-00 -w "Traffic Lights" WorkspaceAdmin
det rbac assign-role -u mle-stop-00 -w "Stop Signs" WorkspaceAdmin

Learn more about pre-canned roles such as Editor, WorkspaceAdmin in the section below.

Determined requires the projects to exist within the workspaces to run and organize the experiments. Project creation requires PERMISSION_TYPE_CREATE_PROJECT permission which is included in Editor, WorkspaceAdmin, and ClusterAdmin roles. Since mle-traffic-01 user is a member of Traffic Lights Team which has the Editor role on the Traffic Lights workspace, they can create the project themselves, no cluster admin access or participation is necessary:

det -u mle-traffic-01 project create "Traffic Lights" "Green"

Tip

We use det -u USER_NAME to dynamically change the username for the current command here.

Similarly, you can run det user login USER_NAME to switch the current CLI user permanently.

As a non-priviledged user, we can now submit an experiment into this project:

# Per quickstart, `cd` into the example directory before the next command.
det -u mle-traffic-01 experiment create const.yaml . --config workspace="Traffic Lights" --config project="Green" --config name="green light"

Repeat the process to submit a second test experiment as a Stop Signs engineer:

det -u mle-stop-00 project create "Stop Signs" "Euro"
det -u mle-stop-00 experiment create const.yaml . --config workspace="Stop Signs" --config project="Euro" --config name="euro stop"

Since the admin and non-admin users on various teams have different access, listing all experiments will produce different results. Admin will see both experiments. Traffic lights and stop signs engineers will only have access to the experiments in their respective workspaces. Compare:

det -u alice experiment list --all
det -u mle-stop-00 experiment list --all
det -u mle-traffic-00 experiment list --all

Manage RBAC

Inspecting the Setup

CLI

To get the current user’s permission list:

det rbac my-permissions

To list all permissions on the role as well as all users and groups who bear it, whether globally or at a workspace level:

det rbac describe-role ROLE

To list all existing roles and their permissions:

det rbac list-roles

To list existing users, group and their membership:

det user list
det user-group list
det user-group describe GROUP_NAME

To list the role assignments for a user or a group:

det rbac list-groups-roles GROUP_NAME
det rbac list-users-roles USER_NAME

WebUI

To see user and group management UI,

  1. Click on your username in the upper left corner.

  2. Click “Settings”

  3. See “User Management” and “Group Management” tabs at the top.

To create new users, use “New User” button at the “User Management” screen.

To see or modify user roles assigned at the global scope,

  1. Click triple-dot icon on the right of the user entry.

  2. Select “Edit” option in the dropdown.

To see or modify group roles assigned at the global scope,

  1. Click triple-dot icon on the right of the group entry.

  2. Select “Add Roles” option in the dropdown.

To see group membership, click on the plus icon on the left of the group entry.

To remove member users, open group membership list and “Remove” button next to the user entry.

To add member users,

  1. Click triple-dot icon on the right of the group entry.

  2. Select “Edit/Add Users” option in the dropdown.

Managing User Groups

CLI

To create a group GROUP_NAME, add and remove users USER_NAME1, USER_NAME2, and USER_NAME3:

det user-group create GROUP_NAME
det user-group add-user GROUP_NAME USER_NAME1,USER_NAME2,USER_NAME3
det user-group remove-user GROUP_NAME USER_NAME1,USER_NAME2,USER_NAME3

To rename a group:

det user-group change-name GROUP_NAME

To delete a group:

det user-group delete GROUP_NAME

To list existing groups, or a particular group membership:

det user-group list
det user-group describe GROUP_NAME

WebUI

To see user group management UI,

  1. Click on your username in the upper left corner.

  2. Click “Settings”

  3. Click “Group Management” tabs up at the top.

To create new groups, use “New Group” button at the “Group Management” screen.

To delete a group,

  1. Click triple-dot icon on the right of the group entry.

  2. Select “Delete” option in the dropdown.

Managing Role Assignments

CLI

To assign or unassign a role for a user or a group globally:

det rbac assign-role -u USER_NAME ROLE_NAME
det rbac unassign-role -u USER_NAME ROLE_NAME
det rbac assign-role -g GROUP_NAME ROLE_NAME
det rbac unassign-role -g GROUP_NAME ROLE_NAME

To assign or unassign a role for a user or a group on a particular workspace, use -w WORKSPACE_NAME switch:

det rbac assign-role -u USER_NAME ROLE_NAME -w WORKSPACE_NAME
det rbac unassign-role -u USER_NAME ROLE_NAME -w WORKSPACE_NAME
det rbac assign-role -g GROUP_NAME ROLE_NAME -w WORKSPACE_NAME
det rbac unassign-role -g GROUP_NAME ROLE_NAME -w WORKSPACE_NAME

WebUI

To assign or unassign a role for a user or a group globally, first go to user or group management UI:

  1. Click on your username in the upper left corner.

  2. Click “Settings”

  3. See “User Management” and “Group Management” tabs up at the top.

Then, for users:

  1. Click triple-dot icon on the right of the user entry.

  2. Select “Edit” option in the dropdown.

For groups:

  1. Click triple-dot icon on the right of the group entry.

  2. Select “Add Roles” option in the dropdown.

To assign or unassign a role for a user or a group on a particular workspace,

  1. Go to the workspaces page, select the target workspace.

  2. Click “Members” tab at the top.

To create new role assignments, use “Add Members” button at the top To remove existing role assignments, click triple-dot menu and select “Remove” option.

About RBAC Concepts

User Groups

User Groups in Determined are organizational units containing one or more Determined users. User Groups currently have no inherent functionality and are not directly useful unless paired with the RBAC feature. Users with cluster admin permissions may create groups and add as many users to them as needed.

RBAC Permissions and Scopes

RBAC allows granting users or user groups a permission to do certain actions, such as various API calls, on certain resources, such as experiments. This is achieved using role assignments, which are comprised of security principal, role, and scope. Roles, in turn, are comprised of permissions.

Security Principal

Security Principal is an entity that is performing an action on a resource. Determined supports invidividual users or user groups as a security principal.

Scope

Scope in Determined refers to where a user may exercise their permitted actions and currently has two possible values: global and workspace^specific. A global^level permission is valid anywhere in Determined, allowing the user to perform the action on any workspace. A workspace^level permission restricts actions so that they are only permissible on the specified workspaces. When using workspace^level permissions, the admin must specify which workspace(s) the permission is valid for.

Role

Role is a collection of permissions. It allows combining commonly used permissions, for example when several permissions are used by the same persona, like a ML Engineer. Determined currently supports several built-in roles.

Permission

A description of a type of access to a resource or set of resources. Permissions typically map to an action on an entity type, for example:

  • PERMISSION_TYPE_VIEW_EXPERIMENT_METADATA: view high-level experiment properties.

  • PERMISSION_TYPE_VIEW_EXPERIMENT_ARTIFACTS: view experiment code and checkpoints.

  • PERMISSION_TYPE_ADMINISTRATE_USER: manage user accounts. This is only available on the global scope.

  • PERMISSION_TYPE_ASSIGN_ROLES: assign roles.

Usage Reference

CLI

Determined CLI has a help built-in. Please see help for the top level commands, as well as their subcommands:

det user -h
det user-group -h
det rbac -h
det rbac assign-role -h

Pre-canned roles

Determined ships with several pre-canned roles. We are looking to add an ability to edit or create custom roles in a future release.

To list all existing cluster roles and the concrete permissions they include:

det rbac list-roles

Viewer

Viewer role allows a user to see workspaces, projects, experiments, as well as experiment metadata and artifacts within its scope.

Editor

Editor role supersedes the Viewer role, and includes permissions to create, edit, or delete projects and experiments within its scope.

WorkspaceAdmin

WorkspaceAdmin role supersedes the Editor role, and includes permissions to edit or delete workspaces, and modify role assignments within its scope.

Users who posess this role on a particular workspace can assign roles to other users on this workspace, that is, add other members (viewers, editors, or workspace admins) to the workspace.

WorkspaceCreator

WorkspaceCreator role grants the single permission to create new workspaces. It can only be assigned globally.

Note

By default, when a user creates a workspace, they automatically get assigned the WorkspaceAdmin role.

This behavior can be configured using master config:

security:
       authz:
              workspace_creator_assign_role:
                 enabled: true
                 role_id: ROLE_ID

where ROLE_ID is the integer role identifier, as listed in det rbac list-roles. To disable the assignment of any roles to the newly created workspace, set enabled: false.

ClusterAdmin

ClusterAdmin is the highest role intended for cluster administrators or superusers. It includes all permissions, and can only be assigned globally.

Migrating existing installation to RBAC

  1. Upgrade Determined to the latest RBAC-enabled version.

  2. Enable RBAC UI in the master config:

    security:
       authz:
          rbac_ui_enabled: true
    
  3. Restart Determined for the config change to take effect. This config option will enable RBAC APIs and UI, but the RBAC rules will not be enforced, allowing administrators to set it up first.

  4. For all cluster administrators or superusers, grant the ClusterAdmin role. This will ensure the admins are not “locked out” once strict RBAC enforcement is enabled.

    det rbac assign-role -u ADMIN_USER_NAME ClusterAdmin
    
  5. Enable RBAC enforcement in the master config:

    security:
       authz:
          type: rbac
    
  6. Restart master for the change to take effect.

  7. Proceed to configure RBAC as desired.

Note

Workspace creators for workspaces created after upgrading to 0.19.6+ will have WorkspaceAdmin role assigned for their workspaces.

Users will have no default access otherwise.

Caveats

RBAC Support for Notebooks, Tensorboards, Shells, and Commands

Currently, only experiments are organized within workspaces and projects. Other task types, notebooks, tensorboards, sheels, and commands (NTSC) are global entities. In the future, we plan to migrate NTSCs to workspaces to make them covered under RBAC model.

When RBAC is enabled, as an interim measure, NTSC can only be accessed by the user who created the task, i.e. its owner, or by the users who have ADMINISTRATE_USER permission (i.e. ClusterAdmin).

When RBAC is disabled, by default, every logged in user can access all NTSC on the cluster. We offer an option which restricts the access to NTSC only to the admins and the user who created the task, i.e. its owner. It can be enabled in the master config:

security:
  authz:
    _strict_ntsc_enabled: true