Command-line Interface¶

The Determined command-line interface (CLI) is installed under the name det. Most uses of it require arguments specifying a type of object to act on and the action to perform, followed by any additional arguments or options. For example,

det experiment config 17

displays the configuration for experiment 17. Some commands have an additional level of nesting:

det experiment label add 17 foobar

adds the label “foobar” to experiment 17.

The names of some objects, actions, and options may be abbreviated; alternative names are shown below separated by commas. For example, these two commands are equivalent:

det -u admin experiment list

det --user admin experiment list

as are these two:

det e config 17

det experiment config 17

Additionally, when list is a valid action, it is the default when no action is specified, so det e and det s are equivalent to det experiment list and det slot list, for example.

Additionally, when list is a valid action, it is the default when no action is specified, so the following pairs of commands are also equivalent:

det e

det experiment list

det s

det slot list

Providing -h or --help as an argument anywhere will cause the CLI to exit after printing help text for the object or action specified up to that point.

Full Listing¶

Determined command-line client

usage: det [-h] [-u username] [-m address] [-v] command ...

Positional Arguments¶

command: Possible choices: help, agent, a, command, cmd, checkpoint, c, experiment, e, master, notebook, preview-search, shell, slot, s, task, template, tpl, tensorboard, trial, t, user, u, version

Named Arguments¶

-u, --user

run as the given user

-m, --master

master address

Default: “localhost:8080”

-v, --version

print CLI version and exit

Sub-commands:¶

help¶

show help for this command

det help [-h]

agent (a)¶

manage agents

det agent [-h] subcommand ...

Positional Arguments¶

subcommand: Possible choices: help, disable, enable, list

Sub-commands:¶

help¶

show help for this command

det agent help [-h]

disable¶

disable agent

det agent disable [-h] [--all] [agent_id]

Positional Arguments¶

agent_id: agent ID

Named Arguments¶

--all

disable all agents

Default: False

enable¶

enable agent

det agent enable [-h] [--all] [agent_id]

Positional Arguments¶

agent_id: agent ID

Named Arguments¶

--all

enable all agents

Default: False

list¶

list agents

det agent list [-h]

command (cmd)¶

manage commands

det command [-h] subcommand ...

Positional Arguments¶

subcommand: Possible choices: help, config, kill, list, ls, logs, run

Sub-commands:¶

help¶

show help for this command

det command help [-h]

config¶

display command config

det command config [-h] id

Positional Arguments¶

id: command ID

kill¶

forcibly terminate a command

det command kill [-h] [-f] command_id [command_id ...]

Positional Arguments¶

command_id: command ID

Named Arguments¶

-f, --force

ignore errors

Default: False

list (ls)¶

list commands

det command list [-h] [-q] [--all]

Named Arguments¶

-q, --quiet

only display the IDs

Default: False

--all, -a

show all commands (including other users’)

Default: False

logs¶

fetch command logs

det command logs [-h] [-f] [--tail TAIL] command_id

Positional Arguments¶

command_id: command ID

Named Arguments¶

-f, --follow

follow the logs of a command, similar to tail -f

Default: False

--tail

number of lines to show, counting from the end of the log

Default: 10

run¶

create command

det command run [-h] [--config-file CONFIG_FILE] [-v VOLUME] [-c CONTEXT]
                [--config CONFIG] [--template TEMPLATE] [-d]
                ...

Positional Arguments¶

entrypoint: entrypoint command and arguments to execute

Named Arguments¶

--config-file

command config file (.yaml)

-v, --volume

A mount specification in the form of <host path>:<container path>. The given path on the host machine will be mounted under the given path in the command container.

Default: []

-c, --context

The filepath to a directory that contains the set of files used to execute the command. All files under this directory will be packaged, maintaining the existing directory structure. The total byte contents of the directory must not exceed 96 MB. By default, the context directory will be empty.

--config

Additional configuration arguments for setting up a command. Arguments should be specified as key=value. Nested configuration keys can be specified by dot notation, e.g., resources.slots=4. List values can be specified by comma-separated values.

Default: []

--template

name of template to apply to the command configuration

-d, --detach

run in the background and print the ID

Default: False

checkpoint (c)¶

manage checkpoints

det checkpoint [-h] subcommand ...

Positional Arguments¶

subcommand: Possible choices: help, download

Sub-commands:¶

help¶

show help for this command

det checkpoint help [-h]

download¶

download checkpoint from S3 or GCS

det checkpoint download [-h] trial_id step_id output_dir

Positional Arguments¶

trial_id

trial ID

step_id

step ID

output_dir

output directory

Default: “.”

experiment (e)¶

manage experiments

det experiment [-h] subcommand ...

Positional Arguments¶

subcommand: Possible choices: help, activate, archive, cancel, config, create, describe, download-model-def, kill, label, list, list-checkpoints, lc, list-trials, lt, pause, set, unarchive, wait

Sub-commands:¶

help¶

show help for this command

det experiment help [-h]

activate¶

activate experiment

det experiment activate [-h] experiment_id

Positional Arguments¶

experiment_id: experiment ID to activate

archive¶

archive experiment

det experiment archive [-h] experiment_id

Positional Arguments¶

experiment_id: experiment ID to archive

cancel¶

cancel experiment

det experiment cancel [-h] experiment_id

Positional Arguments¶

experiment_id: experiment ID to cancel

config¶

display experiment config

det experiment config [-h] experiment_id

Positional Arguments¶

experiment_id: experiment ID

create¶

create experiment

det experiment create [-h] [-g] [--template TEMPLATE] [-f | --paused | -t]
                      config_file model_def

Positional Arguments¶

config_file: experiment config file (.yaml)
model_def: file or directory containing model definition

Named Arguments¶

-g, --git

Associate git metadata with this experiment. This flag assumes that git is installed, a .git repository exists in the model definition directory, and that the git working tree of that repository is empty.

Default: False

--template

name of template to apply to the experiment configuration

-f, --follow-first-trial

follow the logs of the first trial that is created

Default: False

--paused

do not activate the experiment

Default: False

-t, --test-mode

Test the experiment configuration and model definition by creating and scheduling a very small experiment. This command will verify that a training step and validation step run successfully and that checkpoints can be saved. The test experiment will be archived on creation.

Default: False

describe¶

describe experiment

det experiment describe [-h] [--metrics] [--csv | --json | --outdir OUTDIR]
                        experiment_ids

Positional Arguments¶

experiment_ids: comma-separated list of experiment IDs to describe

Named Arguments¶

--metrics

display full metrics

Default: False

--csv

print as CSV

Default: False

--json

print as JSON

Default: False

--outdir

directory to save output

download-model-def¶

download model definition

det experiment download-model-def [-h] [--output-dir OUTPUT_DIR] experiment_id

Positional Arguments¶

experiment_id: experiment ID

Named Arguments¶

--output-dir

output directory

Default: .

kill¶

kill experiment

det experiment kill [-h] experiment_id

Positional Arguments¶

experiment_id: experiment ID

label¶

manage experiment labels

det experiment label [-h] subsubcommand ...

Positional Arguments¶

subsubcommand: Possible choices: help, add, remove

Sub-commands:¶

help¶

show help for this command

det experiment label help [-h]

add¶

add label

det experiment label add [-h] experiment_id label

Positional Arguments¶

experiment_id: experiment ID
label: label

remove¶

remove label

det experiment label remove [-h] experiment_id label

Positional Arguments¶

experiment_id: experiment ID
label: label

list¶

list experiments

det experiment list [-h] [--all] [--csv]

Named Arguments¶

--all, -a

show all experiments (including archived and other users’)

Default: False

--csv

print as CSV

Default: False

list-checkpoints (lc)¶

list checkpoints of experiment

det experiment list-checkpoints [-h] [--best BEST] [-d DOWNLOAD_DIR] [--csv]
                                experiment_id

Positional Arguments¶

experiment_id: experiment ID

Named Arguments¶

--best

Return the best N checkpoints for this experiment. If this flag is used, only checkpoints with an associated validation metric will be considered.

-d, --download-dir

download the listed checkpoints to this directory. The resources of each checkpoint will be saved in a subdirectory labeled with the experiment ID, trial ID, and step ID. This flag is only supported for experiments configured to use S3 or GCS checkpoint storage.

--csv

print as CSV

Default: False

list-trials (lt)¶

list trials of experiment

det experiment list-trials [-h] [--csv] experiment_id

Positional Arguments¶

experiment_id: experiment ID

Named Arguments¶

--csv

print as CSV

Default: False

pause¶

pause experiment

det experiment pause [-h] experiment_id

Positional Arguments¶

experiment_id: experiment ID to pause

set¶

set experiment attributes

det experiment set [-h] subsubcommand ...

Positional Arguments¶

subsubcommand: Possible choices: help, description, gc-policy, max-slots, weight

Sub-commands:¶

help¶

show help for this command

det experiment set help [-h]

description¶

set experiment description

det experiment set description [-h] experiment_id description

Positional Arguments¶

experiment_id: experiment ID to modify
description: experiment description

gc-policy¶

set experiment GC policy and run GC

det experiment set gc-policy [-h] --save-experiment-best SAVE_EXPERIMENT_BEST
                             --save-trial-best SAVE_TRIAL_BEST
                             --save-trial-latest SAVE_TRIAL_LATEST [--yes]
                             experiment_id

Positional Arguments¶

experiment_id: experiment ID to modify

Named Arguments¶

--save-experiment-best

number of best checkpoints per experiment to save

--save-trial-best

number of best checkpoints per trial to save

--save-trial-latest

number of latest checkpoints per trial to save

--yes

automatically answer yes to prompts

Default: False

max-slots¶

set max_slots of experiment

det experiment set max-slots [-h] experiment_id max_slots

Positional Arguments¶

experiment_id: experiment ID to modify
max_slots: max slots

weight¶

set weight of experiment

det experiment set weight [-h] experiment_id weight

Positional Arguments¶

experiment_id: experiment ID to modify
weight: weight

unarchive¶

unarchive experiment

det experiment unarchive [-h] experiment_id

Positional Arguments¶

experiment_id: experiment ID to unarchive

wait¶

wait for experiment to reach terminal state

det experiment wait [-h] [--polling-interval POLLING_INTERVAL] experiment_id

Positional Arguments¶

experiment_id: experiment ID

Named Arguments¶

--polling-interval

the interval (in seconds) to poll for updated state

Default: 5

master¶

manage master

det master [-h] subcommand ...

Positional Arguments¶

subcommand: Possible choices: help, logs

Sub-commands:¶

help¶

show help for this command

det master help [-h]

logs¶

fetch master logs

det master logs [-h] [-f] [--tail TAIL]

Named Arguments¶

-f, --follow

follow the logs of master, similar to tail -f

Default: False

--tail

number of lines to show, counting from the end of the log (default is all)

notebook¶

manage notebooks

det notebook [-h] subcommand ...

Positional Arguments¶

subcommand: Possible choices: help, config, kill, list, ls, logs, open, start

Sub-commands:¶

help¶

show help for this command

det notebook help [-h]

config¶

display notebook config

det notebook config [-h] id

Positional Arguments¶

id: notebook ID

kill¶

kill a notebook

det notebook kill [-h] [-f] notebook_id [notebook_id ...]

Positional Arguments¶

notebook_id: notebook ID

Named Arguments¶

-f, --force

ignore errors

Default: False

list (ls)¶

list notebooks

det notebook list [-h] [-q] [--all]

Named Arguments¶

-q, --quiet

only display the IDs

Default: False

--all, -a

show all notebooks (including other users’)

Default: False

logs¶

fetch notebook logs

det notebook logs [-h] [-f] [--tail TAIL] notebook_id

Positional Arguments¶

notebook_id: notebook ID

Named Arguments¶

-f, --follow

follow the logs of a notebook, similar to tail -f

Default: False

--tail

number of lines to show, counting from the end of the log

Default: 10

open¶

open an existing notebook

det notebook open [-h] notebook_id

Positional Arguments¶

notebook_id: notebook ID

start¶

start a new notebook

det notebook start [-h] [--config-file CONFIG_FILE] [-v VOLUME] [-c CONTEXT]
                   [--config CONFIG] [--template TEMPLATE] [--no-browser] [-d]

Named Arguments¶

--config-file

command config file (.yaml)

-v, --volume

A mount specification in the form of <host path>:<container path>. The given path on the host machine will be mounted under the given path in the command container.

Default: []

-c, --context

The filepath to a directory that contains the set of files used to execute the command. All files under this directory will be packaged, maintaining the existing directory structure. The total byte contents of the directory must not exceed 96 MB. By default, the context directory will be empty.

--config

Additional configuration arguments for setting up a command. Arguments should be specified as key=value. Nested configuration keys can be specified by dot notation, e.g., resources.slots=4. List values can be specified by comma-separated values.

Default: []

--template

name of template to apply to the notebook configuration

--no-browser

don’t open the notebook in a browser after startup

Default: False

-d, --detach

run in the background and print the ID

Default: False

preview-search¶

preview search

det preview-search [-h] config_file

Positional Arguments¶

config_file: experiment config file (.yaml)

shell¶

manage shells

det shell [-h] subcommand ...

Positional Arguments¶

subcommand: Possible choices: help, config, kill, list, logs, open, start

Sub-commands:¶

help¶

show help for this command

det shell help [-h]

config¶

display shell config

det shell config [-h] id

Positional Arguments¶

id: shell ID

kill¶

kill a shell

det shell kill [-h] [-f] shell_id [shell_id ...]

Positional Arguments¶

shell_id: shell ID

Named Arguments¶

-f, --force

ignore errors

Default: False

list¶

list shells

det shell list [-h] [-q] [--all]

Named Arguments¶

-q, --quiet

only display the IDs

Default: False

--all, -a

show all shells (including other users’)

Default: False

logs¶

fetch shell logs

det shell logs [-h] [-f] [--tail TAIL] shell_id

Positional Arguments¶

shell_id: shell ID

Named Arguments¶

-f, --follow

follow the logs of a shell, similar to tail -f

Default: False

--tail

number of lines to show, counting from the end of the log

Default: 10

open¶

open an existing shell

det shell open [-h] [-o SSH_OPTS] shell_id

Positional Arguments¶

shell_id: shell ID

Named Arguments¶

-o, --ssh-opts

additional ssh options when connecting to the shell

Default: “”

start¶

start a new shell

det shell start [-h] [--config-file CONFIG_FILE] [-v VOLUME] [-c CONTEXT]
                [-o SSH_OPTS] [--config CONFIG] [-p] [--template TEMPLATE]
                [-d]

Named Arguments¶

--config-file

command config file (.yaml)

-v, --volume

A mount specification in the form of <host path>:<container path>. The given path on the host machine will be mounted under the given path in the command container.

Default: []

-c, --context

The filepath to a directory that contains the set of files used to execute the command. All files under this directory will be packaged, maintaining the existing directory structure. The total byte contents of the directory must not exceed 96 MB. By default, the context directory will be empty.

-o, --ssh-opts

additional SSH options when connecting to the shell

Default: “”

--config

Additional configuration arguments for setting up a command. Arguments should be specified as key=value. Nested configuration keys can be specified by dot notation, e.g., resources.slots=4. List values can be specified by comma-separated values.

Default: []

-p, --passphrase

passphrase to encrypt the shell private key

Default: False

--template

name of template to apply to the shell configuration

-d, --detach

run in the background and print the ID

Default: False

slot (s)¶

manage slots

det slot [-h] subcommand ...

Positional Arguments¶

subcommand: Possible choices: help, disable, enable, list

Sub-commands:¶

help¶

show help for this command

det slot help [-h]

disable¶

disable slot on agent

det slot disable [-h] agent_id slot_id

Positional Arguments¶

agent_id: agent ID
slot_id: slot ID

enable¶

enable slot on agent

det slot enable [-h] agent_id slot_id

Positional Arguments¶

agent_id: agent ID
slot_id: slot ID

list¶

list slots in cluster

det slot list [-h]

task¶

manage tasks (commands, experiments, notebooks, shells, tensorboards)

det task [-h] subcommand ...

Positional Arguments¶

subcommand: Possible choices: help, list

Sub-commands:¶

help¶

show help for this command

det task help [-h]

list¶

list tasks in cluster

det task list [-h] [--csv]

Named Arguments¶

--csv

print as CSV

Default: False

template (tpl)¶

manage config templates

det template [-h] subcommand ...

Positional Arguments¶

subcommand: Possible choices: help, describe, list, ls, remove, rm, set

Sub-commands:¶

help¶

show help for this command

det template help [-h]

describe¶

describe config template

det template describe [-h] template_name

Positional Arguments¶

template_name: template name

list (ls)¶

list config templates

det template list [-h] [-d]

Named Arguments¶

-d, --details

show the configs of the templates

Default: False

remove (rm)¶

remove config template

det template remove [-h] template_name

Positional Arguments¶

template_name: template name

set¶

set config template

det template set [-h] template_name template_file

Positional Arguments¶

template_name: template name
template_file: config template file (.yaml)

tensorboard¶

manage TensorBoard instances

det tensorboard [-h] subcommand ...

Positional Arguments¶

subcommand: Possible choices: help, kill, list, ls, logs, open, start

Sub-commands:¶

help¶

show help for this command

det tensorboard help [-h]

kill¶

kill TensorBoard instance

det tensorboard kill [-h] [-f] tensorboard_id [tensorboard_id ...]

Positional Arguments¶

tensorboard_id: TensorBoard ID

Named Arguments¶

-f, --force

ignore errors

Default: False

list (ls)¶

list TensorBoard instances

det tensorboard list [-h] [-q] [--all]

Named Arguments¶

-q, --quiet

only display the IDs

Default: False

--all, -a

show all TensorBoards (including other users’)

Default: False

logs¶

fetch TensorBoard instance logs

det tensorboard logs [-h] [-f] [--tail TAIL] tensorboard_id

Positional Arguments¶

tensorboard_id: TensorBoard ID

Named Arguments¶

-f, --follow

follow the logs of a TensorBoard instance, similar to tail -f

Default: False

--tail

number of lines to show, counting from the end of the log

Default: 10

open¶

open existing TensorBoard instance

det tensorboard open [-h] tensorboard_id

Positional Arguments¶

tensorboard_id: TensorBoard ID

start¶

start new TensorBoard instance

det tensorboard start [-h] [-t TRIAL_IDS [TRIAL_IDS ...]] [--no-browser] [-d]
                      [experiment_ids [experiment_ids ...]]

Positional Arguments¶

experiment_ids: experiment IDs to load into TensorBoard. At most 100 trials from the specified experiment will be loaded into TensorBoard. If the experiment has more trials, the 100 best-performing trials will be used.

Named Arguments¶

-t, --trial-ids

trial IDs to load into TensorBoard; at most 100 trials are allowed per TensorBoard instance

--no-browser

don’t open TensorBoard in a browser after startup

Default: False

-d, --detach

run in the background and print the ID

Default: False

trial (t)¶

manage trials

det trial [-h] subcommand ...

Positional Arguments¶

subcommand: Possible choices: help, describe, download, kill, logs

Sub-commands:¶

help¶

show help for this command

det trial help [-h]

describe¶

describe trial

det trial describe [-h] [--metrics] [--csv | --json] trial_id

Positional Arguments¶

trial_id: trial ID

Named Arguments¶

--metrics

display full metrics

Default: False

--csv

print as CSV

Default: False

--json

print JSON

Default: False

download¶

download checkpoint for trial

det trial download [-h] (--best | --latest | --uuid UUID) [-o OUTPUT_DIR]
                   [--sort-by SORT_BY] [--smaller-is-better SMALLER_IS_BETTER]
                   [-q]
                   trial_id

Positional Arguments¶

trial_id: trial ID

Named Arguments¶

--best

download the checkpoint with the best validation metric

Default: False

--latest

download the most recent checkpoint

Default: False

--uuid

download a checkpoint by specifying its UUID

-o, --output-dir

Desired output directory for the checkpoint

--sort-by

The name of the validation metric to sort on. This argument is only used with –best. If –best is passed without –sort-by, the experiment’s searcher metric is assumed. If this argument is specified, –smaller-is-better must also be specified.

--smaller-is-better

The sort order for metrics when using –best with –sort-by. For example, ‘accuracy’ would require passing ‘–smaller-is-better false’. If –sort-by is specified, this argument must be specified.

-q, --quiet

only print the path to the checkpoint

Default: False

kill¶

forcibly terminate a trial

det trial kill [-h] trial_id

Positional Arguments¶

trial_id: trial ID

logs¶

fetch trial logs

det trial logs [-h] [-f] [--tail TAIL] trial_id

Positional Arguments¶

trial_id: trial ID

Named Arguments¶

-f, --follow

follow the logs of a running trial, similar to tail -f

Default: False

--tail

number of lines to show, counting from the end of the log (default is all)

user (u)¶

manage users

det user [-h] subcommand ...

Positional Arguments¶

subcommand: Possible choices: help, activate, change-password, create, deactivate, link-with-agent-user, list, login, logout, whoami

Sub-commands:¶

help¶

show help for this command

det user help [-h]

activate¶

activate user

det user activate [-h] username

Positional Arguments¶

username: name of user to activate

change-password¶

change password for user

det user change-password [-h] [target_user]

Positional Arguments¶

target_user: name of user to change password of

create¶

create user

det user create [-h] [--admin] username

Positional Arguments¶

username: name of new user

Named Arguments¶

--admin

give new user admin rights

Default: False

deactivate¶

deactivate user

det user deactivate [-h] username

Positional Arguments¶

username: name of user to deactivate

link-with-agent-user¶

link a user with UID/GID on agent

det user link-with-agent-user [-h] [--agent-uid AGENT_UID]
                              [--agent-user AGENT_USER]
                              [--agent-gid AGENT_GID]
                              [--agent-group AGENT_GROUP]
                              det_username

Positional Arguments¶

det_username: name of Determined user to link

Named Arguments¶

--agent-uid: UID on the agent to run tasks as
--agent-user: user on the agent to run tasks as
--agent-gid: GID on agent to run tasks as
--agent-group: group on the agent to run tasks as

list¶

list users

det user list [-h]

logout¶

log out user

det user logout [-h]

whoami¶

print the active user

det user whoami [-h]

version¶

show version information

det version [-h]

Environment Variables¶

DET_MASTER: The network address of the master of the Determined installation. The value can be overridden using the -m flag.

Examples¶

det e, det experiment, det experiment list: Show information about experiments in the cluster.
det -m 1.2.3.4 e, DET_MASTER=1.2.3.4 det e: Show information about experiments in the cluster at the network address 1.2.3.4.
det t logs -f 289: Show the existing logs for trial 289 and continue showing new logs as they come in.
det e label add 17 foobar: Add the label “foobar” to experiment 17.
det e describe 493 --metrics --csv: Display information about experiment 493, including full metrics information, in CSV format.
det e create -f --paused const.yaml .: Create an experiment with the configuration file const.yaml and the code contained in the current directory. The experiment will be created in a paused state (that is, it will not be scheduled on the cluster until it is activated).
det e set max-slots 85 4: Ensure that experiment 85 does not take up more than 4 slots in the cluster.
det u create --admin hoid: Create a new user named “hoid” with admin privileges.
det version: Show detailed information about the CLI and master. Note that this command does not take both an object and an action.