Command-line Interface

The PEDL command-line interface (CLI) is installed under the name pedl. Most uses of it require arguments specifying a type of object to act on and the action to perform, followed by any additional arguments or options. For example,

pedl experiment config 17

displays the configuration for experiment 17. Some commands have an additional level of nesting:

pedl experiment label add 17 foobar

adds the label “foobar” to experiment 17.

The names of some objects, actions, and options may be abbreviated; alternative names are shown below separated by commas. For example, these two commands are equivalent:

pedl -u admin experiment list

pedl --user admin experiment list

as are these two:

pedl e config 17

pedl experiment config 17

Additionally, when list is a valid action, it is the default when no action is specified, so pedl e and pedl s are equivalent to pedl experiment list and pedl slot list, for example.

Additionally, when list is a valid action, it is the default when no action is specified, so the following pairs of commands are also equivalent:

pedl e

pedl experiment list

pedl s

pedl slot list

Providing -h or --help as an argument anywhere will cause the CLI to exit after printing help text for the object or action specified up to that point.

Full Listing

PEDL command-line client

usage: pedl [-h] [-u username] [-m address] [-v] command ...

Positional Arguments

command

Possible choices: help, agent, a, command, cmd, checkpoint, c, experiment, e, master, notebook, preview-search, shell, slot, s, task, template, tpl, tensorboard, trial, t, user, u, version

Named Arguments

-u, --user

run as the given user

-m, --master

master address

Default: “localhost:8080”

-v, --version

print CLI version and exit

Sub-commands:

help

show help for this command

pedl help [-h]

agent (a)

manage agents

pedl agent [-h] subcommand ...
Positional Arguments
subcommand

Possible choices: help, disable, enable, list

Sub-commands:
help

show help for this command

pedl agent help [-h]
disable

disable agent

pedl agent disable [-h] [--all] [agent_id]
Positional Arguments
agent_id

agent ID

Named Arguments
--all

disable all agents

Default: False

enable

enable agent

pedl agent enable [-h] [--all] [agent_id]
Positional Arguments
agent_id

agent ID

Named Arguments
--all

enable all agents

Default: False

list

list agents

pedl agent list [-h]

command (cmd)

manage commands

pedl command [-h] subcommand ...
Positional Arguments
subcommand

Possible choices: help, config, kill, list, ls, logs, run

Sub-commands:
help

show help for this command

pedl command help [-h]
config

display command config

pedl command config [-h] id
Positional Arguments
id

command ID

kill

forcibly terminate a command

pedl command kill [-h] [-f] command_id [command_id ...]
Positional Arguments
command_id

command ID

Named Arguments
-f, --force

ignore errors

Default: False

list (ls)

list commands

pedl command list [-h] [-q] [--all]
Named Arguments
-q, --quiet

only display the IDs

Default: False

--all, -a

show all commands (including other users’)

Default: False

logs

fetch command logs

pedl command logs [-h] [-f] [--tail TAIL] command_id
Positional Arguments
command_id

command ID

Named Arguments
-f, --follow

follow the logs of a command, similar to tail -f

Default: False

--tail

number of lines to show, counting from the end of the log

Default: 10

run

create command

pedl command run [-h] [--config-file CONFIG_FILE] [-v VOLUME] [-c CONTEXT]
                 [--config CONFIG] [--template TEMPLATE] [-d]
                 ...
Positional Arguments
entrypoint

entrypoint command and arguments to execute

Named Arguments
--config-file

command config file (.yaml)

-v, --volume

A mount specification in the form of <host path>:<container path>. The given path on the host machine will be mounted under the given path in the command container.

Default: []

-c, --context

The filepath to a directory that contains the set of files used to execute the command. All files under this directory will be packaged, maintaining the existing directory structure. The total byte contents of the directory must not exceed 96 MB. By default, the context directory will be empty.

--config

Additional configuration arguments for setting up a command. Arguments should be specified as key=value. Nested configuration keys can be specified by dot notation, e.g., resources.slots=4. List values can be specified by comma-separated values.

Default: []

--template

name of template to apply to the command configuration

-d, --detach

run in the background and print the ID

Default: False

checkpoint (c)

manage checkpoints

pedl checkpoint [-h] subcommand ...
Positional Arguments
subcommand

Possible choices: help, download

Sub-commands:
help

show help for this command

pedl checkpoint help [-h]
download

download checkpoint from S3 or GCS

pedl checkpoint download [-h] trial_id step_id output_dir
Positional Arguments
trial_id

trial ID

step_id

step ID

output_dir

output directory

Default: “.”

experiment (e)

manage experiments

pedl experiment [-h] subcommand ...
Positional Arguments
subcommand

Possible choices: help, activate, archive, cancel, config, create, describe, download-model-def, kill, label, list, list-checkpoints, lc, list-trials, lt, pause, set, unarchive, wait

Sub-commands:
help

show help for this command

pedl experiment help [-h]
activate

activate experiment

pedl experiment activate [-h] experiment_id
Positional Arguments
experiment_id

experiment ID to activate

archive

archive experiment

pedl experiment archive [-h] experiment_id
Positional Arguments
experiment_id

experiment ID to archive

cancel

cancel experiment

pedl experiment cancel [-h] experiment_id
Positional Arguments
experiment_id

experiment ID to cancel

config

display experiment config

pedl experiment config [-h] experiment_id
Positional Arguments
experiment_id

experiment ID

create

create experiment

pedl experiment create [-h] [-g] [--template TEMPLATE] [-f | --paused | -t]
                       config_file model_def
Positional Arguments
config_file

experiment config file (.yaml)

model_def

file or directory containing model definition

Named Arguments
-g, --git

Associate git metadata with this experiment. This flag assumes that git is installed, a .git repository exists in the model definition directory, and that the git working tree of that repository is empty.

Default: False

--template

name of template to apply to the experiment configuration

-f, --follow-first-trial

follow the logs of the first trial that is created

Default: False

--paused

do not activate the experiment

Default: False

-t, --test-mode

Test the experiment configuration and model definition by creating and scheduling a very small experiment. This command will verify that a training step and validation step run successfully and that checkpoints can be saved. The test experiment will be archived on creation.

Default: False

describe

describe experiment

pedl experiment describe [-h] [--metrics] [--csv | --json | --outdir OUTDIR]
                         experiment_ids
Positional Arguments
experiment_ids

comma-separated list of experiment IDs to describe

Named Arguments
--metrics

display full metrics

Default: False

--csv

print as CSV

Default: False

--json

print as JSON

Default: False

--outdir

directory to save output

download-model-def

download model definition

pedl experiment download-model-def [-h] [--output-dir OUTPUT_DIR]
                                   experiment_id
Positional Arguments
experiment_id

experiment ID

Named Arguments
--output-dir

output directory

Default: .

kill

kill experiment

pedl experiment kill [-h] experiment_id
Positional Arguments
experiment_id

experiment ID

label

manage experiment labels

pedl experiment label [-h] subsubcommand ...
Positional Arguments
subsubcommand

Possible choices: help, add, remove

Sub-commands: help

show help for this command

pedl experiment label help [-h]
add

add label

pedl experiment label add [-h] experiment_id label
Positional Arguments
experiment_id

experiment ID

label

label

remove

remove label

pedl experiment label remove [-h] experiment_id label
Positional Arguments
experiment_id

experiment ID

label

label

list

list experiments

pedl experiment list [-h] [--all] [--csv]
Named Arguments
--all, -a

show all experiments (including archived and other users’)

Default: False

--csv

print as CSV

Default: False

list-checkpoints (lc)

list checkpoints of experiment

pedl experiment list-checkpoints [-h] [--best BEST] [-d DOWNLOAD_DIR] [--csv]
                                 experiment_id
Positional Arguments
experiment_id

experiment ID

Named Arguments
--best

Return the best N checkpoints for this experiment. If this flag is used, only checkpoints with an associated validation metric will be considered.

-d, --download-dir

download the listed checkpoints to this directory. The resources of each checkpoint will be saved in a subdirectory labeled with the experiment ID, trial ID, and step ID. This flag is only supported for experiments configured to use S3 or GCS checkpoint storage.

--csv

print as CSV

Default: False

list-trials (lt)

list trials of experiment

pedl experiment list-trials [-h] [--csv] experiment_id
Positional Arguments
experiment_id

experiment ID

Named Arguments
--csv

print as CSV

Default: False

pause

pause experiment

pedl experiment pause [-h] experiment_id
Positional Arguments
experiment_id

experiment ID to pause

set

set experiment attributes

pedl experiment set [-h] subsubcommand ...
Positional Arguments
subsubcommand

Possible choices: help, description, gc-policy, max-slots, weight

Sub-commands: help

show help for this command

pedl experiment set help [-h]
description

set experiment description

pedl experiment set description [-h] experiment_id description
Positional Arguments
experiment_id

experiment ID to modify

description

experiment description

gc-policy

set experiment GC policy and run GC

pedl experiment set gc-policy [-h] --save-experiment-best SAVE_EXPERIMENT_BEST
                              --save-trial-best SAVE_TRIAL_BEST
                              --save-trial-latest SAVE_TRIAL_LATEST [--yes]
                              experiment_id
Positional Arguments
experiment_id

experiment ID to modify

Named Arguments
--save-experiment-best

number of best checkpoints per experiment to save

--save-trial-best

number of best checkpoints per trial to save

--save-trial-latest

number of latest checkpoints per trial to save

--yes

automatically answer yes to prompts

Default: False

max-slots

set max_slots of experiment

pedl experiment set max-slots [-h] experiment_id max_slots
Positional Arguments
experiment_id

experiment ID to modify

max_slots

max slots

weight

set weight of experiment

pedl experiment set weight [-h] experiment_id weight
Positional Arguments
experiment_id

experiment ID to modify

weight

weight

unarchive

unarchive experiment

pedl experiment unarchive [-h] experiment_id
Positional Arguments
experiment_id

experiment ID to unarchive

wait

wait for experiment to reach terminal state

pedl experiment wait [-h] [--polling-interval POLLING_INTERVAL] experiment_id
Positional Arguments
experiment_id

experiment ID

Named Arguments
--polling-interval

the interval (in seconds) to poll for updated state

Default: 5

master

manage master

pedl master [-h] subcommand ...
Positional Arguments
subcommand

Possible choices: help, logs

Sub-commands:
help

show help for this command

pedl master help [-h]
logs

fetch master logs

pedl master logs [-h] [-f] [--tail TAIL]
Named Arguments
-f, --follow

follow the logs of master, similar to tail -f

Default: False

--tail

number of lines to show, counting from the end of the log (default is all)

notebook

manage notebooks

pedl notebook [-h] subcommand ...
Positional Arguments
subcommand

Possible choices: help, config, kill, list, ls, logs, open, start

Sub-commands:
help

show help for this command

pedl notebook help [-h]
config

display notebook config

pedl notebook config [-h] id
Positional Arguments
id

notebook ID

kill

kill a notebook

pedl notebook kill [-h] [-f] notebook_id [notebook_id ...]
Positional Arguments
notebook_id

notebook ID

Named Arguments
-f, --force

ignore errors

Default: False

list (ls)

list notebooks

pedl notebook list [-h] [-q] [--all]
Named Arguments
-q, --quiet

only display the IDs

Default: False

--all, -a

show all notebooks (including other users’)

Default: False

logs

fetch notebook logs

pedl notebook logs [-h] [-f] [--tail TAIL] notebook_id
Positional Arguments
notebook_id

notebook ID

Named Arguments
-f, --follow

follow the logs of a notebook, similar to tail -f

Default: False

--tail

number of lines to show, counting from the end of the log

Default: 10

open

open an existing notebook

pedl notebook open [-h] notebook_id
Positional Arguments
notebook_id

notebook ID

start

start a new notebook

pedl notebook start [-h] [--config-file CONFIG_FILE] [-v VOLUME] [-c CONTEXT]
                    [--config CONFIG] [--template TEMPLATE] [--no-browser]
                    [-d]
Named Arguments
--config-file

command config file (.yaml)

-v, --volume

A mount specification in the form of <host path>:<container path>. The given path on the host machine will be mounted under the given path in the command container.

Default: []

-c, --context

The filepath to a directory that contains the set of files used to execute the command. All files under this directory will be packaged, maintaining the existing directory structure. The total byte contents of the directory must not exceed 96 MB. By default, the context directory will be empty.

--config

Additional configuration arguments for setting up a command. Arguments should be specified as key=value. Nested configuration keys can be specified by dot notation, e.g., resources.slots=4. List values can be specified by comma-separated values.

Default: []

--template

name of template to apply to the notebook configuration

--no-browser

don’t open the notebook in a browser after startup

Default: False

-d, --detach

run in the background and print the ID

Default: False

preview search

pedl preview-search [-h] config_file
Positional Arguments
config_file

experiment config file (.yaml)

shell

manage shells

pedl shell [-h] subcommand ...
Positional Arguments
subcommand

Possible choices: help, config, kill, list, logs, open, start

Sub-commands:
help

show help for this command

pedl shell help [-h]
config

display shell config

pedl shell config [-h] id
Positional Arguments
id

shell ID

kill

kill a shell

pedl shell kill [-h] [-f] shell_id [shell_id ...]
Positional Arguments
shell_id

shell ID

Named Arguments
-f, --force

ignore errors

Default: False

list

list shells

pedl shell list [-h] [-q] [--all]
Named Arguments
-q, --quiet

only display the IDs

Default: False

--all, -a

show all shells (including other users’)

Default: False

logs

fetch shell logs

pedl shell logs [-h] [-f] [--tail TAIL] shell_id
Positional Arguments
shell_id

shell ID

Named Arguments
-f, --follow

follow the logs of a shell, similar to tail -f

Default: False

--tail

number of lines to show, counting from the end of the log

Default: 10

open

open an existing shell

pedl shell open [-h] [-o SSH_OPTS] shell_id
Positional Arguments
shell_id

shell ID

Named Arguments
-o, --ssh-opts

additional ssh options when connecting to the shell

Default: “”

start

start a new shell

pedl shell start [-h] [--config-file CONFIG_FILE] [-v VOLUME] [-c CONTEXT]
                 [-o SSH_OPTS] [--config CONFIG] [-p] [--template TEMPLATE]
                 [-d]
Named Arguments
--config-file

command config file (.yaml)

-v, --volume

A mount specification in the form of <host path>:<container path>. The given path on the host machine will be mounted under the given path in the command container.

Default: []

-c, --context

The filepath to a directory that contains the set of files used to execute the command. All files under this directory will be packaged, maintaining the existing directory structure. The total byte contents of the directory must not exceed 96 MB. By default, the context directory will be empty.

-o, --ssh-opts

additional SSH options when connecting to the shell

Default: “”

--config

Additional configuration arguments for setting up a command. Arguments should be specified as key=value. Nested configuration keys can be specified by dot notation, e.g., resources.slots=4. List values can be specified by comma-separated values.

Default: []

-p, --passphrase

passphrase to encrypt the shell private key

Default: False

--template

name of template to apply to the shell configuration

-d, --detach

run in the background and print the ID

Default: False

slot (s)

manage slots

pedl slot [-h] subcommand ...
Positional Arguments
subcommand

Possible choices: help, disable, enable, list

Sub-commands:
help

show help for this command

pedl slot help [-h]
disable

disable slot on agent

pedl slot disable [-h] agent_id slot_id
Positional Arguments
agent_id

agent ID

slot_id

slot ID

enable

enable slot on agent

pedl slot enable [-h] agent_id slot_id
Positional Arguments
agent_id

agent ID

slot_id

slot ID

list

list slots in cluster

pedl slot list [-h]

task

manage tasks

pedl task [-h] subcommand ...
Positional Arguments
subcommand

Possible choices: help, list

Sub-commands:
help

show help for this command

pedl task help [-h]
list

list tasks in cluster

pedl task list [-h] [--csv]
Named Arguments
--csv

print as CSV

Default: False

template (tpl)

manage config templates

pedl template [-h] subcommand ...
Positional Arguments
subcommand

Possible choices: help, describe, list, ls, remove, rm, set

Sub-commands:
help

show help for this command

pedl template help [-h]
describe

describe config template

pedl template describe [-h] template_name
Positional Arguments
template_name

template name

list (ls)

list config templates

pedl template list [-h] [-d]
Named Arguments
-d, --details

show the configs of the templates

Default: False

remove (rm)

remove config template

pedl template remove [-h] template_name
Positional Arguments
template_name

template name

set

set config template

pedl template set [-h] template_name template_file
Positional Arguments
template_name

template name

template_file

config template file (.yaml)

tensorboard

manage TensorBoard instances

pedl tensorboard [-h] subcommand ...
Positional Arguments
subcommand

Possible choices: help, kill, list, ls, logs, open, start

Sub-commands:
help

show help for this command

pedl tensorboard help [-h]
kill

kill TensorBoard instance

pedl tensorboard kill [-h] [-f] tensorboard_id [tensorboard_id ...]
Positional Arguments
tensorboard_id

TensorBoard ID

Named Arguments
-f, --force

ignore errors

Default: False

list (ls)

list TensorBoard instances

pedl tensorboard list [-h] [-q] [--all]
Named Arguments
-q, --quiet

only display the IDs

Default: False

--all, -a

show all TensorBoards (including other users’)

Default: False

logs

fetch TensorBoard instance logs

pedl tensorboard logs [-h] [-f] [--tail TAIL] tensorboard_id
Positional Arguments
tensorboard_id

TensorBoard ID

Named Arguments
-f, --follow

follow the logs of a TensorBoard instance, similar to tail -f

Default: False

--tail

number of lines to show, counting from the end of the log

Default: 10

open

open existing TensorBoard instance

pedl tensorboard open [-h] tensorboard_id
Positional Arguments
tensorboard_id

TensorBoard ID

start

start new TensorBoard instance

pedl tensorboard start [-h] [-t TRIAL_IDS [TRIAL_IDS ...]] [--no-browser] [-d]
                       [experiment_ids [experiment_ids ...]]
Positional Arguments
experiment_ids

experiment IDs to load into TensorBoard. At most 100 trials from the specified experiment will be loaded into TensorBoard. If the experiment has more trials, the 100 best-performing trials will be used.

Named Arguments
-t, --trial-ids

trial IDs to load into TensorBoard; at most 100 trials are allowed per TensorBoard instance

--no-browser

don’t open TensorBoard in a browser after startup

Default: False

-d, --detach

run in the background and print the ID

Default: False

trial (t)

manage trials

pedl trial [-h] subcommand ...
Positional Arguments
subcommand

Possible choices: help, describe, download, kill, logs

Sub-commands:
help

show help for this command

pedl trial help [-h]
describe

describe trial

pedl trial describe [-h] [--metrics] [--csv | --json] trial_id
Positional Arguments
trial_id

trial ID

Named Arguments
--metrics

display full metrics

Default: False

--csv

print as CSV

Default: False

--json

print JSON

Default: False

download

download checkpoint for trial

pedl trial download [-h] (--best | --latest | --uuid UUID) [-q] trial_id
Positional Arguments
trial_id

trial ID

Named Arguments
--best

download the checkpoint with the best validation metric

Default: False

--latest

download the most recent checkpoint

Default: False

--uuid

download a checkpoint by specifying its uuid

-q, --quiet

only print the path to the checkpoint

Default: False

kill

forcibly terminate a trial

pedl trial kill [-h] trial_id
Positional Arguments
trial_id

trial ID

logs

fetch trial logs

pedl trial logs [-h] [-f] [--tail TAIL] trial_id
Positional Arguments
trial_id

trial ID

Named Arguments
-f, --follow

follow the logs of a running trial, similar to tail -f

Default: False

--tail

number of lines to show, counting from the end of the log (default is all)

user (u)

manage users

pedl user [-h] subcommand ...
Positional Arguments
subcommand

Possible choices: help, activate, change-password, create, deactivate, link-with-agent-user, list, login, logout, whoami

Sub-commands:
help

show help for this command

pedl user help [-h]
activate

activate user

pedl user activate [-h] username
Positional Arguments
username

name of user to activate

change-password

change password for user

pedl user change-password [-h] [target_user]
Positional Arguments
target_user

name of user to change password of

create

create user

pedl user create [-h] [--admin] username
Positional Arguments
username

name of new user

Named Arguments
--admin

give new user admin rights

Default: False

deactivate

deactivate user

pedl user deactivate [-h] username
Positional Arguments
username

name of user to deactivate

link a user with UID/GID on agent

pedl user link-with-agent-user [-h] [--agent-uid AGENT_UID]
                               [--agent-user AGENT_USER]
                               [--agent-gid AGENT_GID]
                               [--agent-group AGENT_GROUP]
                               pedl_username
Positional Arguments
pedl_username

name of PEDL user to link

Named Arguments
--agent-uid

UID on the agent to run tasks as

--agent-user

user on the agent to run tasks as

--agent-gid

GID on agent to run tasks as

--agent-group

group on the agent to run tasks as

list

list users

pedl user list [-h]
login

log in user

pedl user login [-h] [username]
Positional Arguments
username

name of user to log in as

logout

log out user

pedl user logout [-h]
whoami

print the active user

pedl user whoami [-h]

version

show version information

pedl version [-h]

Environment Variables

  • PEDL_MASTER: The network address of the master of the PEDL installation. The value can be overridden using the -m flag.

Examples

  • pedl e, pedl experiment, pedl experiment list: Show information about experiments in the cluster.

  • pedl -m 1.2.3.4 e, PEDL_MASTER=1.2.3.4 pedl e: Show information about experiments in the cluster at the network address 1.2.3.4.

  • pedl t logs -f 289: Show the existing logs for trial 289 and continue showing new logs as they come in.

  • pedl e label add 17 foobar: Add the label “foobar” to experiment 17.

  • pedl e describe 493 --metrics --csv: Display information about experiment 493, including full metrics information, in CSV format.

  • pedl e create -f --paused const.yaml .: Create an experiment with the configuration file const.yaml and the code contained in the current directory. The experiment will be created in a paused state (that is, it will not be scheduled on the cluster until it is activated).

  • pedl e set max-slots 85 4: Ensure that experiment 85 does not take up more than 4 slots in the cluster.

  • pedl u create --admin hoid: Create a new user named “hoid” with admin privileges.

  • pedl version: Show detailed information about the CLI and master. Note that this command does not take both an object and an action.