Command-line Interface (CLI)

Installation

Users can also interact with Determined using a command-line interface. The CLI is distributed as a Python wheel package; once the wheel has been installed (see Installation for details), the CLI can be used via the det command.

Each ML engineer that wants to use Determined should install a copy of the Determined CLI on their local development machine. The CLI can be installed via pip:

pip install determined

The CLI requires Python >= 3.6. We suggest installing the CLI into a virtualenv, although this is optional. To install the CLI into a virtualenv, first activate the virtualenv and then type the command above.

After the CLI has been installed, it should be configured to connect to the Determined master at the appropriate IP address. This can be accomplished by setting the DET_MASTER environment variable:

export DET_MASTER=<master IP>

You may want to place this into the appropriate configuration file for your login shell (e.g., .bashrc).

More information about using the Determined CLI can be found in the reference documentation or by running det --help.

When trying to install the Determined command-line interface, I encounter this distutils error

Uninstalling a distutils installed project (...) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.

If a Python library has previously been installed in your environment with distutils or conda, pip may not be able to upgrade or downgrade the library to the version required by Determined. There are two recommended solutions:

  1. Install the Determined command-line interface into a fresh virtualenv with no previous Python packages installed.

  2. Use --ignore-installed with pip to force overwriting the library version(s).

    pip install --ignore-installed determined-cli

After upgrading Determined and trying to use the CLI, I get det: command not found error

When upgrading from a version earlier than 0.15.0, some users may need to uninstall Determined and then install it again by running:

pip uninstall -y determined-common determined determined-cli determined-deploy
pip install determined

Usage

CLI subcommands usually follow a <noun> <verb> form, similar to the paradigm of ip. Certain abbreviations are supported, and a missing verb is the same as list, when possible.

For example, the different commands within each of the blocks below all do the same thing:

# List all experiments.
$ det experiment list
$ det exp list
$ det e list
$ det e

# List all agents.
$ det agent list
$ det a list
$ det a

# List all slots.
$ det slot list
$ det slot
$ det s

For a complete description of the available nouns and abbreviations, see the output of det help. Each noun also provides a help verb that describes the possible verbs for that noun. Or you can provide -h or --help as an argument anywhere will cause the CLI to exit after printing help text for the object or action specified up to that point.

Setting the Master

The CLI should be installed on any machine where a user would like to access Determined. The -m or --master flag determines the network address of the Determined master that the CLI connects to. If this flag is not specified, the value of the DET_MASTER environment variable is used; if that environment variable is not set, the default address is localhost. The master address can be specified in three different formats:

  • example.org:port (if port is omitted, it defaults to 8080)

  • http://example.org:port (if port is omitted, it defaults to 80)

  • https://example.org:port (if port is omitted, it defaults to 443)

Examples:

# Connect to localhost, port 8080.
$ det experiment list

# Connect to example.org, port 8888.
$ det -m example.org:8888 e list

# Connect to example.org, port 80.
$ det -m http://example.org e list

# Connect to example.org, port 443.
$ det -m https://example.org e list

# Connect to example.org, port 8080.
$ det -m example.org e list

# Set default Determined master address to example.org, port 8888.
$ export DET_MASTER="example.org:8888"

Examples

  • det e, det experiment, det experiment list: Show information about experiments in the cluster.

  • det -m 1.2.3.4 e, DET_MASTER=1.2.3.4 det e: Show information about experiments in the cluster at the network address 1.2.3.4.

  • det t logs -f 289: Show the existing logs for trial 289 and continue showing new logs as they come in.

  • det e label add 17 foobar: Add the label “foobar” to experiment 17.

  • det e describe 493 --metrics --csv: Display information about experiment 493, including full metrics information, in CSV format.

  • det e create -f --paused const.yaml .: Create an experiment with the configuration file const.yaml and the code contained in the current directory. The experiment will be created in a paused state (that is, it will not be scheduled on the cluster until it is activated).

  • det e set max-slots 85 4: Ensure that experiment 85 does not take up more than 4 slots in the cluster.

  • det u create --admin hoid: Create a new user named “hoid” with admin privileges.

  • det version: Show detailed information about the CLI and master. Note that this command does not take both an object and an action.

Environment Variables

  • DET_MASTER: The network address of the master of the Determined installation. The value can be overridden using the -m flag.

Full Listing

Determined command-line client

usage: det [-h]