Shortcuts

Command ConfigurationΒΆ

A command/notebook configuration file is a YAML file that provides settings for commands and notebook tasks. Typically, the configuration is passed as a command-line argument when commands and notebooks are created with the Determined CLI. It may contain the following fields. All fields are optional unless otherwise specified.

  • description: A human-readable description of the command/notebook. This does not need to be unique. The default description consists of a timestamp and the entrypoint of the command.

  • environment: Specifies the environment of the container that is used to execute the command/notebook. Default values are in bold.

    • image: Specifies a Docker image to use when executing the command. The image must be available via docker pull to every Determined agent host in the cluster. Users can customize environment variables for GPU vs. CPU agents differently by specifying a dict with two keys, cpu and gpu. Defaults to determinedai/environments:py-3.6.9-pytorch-1.4-tf-1.14-cpu for CPU agents and determinedai/environments:cuda-10-py-3.6.9-pytorch-1.4-tf-1.14-gpu for GPU agents.

    • force_pull_image: Forcibly pull the image from the Docker registry and bypass the Docker cache. Defaults to false.

    • environment_variables: Specifies a list of environment variables for the command container. Each element of the list should be a string of the form NAME=VALUE. See Environment Variables for more details.

    • registry_auth: Specifies the Docker registry credentials to use when pulling a Docker image, if needed.

      • username (required)

      • password (required)

      • server (optional)

      • email (optional)

  • resources: The resources Determined allows a command/notebook to use.

    • slots: Specifies the number of slots to use for the command/notebook. The default value is 1. The maximum value is the number of slots on the agent in the cluster with the most slots. For example, Determined will be unable to schedule a command that requests 4 slots if the Determined cluster is composed of agents with 2 slots each.

  • bind_mounts: Specifies a collection of directories that are bind-mounted into the Docker containers for execution. This can be used to allow commands to access additional data that is not contained in the command context. This field should consist of an array of entries. Note that users should ensure that the specified host paths are accessible on all agent hosts (e.g., by configuring a network file system appropriately). Defaults to an empty list.

    • host_path: (required) The file system path on each agent to use. Must be an absolute filepath.

    • container_path: (required) The file system path in the container to use. May be a relative filepath, in which case it will be mounted relative to the working directory inside the container. It is not allowed to mount directly into the working directory (container_path == ".") to reduce the risk of cluttering the host filesystem.

    • read_only: Whether the bind-mount should be a read-only mount. Defaults to false.

    • propagation: (Advanced users only) Optional propagation behavior for replicas of the bind-mount. Defaults to rprivate.

Hint

In addition to --config-file, det cmd run offers a --config flag to directly set configuration options from the command line with field=value syntax. field can use the dot character (.) to indicate nested fields (e.g., environment.python). value will be interpreted as a comma-separated list if it contains any commas. Options set by --config take precedence over those set by --config-file.