Environment Configuration

PEDL launches trials of experiments, called “trial runners”, and PEDL commands in customizable Docker containers. The configuration of the container is referred to as the environment.

Default Environment

In the current version of PEDL, the trial runners and commands are executed in containers with the following default settings:

  • Ubuntu 16.04

  • CUDA 10.0

  • Python 3.6.9

  • TensorFlow 1.14.0

  • PyTorch 1.3.0

  • Keras 2.2.4

PEDL will automatically select GPU-specific versions of each library when running on agents with GPUs. The trial environment can be further customized by defining a startup-hook.sh that gets run on container startup.

In addition to the above settings, all trial runner containers are launched with additional PEDL-specific harness code that orchestrates model training and evaluation in the container. Trial runner containers are also loaded with the experiment’s model definition and values of the hyperparameters for the current trial.

Customizing Environments

Customizing environments can be done by specifying environment variables for a task container or using a startup-hook.sh.

Environment Variables

For both trial runners and commands, PEDL allows users to configure the environment variables inside the container through the environment -> environment_variables configuration field of the experiment config. The format is a list of strings in the format NAME=VALUE:

environment:
  environment_variables:
    - A=hello world
    - B=$A
    - C=${B}
    # `A`, `B`, and `C` will each have the value `hello_world` in the container.

Variables are set sequentially, which can affect variables that depend on expansion of other variables.