Quick Start Chapter 1: Training a Model

Here we'll walk through training a model in PEDL using a provided example.

  1. From the examples page, download the MNIST TensorFlow example, mnist_tf.tgz, which contains all the files needed to run a convolutional neural net in TensorFlow using PEDL.

  2. Extract the downloaded example file mnist_tf.tgz into a directory mnist_tf. Double-clicking the file in Mac Finder or Windows File Explorer is typically sufficient. Or, in a command line, navigate to the directory containing the file and run the following commands:

    mkdir mnist_tf
    tar -xvzf mnist_tf.tgz -C mnist_tf

    After extraction, you will find that the mnist_tf directory contains a collection of .py files and .yaml files. Generally, the Python file(s) will specify the model and dataset, while the .yaml files specify the experiment configuration. In this particular example, the model definition is in a single Python file, whereas each .yaml file specifies a separate experiment configuration for that model.

  3. Navigate to the directory mnist_tf containing the extracted files of the MNIST TensorFlow example.

  4. Double-check that the PEDL CLI is installed. If so, the command pedl -h should return information about the CLI.

    If you are planning to run PEDL in a virtualenv, activate the virtualenv before doing this step.

  5. To create an experiment, use the following command:

    pedl experiment create const.yaml .

    experiment can also be abbreviated to e:

    pedl e create const.yaml .

    The generated experiment uses the const.yaml experiment configuration as well as the convolutional neural net defined in model_def.py on the MNIST dataset. The entire directory is passed in to the experiment definition. The -f option can be added after create to follow the logs of the experiment.

    In general, the command to create a new experiment in PEDL is:

    pedl e create <experiment configuration file> <model definition directory>

    In the above instructions, the const.yaml experiment configuration file sets the searcher mode to train one model with fixed hyperparameters; replacing it by another provided experiment configuration file will run a different hyperparameter search mode (e.g., random, grid, or adaptive) for the model. See QS2: hyperparameter search.

    The model definition and data source can be specified by either one Python file or a directory. See QS4: defining models.

  6. To see more details for the experiment, use the following command. The experiment number is returned by the PEDL CLI during experiment creation in a message of the form "Created experiment 12".

    pedl e describe <experiment number>
    More information on how to interact with PEDL experiments can be found with pedl e -h.

    The command pedl t describe <trial number> will give details on a particular trial within an experiment.

  7. If desired, an experiment can be canceled before it is completed using this command:

    pedl e cancel <experiment number>
  8. Try creating experiments with other models and frameworks provided on the examples page using a similar workflow. If the example contains an __init__.py file, the whole directory should be passed in as the model definition; otherwise, there should be only one .py file that contains the model definition. The one exception is the mnist_pytorch example, which contains two distinct model definitions in separate .py files.