Quick Start Chapter 4: Defining Models in PEDL

The model definition is the interface between PEDL and the user's preferred application framework, e.g., Keras or TensorFlow. See the model definition page for details. The examples page contains model definitions for all supported frameworks.

To specify the deep learning model, users will need to build the architecture of a network, specify the underlying iterative optimization algorithm, and define metrics such as loss, training metrics, and validation metrics. These tasks can be handled using the preferred framework. However, users should wrap these definitions in functions and classes that interface with PEDL. These form the model definition.

PEDL currently supports two types of model definitions:

  1. Standard Model Definition: Implement PEDL's provided Trial interface for your desired task. This option provides finer-grained control over PEDL model construction and computation.
  2. Simple Model Definition: Specify a directory of model code together with an entrypoint script that executes a training and validation procedure. This option requires very few code changes to set up and may be simplest if you're new to PEDL. This is currently supported for Keras only.

We'll introduce the standard model definition in more detail here. There are five interfaces supported: TensorFlow, TensorFlow Estimator, Keras Sequential, Keras Functional, and PyTorch. The interfaces are all based on the Trial class in PEDL. For example, to use the PyTorch interface, users should subclass PyTorchTrial, which is a subclass of Trial.

Within these framework-specific Trial subclasses, users typically control the training of their neural network by implementing the following methods of the class.

  • Network architecture (build_model() or build_graph()): Typically, the network as defined in the original framework can be copied straight into the content of this method. See the framework-specific documentation for syntax.
  • Batch size (batch_size()): The batch size to use for training.
  • Loss (loss()): Depending on the framework, the loss function may either be directly defined or specified in the model method.
  • Validation metrics (validation_metrics()): Validation metrics can be defined using the framework's API (e.g. keras.metrics) or as user-defined metrics. Within the TensorFlow and PyTorch interfaces, this method defines the validation metrics as nodes within the network architecture. Within the Keras interfaces, however, this method specifies the functions to be applied to network outputs. Specifying validation metrics is required to use PEDL's hyperparameter search capabilities.
  • Training metrics (training_metrics(), optional): Similar to validation metrics.

Examples are provided for every interface supported by PEDL. To write your own model definitions, good places to start are (1) documentation for the appropriate framework-specific Trial class and (2) the model definition files in a corresponding provided example.

The model definition typically also contains code for loading data. Each framework supported by PEDL expects data loaders of the type native to that framework; see the Model Definitions documentation for details on each framework.

Congratulations!

You have reached the end of our quick start guide.