Examples

Example Models

The PEDL distribution contains several example experiments in the examples/ subdirectory. Each experiment consists of a single model definition and one or more experiment configurations.

Framework

Dataset

PyTorch

MNIST

PyTorch

CIFAR-10 CNN

TensorFlow

CIFAR-10 CNN

TensorFlow

MNIST

TensorFlow (Estimator API)

MNIST

TensorFlow (tf.keras)

CIFAR-10 CNN

Custom Plots with Jupyter

The PEDL WebUI includes native support for several plots, but for more complex analysis of experiment metadata, customers can use the plotting functionality provided by Jupyter notebooks. An example notebook is provided in the notebooks example tarball. The example describes how to obtain PEDL experiment metadata in CSV format (by running the pedl CLI) and then demonstrates how to plot several graphs using the Python plotnine library.

TensorBoard Integration

This example shows how to use PEDL with TensorBoard to visualize training and/or validation metrics.

To configure TensorBoard with PEDL, follow these steps:

  1. Set up a directory on a shared file system for TensorBoard event files, e.g. /mnt/tensorboard. All agents must be able to write to this directory.

  2. Add an entry to the experiment config to ensure that the shared directory is mounted into each trial container. In this example, we use /tensorboard as the container path:

bind_mounts:
  - host_path: /mnt/tensorboard
    container_path: /tensorboard
  1. Implement the callbacks() interface in your model definition. Use your container_path (/tensorboard in this example) as the log_directory argument.

from pedl.frameworks.tensorflow import TensorBoard

class MNISTTrial(EstimatorTrial):
   ...
   def callbacks(self, hparams):
       return TensorBoard("/tensorboard")
   ...
  1. Start TensorBoard using the host_path from step 2 to view the experiment metrics as the experiment is running or after it has ended.

tensorboard --logdir=/mnt/tensorboard