DeepSpeed API

In this guide, you’ll learn how to use the DeepSpeed API.

Visit the API reference
det.pytorch.deepspeed API Reference

DeepSpeed is a Microsoft library that supports large-scale, distributed learning with sharded optimizer state training and pipeline parallelism. Determined supports DeepSpeed with the DeepSpeedTrial API. DeepSpeedTrial provides a way to use an automated training loop with DeepSpeed.

Determined DeepSpeed documentation:

• Usage Guide guides you through how to subclass DeepSpeedTrial for your own training experiments.

• Advanced Usage discusses advanced topics like using multiple model engines, manual gradient aggregation, custom data loaders, and custom model parallelism.

• PyTorchTrial to DeepSpeedTrial covers how to convert an existing PyTorchTrial to DeepSpeedTrial.

• DeepSpeed Autotune: User Guide demonstrates how to use DeepSpeed Autotune to take full advantage of your hardware and model.

• API Reference lays out the classes and methods related to DeepSpeed support including the full API specification for DeepSpeedTrial and DeepSpeedTrialContext.