In this guide, you’ll learn how to use the DeepSpeed API.
Visit the API reference
DeepSpeed is a Microsoft library that supports large-scale, distributed
learning with sharded optimizer state training and pipeline parallelism. Determined supports
DeepSpeed with the
DeepSpeedTrial provides a way to use an automated training
loop with DeepSpeed.
Determined DeepSpeed documentation:
Usage Guide guides you through how to subclass
DeepSpeedTrialfor your own training experiments.
Advanced Usage discusses advanced topics like using multiple model engines, manual gradient aggregation, custom data loaders, and custom model parallelism.
PyTorchTrial to DeepSpeedTrial covers how to convert an existing
DeepSpeed Autotune: User Guide demonstrates how to use DeepSpeed Autotune to take full advantage of your hardware and model.
API Reference lays out the classes and methods related to DeepSpeed support including the full API specification for