Hugging Face Transformers#

The Hugging Face Transformers Library is the de facto library for natural language processing (NLP) models. It provides pretrained weights for leading NLP models and lets you easily use these pretrained models for the most common NLP tasks, such as language modeling, text classification, and question answering.

model-hub makes it easy to train transformer models in Determined while keeping the developer experience as close as possible to working directly with transformers. The Determined library serves as an alternative to the Hugging Face Trainer Class and provides access to the benefits of using Determined, including:

Note

To learn more about distributed training with Determined, visit the conceptual overview or the intro to implementing distributed training.

Model Hub Transformers is similar to the no_trainer version of transformers examples in that you have more control over the training and evaluation routines if you want.

Given the above benefits, this library can be particularly useful if any of the following apply:

  • You are an Determined user that wants to get started quickly with transformers.

  • You are a transformers user that wants to easily run more advanced workflows like multi-node distributed training and advanced hyperparameter search.

  • You are a transformers user looking for a single platform to manage experiments, handle checkpoints with automated fault tolerance, and perform hyperparameter search/visualization.

Limitations#

The following Hugging Face transformers features are currently not supported:

  • TensorFlow version of transformers

  • Support for fairscale

  • Running on TPUs