Configure Hyperparameter Ranges

The first step toward automatic hyperparameter tuning is to define the hyperparameter space, e.g., by listing the decisions that may impact model performance. For each hyperparameter in the search space, the machine learning engineer specifies a range of possible values in the experiment configuration:

hyperparameters:
  ...
  dropout_probability:
    type: double
    minval: 0.2
    maxval: 0.5
  ...

Determined supports the following searchable hyperparameter data types:

• int: an integer within a range

• double: a floating point number within a range

• log: a logarithmically scaled floating point number. Users specify a base, and Determined searches the space of exponents within a range.

• categorical: a variable that can take on a value within a specified set of discrete values. The values themselves can be of any type.

The experiment configuration reference details these data types and their associated options.