IDE Setup (PyCharm)¶
Model developers may develop and debug models with their preferred IDE. In this guide, we show how to configure PyCharm in order to develop and debug experiments submitted from Python, i.e., via this pattern for the Native API or this pattern for the Trial API.
python3.7 -m venv /path/to/env/ source /path/to/env/bin/activate
You will then want to install the
determined Python package, as well
the TensorFlow or PyTorch library versions that your code will run
against when submitting to the cluster. For example, if you are using
Determined’s default task container image
for a PyTorch model, you would install the following dependencies in
pip install determined==<Determined version> pip install torch==<Determined's default image PyTorch version, e.g., 1.4.0> pip install torchvision==<Determined's default image torchvision version, e.g., 0.5.0>
If you are using a custom Docker image for running task containers, you must also install the same dependencies in your local development environment.
Determined’s Native API supports local test execution that does not run in a task container. Therefore, you must install the same task container dependencies in your local development environment in order to reliably run locally.
The following steps show how to locally develop and debug Determined model code. These steps are based on our Native PyTorch MNIST example.
Open the top-level model directory in PyCharm.
In PyCharm Preferences, navigate to your project, then
Click the gear icon next to the interpreter dropdown, then
Virtualenv Environmentselected, click the
Existing environmentradio button, then
...next to the
Navigate to your virtualenv’s interpreter. If you followed the prerequisites above, you will find this at
Validate that all imports in
model_def.pyare recognized (i.e., not underlined in red).
Next create a run configuration in PyCharm. Go to
Edit Configurations..., and select
Python. In the
Script pathfield, select
--local --testto the
Parametersfield and click
You can now set break points in your model definition and debug locally by going to
Debug..., and selecting
trial_impl. For example, to step into the training data loader: