Training#

autrainer supports both epoch- and step-based training.

Tip

To create custom training configurations, refer to the custom training configurations quickstart.

Configuring Training#

Training is configured in the main configuration file and comprises the following attributes:

  • iterations: The number of iterations to train the model for.

  • training_type: The type of training, either epoch or step. By default, it is set to epoch.

  • eval_frequency: The frequency in terms of iterations to evaluate the model on the development set. By default, it is set to 1.

  • save_frequency: The frequency in terms of iterations to states of the model, optimizer, and scheduler. By default, it is set to 1.

  • inference_batch_size: The batch size to use during inference. By default, it is set to the training batch size.

The following optional attributes can be set to configure the training process:

  • progress_bar: Whether to display a progress bar during training and evaluation. By default, it is set to True.

  • continue_training: Whether to continue from an already finished run with the same configuration and fewer iterations. By default, it is set to True.

  • remove_continued_runs: Whether to remove the runs that have been continued. By default, it is set to True.

  • save_train_outputs: Whether to save indices, targets, losses, outputs, and predictions (results) on the training set. By default, it is set to True.

  • save_dev_outputs: Whether to save indices, targets, losses, outputs, and predictions (results) on the development set. By default, it is set to True.

  • save_test_outputs: Whether to save indices, targets, losses, outputs, and predictions (results) on the test set. By default, it is set to True.

For brevity, all training attributes with default values are outsourced to the _autrainer_.yaml defaults file and imported in the main configuration file.

Note

Throughout the documentation, the term iteration (as well as iterations, eval_frequency, and save_frequency) refers to a full pass over the training set for epoch-based training, and a single optimization step over a batch of the training set for step-based training.

Trainer#

ModularTaskTrainer manages the training process. It instantiates the model, dataset, criterion, optimizer, scheduler, and callbacks, and trains the model on the dataset. It also logs the training process and saves the model, optimizer, and scheduler states at the end of each epoch.

The cfg of the trainer is the composed main configuration file (e.g., conf/config.yaml) for each training configuration in the sweep.

class autrainer.training.ModularTaskTrainer(cfg, output_directory, experiment_id=None, run_name=None)[source]#

Trainer managing the training of a model given a configuration.

Parameters:

  • cfg (DictConfig) – Run configuration.

  • output_directory (str) – Output directory for the run.

  • experiment_id (Optional[str]) – Experiment ID for the run. If None, the ID is automatically set based on the parent directory of the output directory. Defaults to None.

  • run_name (Optional[str]) – Run name for the run. If None, the name is automatically set based on the output directory. Defaults to None.

train()[source]#

Train the model.

Raises:

ValueError – If the training type is not supported.

Return type:

float

Returns:

The best value of the tracking metric.

On this page