Schedulers

Schedulers are optional and by default not used. This is indicated by the absence of the scheduler attribute in the sweeper configuration (implicitly set to a None configuration file).

In addition to the id, _target_, and scheduler-specific attributes, schedulers can have the following attributes:

• step_frequency (str): The frequency at which the step function is called during training. Possible values are:

  • batch: The step function is called after every batch. Defaults to batch if not specified.

  • evaluation: The step function is called after the number of iterations specified by the eval_frequency of the training configuration.

To use a scheduler, specify it in the configuration file (conf/config.yaml) for the sweeper.

Tip

To create custom schedulers, refer to the custom schedulers tutorial.

Torch Schedulers

Any scheduler from torch.optim.lr_scheduler can be used. A scheduler is specified in the configuration file as a relative import path for the _target_ argument. Any additional arguments (except the id and _target_) are passed as keyword arguments to the scheduler constructor.

For example, the torch.optim.lr_scheduler.StepLR scheduler can be used as follows:

conf/scheduler/StepLR.yaml

id: StepLR
_target_: torch.optim.lr_scheduler.StepLR
step_size: 30

step_frequency: evaluation

Default Configurations

None

This configuration file is used to indicate that no scheduler is used and serves as a no-op placeholder.

conf/scheduler/None.yaml

id: None
_target_: None

StepLR

conf/scheduler/StepLR.yaml

id: StepLR
_target_: torch.optim.lr_scheduler.StepLR
step_size: 30

step_frequency: evaluation