Metrics#
Metrics are specified in the dataset configuration using shorthand syntax.
The tracking_metric
attribute specifies the metric to be used for early stopping.
The metrics
attribute in the dataset configuration specifies a list of metrics to be used during training.
Tip
To create custom metrics, refer to the custom metrics tutorial.
For example, a dataset for classification could specify the following metrics:
1id: ExampleDataset
2_target_: example_dataset.ExampleDataset
3...
4
5tracking_metric: autrainer.metrics.Accuracy
6metrics:
7 - autrainer.metrics.Accuracy
8 - autrainer.metrics.UAR
9 - autrainer.metrics.F1
Abstract Metric#
- class autrainer.metrics.AbstractMetric(name, fn, fallback, **fn_kwargs)[source]#
Abstract class for metrics.
- Parameters:
name (
str
) – The name of the metric.fn (
Callable
) – The function to compute the metric.fallback (
float
) – The fallback value if the metric is NaN.**fn_kwargs (
dict
) – Additional keyword arguments to pass to the function.
- __call__(*args, **kwargs)[source]#
Compute the metric.
- Parameters:
*args – Positional arguments for the function.
**kwargs – Keyword arguments for the function.
- Return type:
float
- Returns:
The score.
- abstract property starting_metric: float#
The starting metric value.
- Returns:
The starting metric value.
- abstract property suffix: str#
The suffix of the metric.
- Returns:
The suffix of the metric.
- abstract static get_best(a)[source]#
Get the best metric value from a series of scores.
- Parameters:
a (
Union
[Series
,ndarray
]) – Pandas series or numpy array of scores.- Return type:
float
- Returns:
Best metric value.
- class autrainer.metrics.BaseAscendingMetric(name, fn, fallback=None, **fn_kwargs)[source]#
Base for ascending metrics with higher values being better.
- Parameters:
name (
str
) – The name of the metric.fn (
Callable
) – The function to compute the metric.fallback (
Optional
[float
]) – The fallback value if the metric is NaN. If None, the fallback value is set to -1e32. Defaults to None.**fn_kwargs (
dict
) – Additional keyword arguments to pass to the function.
- property starting_metric: float#
Ascending metric starting value.
- Returns:
-1e32
- property suffix: str#
Ascending metric suffix.
- Returns:
“max”
- class autrainer.metrics.BaseDescendingMetric(name, fn, fallback=None, **fn_kwargs)[source]#
Base for descending metrics with lower values being better.
- Parameters:
name (
str
) – The name of the metric.fn (
Callable
) – The function to compute the metric.fallback (
Optional
[float
]) – The fallback value if the metric is NaN. If None, the fallback value is set to 1e32. Defaults to None.**fn_kwargs (
dict
) – Additional keyword arguments to pass to the function.
- property starting_metric: float#
Descending metric starting value.
- Returns:
1e32
- property suffix: str#
Descending metric suffix.
- Returns:
“min”
Classification Metrics#
Regression Metrics#
- class autrainer.metrics.CCC[source]#
Concordance correlation coefficient metric using audmetric.concordance_cc.