Validation strategy metrics module

timecave.validation_strategy_metrics

This module contains several metrics to evaluate the performance of model validation methods.

Functions:

Name	Description
PAE	Implements the Predictive Accuracy Error metric.
APAE	Implements the Absolute Predictive Accuracy Error metric.
RPAE	Implements the Relative Predictive Accuracy Error metric.
RAPAE	Implements the Relative Absolute Predictive Accuracy Error metric.
sMPAE	Implements the symmetric Mean Predictive Accuracy Error metric.
MC_metric	Statistical summary for Monte Carlo experiments regarding validation methods.
under_over_estimation	Separate statistical summaries for the underestimation and overestimation cases.

Notes

• PAE and APAE are absolute metrics. Their values may range from \(-L_m\) to \(\infty\) and from \(0\) to \(\infty\), respectively. These should not be used to compare results obtained with different models or using different time series.
• RPAE and RAPAE are relative metrics, as they measure how large the validation error is with respect to the true (test) error, thus eliminating the latter's influence on the metric. Their values lie in the \([-1, \infty]\) and \([0, \infty]\) intervals, respectively. These can be used to compare results for different models and/or time series.
• sMPAE is a scaled, symmetric version of the PAE. It can be used to compare results for different models and/or time series.