permetrics

permetrics.classification

class permetrics.classification.ClassificationMetric(y_true=None, y_pred=None, decimal=5, **kwargs)

Bases: permetrics.evaluator.Evaluator

This is class contains all classification metrics (for both binary and multiple classification problem)

Notes

• Extension of: https://scikit-learn.org/stable/modules/model_evaluation.html#classification-metrics

LL(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=True)

Log Likelihood (LL): Best possible score is …, the higher value is better. Range = (-inf, +inf)

Note: Computes the log likelihood between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

LL metric

Return type

result (float, int, np.ndarray)

MLL(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Mean Log Likelihood (MLL): Best possible score is …, the higher value is better. Range = (-inf, +inf)

Link: https://github.com/benhamner/Metrics/blob/master/Python/ml_metrics/elementwise.py#L235

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

MLL metric

Return type

result (float, int, np.ndarray)

ll(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=True)

Log Likelihood (LL): Best possible score is …, the higher value is better. Range = (-inf, +inf)

Note: Computes the log likelihood between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

LL metric

Return type

result (float, int, np.ndarray)

mean_log_likelihood(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Mean Log Likelihood (MLL): Best possible score is …, the higher value is better. Range = (-inf, +inf)

Link: https://github.com/benhamner/Metrics/blob/master/Python/ml_metrics/elementwise.py#L235

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

MLL metric

Return type

result (float, int, np.ndarray)

mll(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Mean Log Likelihood (MLL): Best possible score is …, the higher value is better. Range = (-inf, +inf)

Link: https://github.com/benhamner/Metrics/blob/master/Python/ml_metrics/elementwise.py#L235

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

MLL metric

Return type

result (float, int, np.ndarray)

single_log_likelihood(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=True)

Log Likelihood (LL): Best possible score is …, the higher value is better. Range = (-inf, +inf)

Note: Computes the log likelihood between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

LL metric

Return type

result (float, int, np.ndarray)

permetrics.evaluator

class permetrics.evaluator.Evaluator(y_true=None, y_pred=None, decimal=5, **kwargs)

Bases: object

This is base class for all performance metrics

EPSILON = 1e-10

get_multi_output_result(result=None, multi_output=None, decimal=None)

Get multiple output results based on selected parameter

Parameters

• result – The raw result from metric

• multi_output – “raw_values” - return multi-output, [weights] - return single output based on weights, else - return mean result

• decimal (int) – The number of fractional parts after the decimal point

Returns

Multiple outputs results based on selected parameter

Return type

final_result

get_non_zero_data(y_true, y_pred, one_dim=True, rule_idx=0)

Get non-zero data based on rule

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• one_dim (bool) – is y_true has 1 dimensions or not

• rule_idx (int) – valid values [0, 1, 2] corresponding to [y_true, y_pred, both true and pred]

Returns

y_true with positive values based on rule y_pred: y_pred with positive values based on rule

Return type

y_true

get_positive_data(y_true, y_pred, one_dim=True, rule_idx=0)

Get positive data based on rule

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• one_dim (bool) – is y_true has 1 dimensions or not

• rule_idx (int) – valid values [0, 1, 2] corresponding to [y_true, y_pred, both true and pred]

Returns

y_true with positive values based on rule y_pred: y_pred with positive values based on rule

Return type

y_true

get_processed_data(y_true=None, y_pred=None, decimal=None)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• clean (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred)

• decimal (int) – The number of fractional parts after the decimal point

Returns

y_true used in evaluation process. y_pred_final: y_pred used in evaluation process one_dim: is y_true has 1 dimensions or not decimal: The number of fractional parts after the decimal point

Return type

y_true_final

set_keyword_arguments(kwargs)

permetrics.regression

class permetrics.regression.RegressionMetric(y_true=None, y_pred=None, decimal=5, **kwargs)

Bases: permetrics.evaluator.Evaluator

This is class contains all regression metrics (for both regression and time-series problem)

Notes

• An extension of scikit-learn metrics section, besides so many new metrics

• Some methods in scikit-learn can’t generate the multi-output metrics, we re-implement all of them and allow multi-output metrics

• Therefore, this class can calculate the multi-output metrics for all methods

• https://scikit-learn.org/stable/modules/model_evaluation.html#regression-metrics

A10(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A10 index (A10): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• a10-index is engineering index for evaluating artificial intelligence models by showing the number of samples

• that fit the prediction values with a deviation of ±10% compared to experimental values

• https://www.mdpi.com/2076-3417/9/18/3715/htm

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A10 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

A20(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A20 index (A20): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• a20-index evaluated metric by showing the number of samples that fit the prediction values with a deviation of ±20% compared to experimental values

• https://www.mdpi.com/2076-3417/9/18/3715/htm

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A20 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

AE(y_true=None, y_pred=None, decimal=None, non_zero=False, positive=False)

Absolute Error (AE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the absolute error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AE metric

Return type

result (np.ndarray)

CE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Cross Entropy (CE) or Entropy (E): Range = (-inf, 0]. Can’t give comment about this one

Notes

• Greater value of Entropy, the greater the uncertainty for probability distribution and smaller the value the less the uncertainty

• https://datascience.stackexchange.com/questions/20296/cross-entropy-loss-explanation

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

CE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

CI(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Confidence Index (or Performance Index): CI (PI): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• > 0.85, Excellent

• 0.76-0.85, Very good

• 0.66-0.75, Good

• 0.61-0.65, Satisfactory

• 0.51-0.60, Poor

• 0.41-0.50, Bad

• ≤ 0.40, Very bad

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

CI (PI) metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

DRV(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Deviation of Runoff Volume (DRV): Best possible score is 1.0, smaller value is better. Range = [1, +inf) Link: https://rstudio-pubs-static.s3.amazonaws.com/433152_56d00c1e29724829bad5fc4fd8c8ebff.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

DRV metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

EVS(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Explained Variance Score (EVS). Best possible score is 1.0, greater value is better. Range = (-inf, 1.0]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in denominator (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

EVS metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

GINI(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Gini coefficient (Gini): Best possible score is 1, bigger value is better. Range = [0, 1]

Notes

• This version is based on below repository matlab code.

• https://github.com/benhamner/Metrics/blob/master/MATLAB/metrics/gini.m

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

Gini metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

GINI_WIKI(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Gini coefficient (Gini): Best possible score is 1, bigger value is better. Range = [0, 1]

Notes

• This version is based on wiki page, may be is the true version

• https://en.wikipedia.org/wiki/Gini_coefficient

• Gini coefficient can theoretically range from 0 (complete equality) to 1 (complete inequality)

• It is sometimes expressed as a percentage ranging between 0 and 100.

• If negative values are possible, then the Gini coefficient could theoretically be more than 1.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

Gini metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

JSD(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Jensen-Shannon Divergence (JSD): Best possible score is 0.0 (identical), smaller value is better . Range = [0, +inf) Link: https://machinelearningmastery.com/divergence-between-probability-distributions/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

JSD metric (bits) for single column or multiple columns

Return type

result (float, int, np.ndarray)

KGE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Kling-Gupta Efficiency (KGE): Best possible score is 1, bigger value is better. Range = (-inf, 1] Link: https://rstudio-pubs-static.s3.amazonaws.com/433152_56d00c1e29724829bad5fc4fd8c8ebff.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

KGE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

KLD(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Kullback-Leibler Divergence (KLD): Best possible score is 0.0 . Range = (-inf, +inf) Link: https://machinelearningmastery.com/divergence-between-probability-distributions/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

KLD metric (bits) for single column or multiple columns

Return type

result (float, int, np.ndarray)

MAAPE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Arctangent Absolute Percentage Error (MAAPE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://support.numxl.com/hc/en-us/articles/115001223463-MAAPE-Mean-Arctangent-Absolute-Percentage-Error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAAPE metric for single column or multiple columns (radian values)

Return type

result (float, int, np.ndarray)

MAE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Absolute Error (MAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MAPE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Absolute Percentage Error (MAPE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MASE(y_true=None, y_pred=None, m=1, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Absolute Scaled Error (MASE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://en.wikipedia.org/wiki/Mean_absolute_scaled_error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• m (int) – m = 1 for non-seasonal data, m > 1 for seasonal data

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MASE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

ME(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Max Error (ME): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

ME metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MRE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Relative Error (MRE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MRE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MSE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Squared Error (MSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MSLE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Mean Squared Log Error (MSLE): Best possible score is 0.0, smaller value is better. Range = [0, +inf) Link: https://peltarion.com/knowledge-center/documentation/modeling-view/build-an-ai-model/loss-functions/mean-squared-logarithmic-error-(msle)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

MSLE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MedAE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Median Absolute Error (MedAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MedAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

NNSE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Normalize Nash-Sutcliffe Efficiency (NNSE): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Link: https://agrimetsoft.com/calculators/Nash%20Sutcliffe%20model%20Efficiency%20coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

NRMSE(y_true=None, y_pred=None, model=0, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Normalized Root Mean Square Error (NRMSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://medium.com/microsoftazure/how-to-better-evaluate-the-goodness-of-fit-of-regressions-990dbf1c0091

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• model (int) – Normalize RMSE by different ways, (Optional, default = 0, valid values = [0, 1, 2, 3]

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NRMSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

NSE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Nash-Sutcliffe Efficiency (NSE): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Link: https://agrimetsoft.com/calculators/Nash%20Sutcliffe%20model%20Efficiency%20coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

PCC(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Pearson’s Correlation Coefficient (PCC or R): Best possible score is 1.0, bigger value is better. Range = [-1, 1] .. rubric:: Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• Remember no absolute in the equations

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

PCD(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Prediction of Change in Direction (PCD): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

PCD metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

R(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Pearson’s Correlation Coefficient (PCC or R): Best possible score is 1.0, bigger value is better. Range = [-1, 1] .. rubric:: Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• Remember no absolute in the equations

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

R2(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Coefficient of Determination (R2): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• https://scikit-learn.org/stable/modules/model_evaluation.html#r2-score

• This is not R^2 (or R*R), and should be denoted as R2, not like above scikit-learn website.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R2 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

R2s(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

(Pearson’s Correlation Index)^2 = R^2 = R2s (R square): Best possible score is 1.0, bigger value is better. Range = [0, 1] .. rubric:: Notes

• Do not misunderstand between R2s and R2 (Coefficient of Determination), they are different

• Most of online tutorials (article, wikipedia,…) or even scikit-learn library are denoted the wrong R2s and R2.

• R^2 = R2s = R squared should be (Pearson’s Correlation Index)^2

• Meanwhile, R2 = Coefficient of Determination

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R2s metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

RAE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Relative Absolute Error (RAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Notes

• https://stackoverflow.com/questions/59499222/how-to-make-a-function-of-mae-and-rae-without-using-librarymetrics

• https://www.statisticshowto.com/relative-absolute-error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

RE(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=False)

Relative Error (RE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the relative error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RE metric

Return type

result (np.ndarray)

RMSE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Root Mean Squared Error (RMSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RMSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

RSE(y_true=None, y_pred=None, n_paras=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Residual Standard Error (RSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Links:

• https://www.statology.org/residual-standard-error-r/

• https://machinelearningmastery.com/degrees-of-freedom-in-machine-learning/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• n_paras (int) – The number of model’s parameters

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

SE(y_true=None, y_pred=None, decimal=None, non_zero=False, positive=False)

Squared Error (SE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Note: Computes the squared error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

SE metric

Return type

result (np.ndarray)

SLE(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=True)

Squared Log Error (SLE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Note: Computes the squared log error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

SLE metric

Return type

result (np.ndarray)

SMAPE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Symmetric Mean Absolute Percentage Error (SMAPE): Best possible score is 0.0, smaller value is better. Range = [0, 1] If you want percentage then multiply with 100%

Link: https://en.wikipedia.org/wiki/Symmetric_mean_absolute_percentage_error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

SMAPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

VAF(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Variance Accounted For between 2 signals (VAF): Best possible score is 100% (identical signal), bigger value is better. Range = (-inf, 100%]

Link: https://www.dcsc.tudelft.nl/~jwvanwingerden/lti/doc/html/vaf.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

VAF metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

WI(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Willmott Index (WI): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• https://www.researchgate.net/publication/319699360_Reference_evapotranspiration_for_Londrina_Parana_Brazil_performance_of_different_estimation_methods

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

WI metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

a10(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A10 index (A10): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• a10-index is engineering index for evaluating artificial intelligence models by showing the number of samples

• that fit the prediction values with a deviation of ±10% compared to experimental values

• https://www.mdpi.com/2076-3417/9/18/3715/htm

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A10 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

a10_index(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A10 index (A10): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• a10-index is engineering index for evaluating artificial intelligence models by showing the number of samples

• that fit the prediction values with a deviation of ±10% compared to experimental values

• https://www.mdpi.com/2076-3417/9/18/3715/htm

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A10 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

a20(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A20 index (A20): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• a20-index evaluated metric by showing the number of samples that fit the prediction values with a deviation of ±20% compared to experimental values

• https://www.mdpi.com/2076-3417/9/18/3715/htm

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A20 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

a20_index(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A20 index (A20): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• a20-index evaluated metric by showing the number of samples that fit the prediction values with a deviation of ±20% compared to experimental values

• https://www.mdpi.com/2076-3417/9/18/3715/htm

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A20 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

ae(y_true=None, y_pred=None, decimal=None, non_zero=False, positive=False)

Absolute Error (AE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the absolute error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AE metric

Return type

result (np.ndarray)

ce(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Cross Entropy (CE) or Entropy (E): Range = (-inf, 0]. Can’t give comment about this one

Notes

• Greater value of Entropy, the greater the uncertainty for probability distribution and smaller the value the less the uncertainty

• https://datascience.stackexchange.com/questions/20296/cross-entropy-loss-explanation

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

CE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

ci(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Confidence Index (or Performance Index): CI (PI): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• > 0.85, Excellent

• 0.76-0.85, Very good

• 0.66-0.75, Good

• 0.61-0.65, Satisfactory

• 0.51-0.60, Poor

• 0.41-0.50, Bad

• ≤ 0.40, Very bad

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

CI (PI) metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

coefficient_of_determination(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Coefficient of Determination (R2): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• https://scikit-learn.org/stable/modules/model_evaluation.html#r2-score

• This is not R^2 (or R*R), and should be denoted as R2, not like above scikit-learn website.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R2 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

confidence_index(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Confidence Index (or Performance Index): CI (PI): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• > 0.85, Excellent

• 0.76-0.85, Very good

• 0.66-0.75, Good

• 0.61-0.65, Satisfactory

• 0.51-0.60, Poor

• 0.41-0.50, Bad

• ≤ 0.40, Very bad

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

CI (PI) metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

cross_entropy(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Cross Entropy (CE) or Entropy (E): Range = (-inf, 0]. Can’t give comment about this one

Notes

• Greater value of Entropy, the greater the uncertainty for probability distribution and smaller the value the less the uncertainty

• https://datascience.stackexchange.com/questions/20296/cross-entropy-loss-explanation

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

CE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

deviation_of_runoff_volume(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Deviation of Runoff Volume (DRV): Best possible score is 1.0, smaller value is better. Range = [1, +inf) Link: https://rstudio-pubs-static.s3.amazonaws.com/433152_56d00c1e29724829bad5fc4fd8c8ebff.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

DRV metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

drv(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Deviation of Runoff Volume (DRV): Best possible score is 1.0, smaller value is better. Range = [1, +inf) Link: https://rstudio-pubs-static.s3.amazonaws.com/433152_56d00c1e29724829bad5fc4fd8c8ebff.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

DRV metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

evs(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Explained Variance Score (EVS). Best possible score is 1.0, greater value is better. Range = (-inf, 1.0]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in denominator (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

EVS metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

explained_variance_score(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Explained Variance Score (EVS). Best possible score is 1.0, greater value is better. Range = (-inf, 1.0]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in denominator (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

EVS metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

get_metric_by_name(metric_name=<class 'str'>, paras=None) → dict

Get single metric by name, specific parameter of metric by dictionary

Parameters

• metric_name (str) – Select name of metric

• paras (dict) – Dictionary of hyper-parameter for that metric

Returns

{ metric_name: value }

Return type

result (dict)

get_metrics_by_dict(metrics_dict: dict) → dict

Get results of list metrics by its name and parameters wrapped by dictionary

For example:

{“RMSE”: { “multi_output”: multi_output, “decimal”: 4 }, “MAE”: { “non_zero”: True, “multi_output”: multi_output, “decimal”: 6}}

Parameters

metrics_dict (dict) – key is metric name and value is dict of parameters

Returns

e.g, { “RMSE”: 0.3524, “MAE”: 0.445263 }

Return type

results (dict)

get_metrics_by_list_names(list_metric_names=<class 'list'>, list_paras=None) → dict

Get results of list metrics by its name and parameters

Parameters

• list_metric_names (list) – e.g, [“RMSE”, “MAE”, “MAPE”]

• list_paras (list) – e.g, [ {“decimal”: 5, None}, {“decimal”: 4, “multi_output”: “raw_values”}, {“decimal”:6, “multi_output”: [2, 3]} ]

Returns

e.g, { “RMSE”: 0.25, “MAE”: [0.3, 0.6], “MAPE”: 0.15 }

Return type

results (dict)

gini(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Gini coefficient (Gini): Best possible score is 1, bigger value is better. Range = [0, 1]

Notes

• This version is based on below repository matlab code.

• https://github.com/benhamner/Metrics/blob/master/MATLAB/metrics/gini.m

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

Gini metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

gini_coefficient(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Gini coefficient (Gini): Best possible score is 1, bigger value is better. Range = [0, 1]

Notes

• This version is based on below repository matlab code.

• https://github.com/benhamner/Metrics/blob/master/MATLAB/metrics/gini.m

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

Gini metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

gini_coefficient_wiki(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Gini coefficient (Gini): Best possible score is 1, bigger value is better. Range = [0, 1]

Notes

• This version is based on wiki page, may be is the true version

• https://en.wikipedia.org/wiki/Gini_coefficient

• Gini coefficient can theoretically range from 0 (complete equality) to 1 (complete inequality)

• It is sometimes expressed as a percentage ranging between 0 and 100.

• If negative values are possible, then the Gini coefficient could theoretically be more than 1.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

Gini metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

gini_wiki(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Gini coefficient (Gini): Best possible score is 1, bigger value is better. Range = [0, 1]

Notes

• This version is based on wiki page, may be is the true version

• https://en.wikipedia.org/wiki/Gini_coefficient

• Gini coefficient can theoretically range from 0 (complete equality) to 1 (complete inequality)

• It is sometimes expressed as a percentage ranging between 0 and 100.

• If negative values are possible, then the Gini coefficient could theoretically be more than 1.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

Gini metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

jensen_shannon_divergence(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Jensen-Shannon Divergence (JSD): Best possible score is 0.0 (identical), smaller value is better . Range = [0, +inf) Link: https://machinelearningmastery.com/divergence-between-probability-distributions/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

JSD metric (bits) for single column or multiple columns

Return type

result (float, int, np.ndarray)

jsd(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Jensen-Shannon Divergence (JSD): Best possible score is 0.0 (identical), smaller value is better . Range = [0, +inf) Link: https://machinelearningmastery.com/divergence-between-probability-distributions/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

JSD metric (bits) for single column or multiple columns

Return type

result (float, int, np.ndarray)

kge(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Kling-Gupta Efficiency (KGE): Best possible score is 1, bigger value is better. Range = (-inf, 1] Link: https://rstudio-pubs-static.s3.amazonaws.com/433152_56d00c1e29724829bad5fc4fd8c8ebff.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

KGE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

kld(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Kullback-Leibler Divergence (KLD): Best possible score is 0.0 . Range = (-inf, +inf) Link: https://machinelearningmastery.com/divergence-between-probability-distributions/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

KLD metric (bits) for single column or multiple columns

Return type

result (float, int, np.ndarray)

kling_gupta_efficiency(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Kling-Gupta Efficiency (KGE): Best possible score is 1, bigger value is better. Range = (-inf, 1] Link: https://rstudio-pubs-static.s3.amazonaws.com/433152_56d00c1e29724829bad5fc4fd8c8ebff.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

KGE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

kullback_leibler_divergence(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Kullback-Leibler Divergence (KLD): Best possible score is 0.0 . Range = (-inf, +inf) Link: https://machinelearningmastery.com/divergence-between-probability-distributions/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

KLD metric (bits) for single column or multiple columns

Return type

result (float, int, np.ndarray)

maape(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Arctangent Absolute Percentage Error (MAAPE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://support.numxl.com/hc/en-us/articles/115001223463-MAAPE-Mean-Arctangent-Absolute-Percentage-Error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAAPE metric for single column or multiple columns (radian values)

Return type

result (float, int, np.ndarray)

mae(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Absolute Error (MAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mape(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Absolute Percentage Error (MAPE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mase(y_true=None, y_pred=None, m=1, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Absolute Scaled Error (MASE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://en.wikipedia.org/wiki/Mean_absolute_scaled_error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• m (int) – m = 1 for non-seasonal data, m > 1 for seasonal data

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MASE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

max_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Max Error (ME): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

ME metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

me(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Max Error (ME): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

ME metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_absolute_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Absolute Error (MAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_absolute_percentage_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Absolute Percentage Error (MAPE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_absolute_scaled_error(y_true=None, y_pred=None, m=1, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Absolute Scaled Error (MASE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://en.wikipedia.org/wiki/Mean_absolute_scaled_error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• m (int) – m = 1 for non-seasonal data, m > 1 for seasonal data

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MASE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_arctangent_absolute_percentage_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Arctangent Absolute Percentage Error (MAAPE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://support.numxl.com/hc/en-us/articles/115001223463-MAAPE-Mean-Arctangent-Absolute-Percentage-Error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAAPE metric for single column or multiple columns (radian values)

Return type

result (float, int, np.ndarray)

mean_relative_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Relative Error (MRE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MRE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_squared_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Squared Error (MSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_squared_log_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Mean Squared Log Error (MSLE): Best possible score is 0.0, smaller value is better. Range = [0, +inf) Link: https://peltarion.com/knowledge-center/documentation/modeling-view/build-an-ai-model/loss-functions/mean-squared-logarithmic-error-(msle)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

MSLE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

medae(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Median Absolute Error (MedAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MedAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

median_absolute_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Median Absolute Error (MedAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MedAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mre(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Relative Error (MRE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MRE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mse(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Squared Error (MSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

msle(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Mean Squared Log Error (MSLE): Best possible score is 0.0, smaller value is better. Range = [0, +inf) Link: https://peltarion.com/knowledge-center/documentation/modeling-view/build-an-ai-model/loss-functions/mean-squared-logarithmic-error-(msle)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

MSLE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

nash_sutcliffe_efficiency(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Nash-Sutcliffe Efficiency (NSE): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Link: https://agrimetsoft.com/calculators/Nash%20Sutcliffe%20model%20Efficiency%20coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

nnse(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Normalize Nash-Sutcliffe Efficiency (NNSE): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Link: https://agrimetsoft.com/calculators/Nash%20Sutcliffe%20model%20Efficiency%20coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

normalized_nash_sutcliffe_efficiency(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Normalize Nash-Sutcliffe Efficiency (NNSE): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Link: https://agrimetsoft.com/calculators/Nash%20Sutcliffe%20model%20Efficiency%20coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

normalized_root_mean_square_error(y_true=None, y_pred=None, model=0, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Normalized Root Mean Square Error (NRMSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://medium.com/microsoftazure/how-to-better-evaluate-the-goodness-of-fit-of-regressions-990dbf1c0091

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• model (int) – Normalize RMSE by different ways, (Optional, default = 0, valid values = [0, 1, 2, 3]

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NRMSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

nrmse(y_true=None, y_pred=None, model=0, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Normalized Root Mean Square Error (NRMSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://medium.com/microsoftazure/how-to-better-evaluate-the-goodness-of-fit-of-regressions-990dbf1c0091

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• model (int) – Normalize RMSE by different ways, (Optional, default = 0, valid values = [0, 1, 2, 3]

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NRMSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

nse(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Nash-Sutcliffe Efficiency (NSE): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Link: https://agrimetsoft.com/calculators/Nash%20Sutcliffe%20model%20Efficiency%20coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

pcc(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Pearson’s Correlation Coefficient (PCC or R): Best possible score is 1.0, bigger value is better. Range = [-1, 1] .. rubric:: Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• Remember no absolute in the equations

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

pcd(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Prediction of Change in Direction (PCD): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

PCD metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

pearson_correlation_coefficient(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Pearson’s Correlation Coefficient (PCC or R): Best possible score is 1.0, bigger value is better. Range = [-1, 1] .. rubric:: Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• Remember no absolute in the equations

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

pearson_correlation_coefficient_square(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

(Pearson’s Correlation Index)^2 = R^2 = R2s (R square): Best possible score is 1.0, bigger value is better. Range = [0, 1] .. rubric:: Notes

• Do not misunderstand between R2s and R2 (Coefficient of Determination), they are different

• Most of online tutorials (article, wikipedia,…) or even scikit-learn library are denoted the wrong R2s and R2.

• R^2 = R2s = R squared should be (Pearson’s Correlation Index)^2

• Meanwhile, R2 = Coefficient of Determination

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R2s metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

prediction_of_change_in_direction(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Prediction of Change in Direction (PCD): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

PCD metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

r(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Pearson’s Correlation Coefficient (PCC or R): Best possible score is 1.0, bigger value is better. Range = [-1, 1] .. rubric:: Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• Remember no absolute in the equations

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

r2(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Coefficient of Determination (R2): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• https://scikit-learn.org/stable/modules/model_evaluation.html#r2-score

• This is not R^2 (or R*R), and should be denoted as R2, not like above scikit-learn website.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R2 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

r2s(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

(Pearson’s Correlation Index)^2 = R^2 = R2s (R square): Best possible score is 1.0, bigger value is better. Range = [0, 1] .. rubric:: Notes

• Do not misunderstand between R2s and R2 (Coefficient of Determination), they are different

• Most of online tutorials (article, wikipedia,…) or even scikit-learn library are denoted the wrong R2s and R2.

• R^2 = R2s = R squared should be (Pearson’s Correlation Index)^2

• Meanwhile, R2 = Coefficient of Determination

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R2s metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

rae(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Relative Absolute Error (RAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Notes

• https://stackoverflow.com/questions/59499222/how-to-make-a-function-of-mae-and-rae-without-using-librarymetrics

• https://www.statisticshowto.com/relative-absolute-error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

re(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=False)

Relative Error (RE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the relative error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RE metric

Return type

result (np.ndarray)

relative_absolute_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Relative Absolute Error (RAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Notes

• https://stackoverflow.com/questions/59499222/how-to-make-a-function-of-mae-and-rae-without-using-librarymetrics

• https://www.statisticshowto.com/relative-absolute-error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

residual_standard_error(y_true=None, y_pred=None, n_paras=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Residual Standard Error (RSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Links:

• https://www.statology.org/residual-standard-error-r/

• https://machinelearningmastery.com/degrees-of-freedom-in-machine-learning/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• n_paras (int) – The number of model’s parameters

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

rmse(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Root Mean Squared Error (RMSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RMSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

root_mean_squared_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Root Mean Squared Error (RMSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RMSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

rse(y_true=None, y_pred=None, n_paras=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Residual Standard Error (RSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Links:

• https://www.statology.org/residual-standard-error-r/

• https://machinelearningmastery.com/degrees-of-freedom-in-machine-learning/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• n_paras (int) – The number of model’s parameters

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

se(y_true=None, y_pred=None, decimal=None, non_zero=False, positive=False)

Squared Error (SE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Note: Computes the squared error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

SE metric

Return type

result (np.ndarray)

single_absolute_error(y_true=None, y_pred=None, decimal=None, non_zero=False, positive=False)

Absolute Error (AE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the absolute error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AE metric

Return type

result (np.ndarray)

single_relative_error(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=False)

Relative Error (RE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the relative error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RE metric

Return type

result (np.ndarray)

single_squared_error(y_true=None, y_pred=None, decimal=None, non_zero=False, positive=False)

Squared Error (SE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Note: Computes the squared error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

SE metric

Return type

result (np.ndarray)

single_squared_log_error(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=True)

Squared Log Error (SLE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Note: Computes the squared log error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

SLE metric

Return type

result (np.ndarray)

sle(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=True)

Squared Log Error (SLE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Note: Computes the squared log error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

SLE metric

Return type

result (np.ndarray)

smape(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Symmetric Mean Absolute Percentage Error (SMAPE): Best possible score is 0.0, smaller value is better. Range = [0, 1] If you want percentage then multiply with 100%

Link: https://en.wikipedia.org/wiki/Symmetric_mean_absolute_percentage_error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

SMAPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

symmetric_mean_absolute_percentage_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Symmetric Mean Absolute Percentage Error (SMAPE): Best possible score is 0.0, smaller value is better. Range = [0, 1] If you want percentage then multiply with 100%

Link: https://en.wikipedia.org/wiki/Symmetric_mean_absolute_percentage_error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

SMAPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

vaf(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Variance Accounted For between 2 signals (VAF): Best possible score is 100% (identical signal), bigger value is better. Range = (-inf, 100%]

Link: https://www.dcsc.tudelft.nl/~jwvanwingerden/lti/doc/html/vaf.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

VAF metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

variance_accounted_for(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Variance Accounted For between 2 signals (VAF): Best possible score is 100% (identical signal), bigger value is better. Range = (-inf, 100%]

Link: https://www.dcsc.tudelft.nl/~jwvanwingerden/lti/doc/html/vaf.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

VAF metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

wi(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Willmott Index (WI): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• https://www.researchgate.net/publication/319699360_Reference_evapotranspiration_for_Londrina_Parana_Brazil_performance_of_different_estimation_methods

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

WI metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

willmott_index(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Willmott Index (WI): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• https://www.researchgate.net/publication/319699360_Reference_evapotranspiration_for_Londrina_Parana_Brazil_performance_of_different_estimation_methods

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

WI metric for single column or multiple columns

Return type

result (float, int, np.ndarray)