permetrics

permetrics.classification

class permetrics.classification.ClassificationMetric(y_true=None, y_pred=None, decimal=5, **kwargs)

Bases: permetrics.evaluator.Evaluator

This is class contains all classification metrics (for both binary and multiple classification problem)

Notes

• Extension of: https://scikit-learn.org/stable/modules/model_evaluation.html#classification-metrics

AS(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate accuracy score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the accuracy score

Return type

accuracy (float, dict)

CM(y_true=None, y_pred=None, labels=None, normalize=None)

Generate confusion matrix and useful information

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• normalize ('true', 'pred', 'all', None) – Normalizes confusion matrix over the true (rows), predicted (columns) conditions or all the population.

Returns

a 2-dimensional list of pairwise counts imap (dict): a map between label and index of confusion matrix imap_count (dict): a map between label and number of true label in y_true

Return type

matrix (np.ndarray)

F1S(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate f1 score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the f1 score

Return type

f1 (float, dict)

F2S(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate f2 score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the f2 score

Return type

f2 (float, dict)

FBS(y_true=None, y_pred=None, beta=1.0, labels=None, average=None, decimal=None)

The beta parameter determines the weight of recall in the combined score. beta < 1 lends more weight to precision, while beta > 1 favors recall (beta -> 0 considers only precision, beta -> +inf only recall).

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• beta (float) – the weight of recall in the combined score, default = 1.0

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the fbeta score

Return type

fbeta (float, dict)

HL(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate hamming loss for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the hamming loss

Return type

hl (float, dict)

LS(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate lift score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the lift score

Return type

ls (float, dict)

MCC(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• beta (float) – the weight of recall in the combined score, default = 1.0

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the Matthews correlation coefficient

Return type

mcc (float, dict)

NPV(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate negative predictive value for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the negative predictive value

Return type

npv (float, dict)

PS(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate precision score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) –

  {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data:

  'micro':

  Calculate metrics globally by considering each element of the label indicator matrix as a label.

  'macro':

  Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.

  'weighted':

  Calculate metrics for each label, and find their average, weighted by support (the number of true instances for each label).

• decimal (int) – The number of fractional parts after the decimal point

Returns

the precision score

Return type

precision (float, dict)

RS(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate recall score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the recall score

Return type

recall (float, dict)

SS(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate specificity score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the specificity score

Return type

ss (float, dict)

accuracy_score(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate accuracy score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the accuracy score

Return type

accuracy (float, dict)

cm(y_true=None, y_pred=None, labels=None, normalize=None)

Generate confusion matrix and useful information

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• normalize ('true', 'pred', 'all', None) – Normalizes confusion matrix over the true (rows), predicted (columns) conditions or all the population.

Returns

a 2-dimensional list of pairwise counts imap (dict): a map between label and index of confusion matrix imap_count (dict): a map between label and number of true label in y_true

Return type

matrix (np.ndarray)

confusion_matrix(y_true=None, y_pred=None, labels=None, normalize=None)

Generate confusion matrix and useful information

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• normalize ('true', 'pred', 'all', None) – Normalizes confusion matrix over the true (rows), predicted (columns) conditions or all the population.

Returns

a 2-dimensional list of pairwise counts imap (dict): a map between label and index of confusion matrix imap_count (dict): a map between label and number of true label in y_true

Return type

matrix (np.ndarray)

f1_score(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate f1 score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the f1 score

Return type

f1 (float, dict)

f1s(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate f1 score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the f1 score

Return type

f1 (float, dict)

f2_score(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate f2 score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the f2 score

Return type

f2 (float, dict)

f2s(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate f2 score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the f2 score

Return type

f2 (float, dict)

fbeta_score(y_true=None, y_pred=None, beta=1.0, labels=None, average=None, decimal=None)

The beta parameter determines the weight of recall in the combined score. beta < 1 lends more weight to precision, while beta > 1 favors recall (beta -> 0 considers only precision, beta -> +inf only recall).

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• beta (float) – the weight of recall in the combined score, default = 1.0

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the fbeta score

Return type

fbeta (float, dict)

fbs(y_true=None, y_pred=None, beta=1.0, labels=None, average=None, decimal=None)

The beta parameter determines the weight of recall in the combined score. beta < 1 lends more weight to precision, while beta > 1 favors recall (beta -> 0 considers only precision, beta -> +inf only recall).

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• beta (float) – the weight of recall in the combined score, default = 1.0

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the fbeta score

Return type

fbeta (float, dict)

get_processed_data(y_true=None, y_pred=None, decimal=None)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• clean (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred)

• decimal (int, None) – The number of fractional parts after the decimal point

Returns

y_true used in evaluation process. y_pred_final: y_pred used in evaluation process one_dim: is y_true has 1 dimensions or not decimal: The number of fractional parts after the decimal point

Return type

y_true_final

hamming_loss(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate hamming loss for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the hamming loss

Return type

hl (float, dict)

hl(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate hamming loss for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the hamming loss

Return type

hl (float, dict)

lift_score(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate lift score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the lift score

Return type

ls (float, dict)

ls(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate lift score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the lift score

Return type

ls (float, dict)

matthews_correlation_coefficient(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• beta (float) – the weight of recall in the combined score, default = 1.0

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the Matthews correlation coefficient

Return type

mcc (float, dict)

mcc(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• beta (float) – the weight of recall in the combined score, default = 1.0

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the Matthews correlation coefficient

Return type

mcc (float, dict)

negative_predictive_value(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate negative predictive value for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the negative predictive value

Return type

npv (float, dict)

npv(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate negative predictive value for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the negative predictive value

Return type

npv (float, dict)

precision_score(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate precision score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) –

  {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data:

  'micro':

  Calculate metrics globally by considering each element of the label indicator matrix as a label.

  'macro':

  Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.

  'weighted':

  Calculate metrics for each label, and find their average, weighted by support (the number of true instances for each label).

• decimal (int) – The number of fractional parts after the decimal point

Returns

the precision score

Return type

precision (float, dict)

ps(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate precision score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) –

  {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data:

  'micro':

  Calculate metrics globally by considering each element of the label indicator matrix as a label.

  'macro':

  Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.

  'weighted':

  Calculate metrics for each label, and find their average, weighted by support (the number of true instances for each label).

• decimal (int) – The number of fractional parts after the decimal point

Returns

the precision score

Return type

precision (float, dict)

recall_score(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate recall score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the recall score

Return type

recall (float, dict)

rs(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate recall score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the recall score

Return type

recall (float, dict)

specificity_score(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate specificity score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the specificity score

Return type

ss (float, dict)

ss(y_true=None, y_pred=None, labels=None, average=None, decimal=None)

Generate specificity score for multiple classification problem

Parameters

• y_true (tuple, list, np.ndarray) – a list of integers or strings for known classes

• y_pred (tuple, list, np.ndarray) – a list of integers or strings for y_pred classes

• labels (tuple, list, np.ndarray) – List of labels to index the matrix. This may be used to reorder or select a subset of labels.

• average (str, None) – {‘micro’, ‘macro’, ‘weighted’} or None, default=None, others=None

• decimal (int) – The number of fractional parts after the decimal point

Returns

the specificity score

Return type

ss (float, dict)

permetrics.evaluator

class permetrics.evaluator.Evaluator(y_true=None, y_pred=None, decimal=5, **kwargs)

Bases: object

This is base class for all performance metrics

EPSILON = 1e-10

get_metric_by_name(metric_name=<class 'str'>, paras=None) → dict

Get single metric by name, specific parameter of metric by dictionary

Parameters

• metric_name (str) – Select name of metric

• paras (dict) – Dictionary of hyper-parameter for that metric

Returns

{ metric_name: value }

Return type

result (dict)

get_metrics_by_dict(metrics_dict: dict) → dict

Get results of list metrics by its name and parameters wrapped by dictionary

For example:

{“RMSE”: { “multi_output”: multi_output, “decimal”: 4 }, “MAE”: { “non_zero”: True, “multi_output”: multi_output, “decimal”: 6}}

Parameters

metrics_dict (dict) – key is metric name and value is dict of parameters

Returns

e.g, { “RMSE”: 0.3524, “MAE”: 0.445263 }

Return type

results (dict)

get_metrics_by_list_names(list_metric_names=<class 'list'>, list_paras=None) → dict

Get results of list metrics by its name and parameters

Parameters

• list_metric_names (list) – e.g, [“RMSE”, “MAE”, “MAPE”]

• list_paras (list) – e.g, [ {“decimal”: 5, None}, {“decimal”: 4, “multi_output”: “raw_values”}, {“decimal”:6, “multi_output”: [2, 3]} ]

Returns

e.g, { “RMSE”: 0.25, “MAE”: [0.3, 0.6], “MAPE”: 0.15 }

Return type

results (dict)

get_multi_output_result(result=None, multi_output=None, decimal=None)

Get multiple output results based on selected parameter

Parameters

• result – The raw result from metric

• multi_output – “raw_values” - return multi-output, [weights] - return single output based on weights, else - return mean result

• decimal (int) – The number of fractional parts after the decimal point

Returns

Multiple outputs results based on selected parameter

Return type

final_result

get_processed_data(y_true=None, y_pred=None, decimal=None)

set_keyword_arguments(kwargs)

permetrics.regression

class permetrics.regression.RegressionMetric(y_true=None, y_pred=None, decimal=5, **kwargs)

Bases: permetrics.evaluator.Evaluator

This is class contains all regression metrics (for both regression and time-series problem)

Notes

• An extension of scikit-learn metrics section, besides so many new metrics

• Some methods in scikit-learn can’t generate the multi-output metrics, we re-implement all of them and allow multi-output metrics

• Therefore, this class can calculate the multi-output metrics for all methods

• https://scikit-learn.org/stable/modules/model_evaluation.html#regression-metrics

A10(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A10 index (A10): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• a10-index is engineering index for evaluating artificial intelligence models by showing the number of samples

• that fit the prediction values with a deviation of ±10% compared to experimental values

• https://www.mdpi.com/2076-3417/9/18/3715/htm

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A10 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

A20(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A20 index (A20): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• a20-index evaluated metric by showing the number of samples that fit the prediction values with a deviation of ±20% compared to experimental values

• https://www.mdpi.com/2076-3417/9/18/3715/htm

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A20 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

A30(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A30 index (A30): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Note: a30-index evaluated metric by showing the number of samples that fit the prediction values with a deviation of ±30% compared to experimental values

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A30 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

ACOD(y_true=None, y_pred=None, X_shape=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Adjusted Coefficient of Determination (ACOD/AR2): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• https://dziganto.github.io/data%20science/linear%20regression/machine%20learning/python/Linear-Regression-101-Metrics/

• Scikit-learn and other websites denoted COD as R^2 (or R squared), it leads to the misunderstanding of R^2 in which R is PCC.

• We should denote it as COD or R2 only.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• X_shape (tuple, list, np.ndarray) – The shape of X_train dataset

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AR2 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

AE(y_true=None, y_pred=None, decimal=None, non_zero=False, positive=False)

Absolute Error (AE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the absolute error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AE metric

Return type

result (np.ndarray)

APCC(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Absolute Pearson’s Correlation Coefficient (APCC or AR): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AR metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

AR(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Absolute Pearson’s Correlation Coefficient (APCC or AR): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AR metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

AR2(y_true=None, y_pred=None, X_shape=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Adjusted Coefficient of Determination (ACOD/AR2): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• https://dziganto.github.io/data%20science/linear%20regression/machine%20learning/python/Linear-Regression-101-Metrics/

• Scikit-learn and other websites denoted COD as R^2 (or R squared), it leads to the misunderstanding of R^2 in which R is PCC.

• We should denote it as COD or R2 only.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• X_shape (tuple, list, np.ndarray) – The shape of X_train dataset

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AR2 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

CE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Cross Entropy (CE) or Entropy (E): Range = (-inf, 0]. Can’t give comment about this one

Notes

• Greater value of Entropy, the greater the uncertainty for probability distribution and smaller the value the less the uncertainty

• https://datascience.stackexchange.com/questions/20296/cross-entropy-loss-explanation

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

CE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

CI(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Confidence Index (or Performance Index): CI (PI): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• > 0.85, Excellent

• 0.76-0.85, Very good

• 0.66-0.75, Good

• 0.61-0.65, Satisfactory

• 0.51-0.60, Poor

• 0.41-0.50, Bad

• ≤ 0.40, Very bad

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

CI (PI) metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

COD(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Coefficient of Determination (COD/R2): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• https://scikit-learn.org/stable/modules/model_evaluation.html#r2-score

• Scikit-learn and other websites denoted COD as R^2 (or R squared), it leads to the misunderstanding of R^2 in which R is PCC.

• We should denote it as COD or R2 only.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R2 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

DRV(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Deviation of Runoff Volume (DRV): Best possible score is 1.0, smaller value is better. Range = [1, +inf) Link: https://rstudio-pubs-static.s3.amazonaws.com/433152_56d00c1e29724829bad5fc4fd8c8ebff.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

DRV metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

EVS(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Explained Variance Score (EVS). Best possible score is 1.0, greater value is better. Range = (-inf, 1.0]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in denominator (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

EVS metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

GINI(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Gini coefficient (Gini): Best possible score is 1, bigger value is better. Range = [0, 1]

Notes

• This version is based on below repository matlab code.

• https://github.com/benhamner/Metrics/blob/master/MATLAB/metrics/gini.m

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

Gini metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

GINI_WIKI(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Gini coefficient (Gini): Best possible score is 1, bigger value is better. Range = [0, 1]

Notes

• This version is based on wiki page, may be is the true version

• https://en.wikipedia.org/wiki/Gini_coefficient

• Gini coefficient can theoretically range from 0 (complete equality) to 1 (complete inequality)

• It is sometimes expressed as a percentage ranging between 0 and 100.

• If negative values are possible, then the Gini coefficient could theoretically be more than 1.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

Gini metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

JSD(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Jensen-Shannon Divergence (JSD): Best possible score is 0.0 (identical), smaller value is better . Range = [0, +inf) Link: https://machinelearningmastery.com/divergence-between-probability-distributions/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

JSD metric (bits) for single column or multiple columns

Return type

result (float, int, np.ndarray)

KGE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Kling-Gupta Efficiency (KGE): Best possible score is 1, bigger value is better. Range = (-inf, 1] Link: https://rstudio-pubs-static.s3.amazonaws.com/433152_56d00c1e29724829bad5fc4fd8c8ebff.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

KGE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

KLD(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Kullback-Leibler Divergence (KLD): Best possible score is 0.0 . Range = (-inf, +inf) Link: https://machinelearningmastery.com/divergence-between-probability-distributions/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

KLD metric (bits) for single column or multiple columns

Return type

result (float, int, np.ndarray)

MAAPE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Arctangent Absolute Percentage Error (MAAPE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://support.numxl.com/hc/en-us/articles/115001223463-MAAPE-Mean-Arctangent-Absolute-Percentage-Error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAAPE metric for single column or multiple columns (radian values)

Return type

result (float, int, np.ndarray)

MAE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Absolute Error (MAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MAPE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Absolute Percentage Error (MAPE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MASE(y_true=None, y_pred=None, m=1, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Absolute Scaled Error (MASE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://en.wikipedia.org/wiki/Mean_absolute_scaled_error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• m (int) – m = 1 for non-seasonal data, m > 1 for seasonal data

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MASE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MBE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Bias Error (MBE): Best possible score is 0.0. Range = (-inf, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MBE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

ME(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Max Error (ME): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

ME metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MPE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Percentage Error (MPE): Best possible score is 0.0. Range = (-inf, +inf) Link: https://www.dataquest.io/blog/understanding-regression-error-metrics/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MRB(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Relative Error (MRE) - Mean Relative Bias (MRB): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MRE (MRB) metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MRE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Relative Error (MRE) - Mean Relative Bias (MRB): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MRE (MRB) metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MSE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Squared Error (MSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MSLE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Mean Squared Log Error (MSLE): Best possible score is 0.0, smaller value is better. Range = [0, +inf) Link: https://peltarion.com/knowledge-center/documentation/modeling-view/build-an-ai-model/loss-functions/mean-squared-logarithmic-error-(msle)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

MSLE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

MedAE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Median Absolute Error (MedAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MedAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

NNSE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Normalize Nash-Sutcliffe Efficiency (NNSE): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Link: https://agrimetsoft.com/calculators/Nash%20Sutcliffe%20model%20Efficiency%20coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

NRMSE(y_true=None, y_pred=None, model=0, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Normalized Root Mean Square Error (NRMSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://medium.com/microsoftazure/how-to-better-evaluate-the-goodness-of-fit-of-regressions-990dbf1c0091

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• model (int) – Normalize RMSE by different ways, (Optional, default = 0, valid values = [0, 1, 2, 3]

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NRMSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

NSE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Nash-Sutcliffe Efficiency (NSE): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Link: https://agrimetsoft.com/calculators/Nash%20Sutcliffe%20model%20Efficiency%20coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

PCC(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Pearson’s Correlation Coefficient (PCC or R): Best possible score is 1.0, bigger value is better. Range = [-1, 1] .. rubric:: Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• Remember no absolute in the equations

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

PCD(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Prediction of Change in Direction (PCD): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

PCD metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

R(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Pearson’s Correlation Coefficient (PCC or R): Best possible score is 1.0, bigger value is better. Range = [-1, 1] .. rubric:: Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• Remember no absolute in the equations

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

R2(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Coefficient of Determination (COD/R2): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• https://scikit-learn.org/stable/modules/model_evaluation.html#r2-score

• Scikit-learn and other websites denoted COD as R^2 (or R squared), it leads to the misunderstanding of R^2 in which R is PCC.

• We should denote it as COD or R2 only.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R2 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

R2s(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

(Pearson’s Correlation Index)^2 = R^2 = R2s (R square): Best possible score is 1.0, bigger value is better. Range = [0, 1] .. rubric:: Notes

• Do not misunderstand between R2s and R2 (Coefficient of Determination), they are different

• Most of online tutorials (article, wikipedia,…) or even scikit-learn library are denoted the wrong R2s and R2.

• R^2 = R2s = R squared should be (Pearson’s Correlation Index)^2

• Meanwhile, R2 = Coefficient of Determination

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R2s metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

RAE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Relative Absolute Error (RAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Notes

• https://stackoverflow.com/questions/59499222/how-to-make-a-function-of-mae-and-rae-without-using-librarymetrics

• https://www.statisticshowto.com/relative-absolute-error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

RB(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=False)

Relative Error (RE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the relative error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RE metric

Return type

result (np.ndarray)

RE(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=False)

Relative Error (RE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the relative error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RE metric

Return type

result (np.ndarray)

RMSE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Root Mean Squared Error (RMSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RMSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

RSE(y_true=None, y_pred=None, n_paras=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Residual Standard Error (RSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Links:

• https://www.statology.org/residual-standard-error-r/

• https://machinelearningmastery.com/degrees-of-freedom-in-machine-learning/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• n_paras (int) – The number of model’s parameters

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

SE(y_true=None, y_pred=None, decimal=None, non_zero=False, positive=False)

Squared Error (SE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Note: Computes the squared error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

SE metric

Return type

result (np.ndarray)

SLE(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=True)

Squared Log Error (SLE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Note: Computes the squared log error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

SLE metric

Return type

result (np.ndarray)

SMAPE(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Symmetric Mean Absolute Percentage Error (SMAPE): Best possible score is 0.0, smaller value is better. Range = [0, 1] If you want percentage then multiply with 100%

Link: https://en.wikipedia.org/wiki/Symmetric_mean_absolute_percentage_error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

SMAPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

VAF(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Variance Accounted For between 2 signals (VAF): Best possible score is 100% (identical signal), bigger value is better. Range = (-inf, 100%]

Link: https://www.dcsc.tudelft.nl/~jwvanwingerden/lti/doc/html/vaf.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

VAF metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

WI(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Willmott Index (WI): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• https://www.researchgate.net/publication/319699360_Reference_evapotranspiration_for_Londrina_Parana_Brazil_performance_of_different_estimation_methods

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

WI metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

a10(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A10 index (A10): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• a10-index is engineering index for evaluating artificial intelligence models by showing the number of samples

• that fit the prediction values with a deviation of ±10% compared to experimental values

• https://www.mdpi.com/2076-3417/9/18/3715/htm

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A10 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

a10_index(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A10 index (A10): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• a10-index is engineering index for evaluating artificial intelligence models by showing the number of samples

• that fit the prediction values with a deviation of ±10% compared to experimental values

• https://www.mdpi.com/2076-3417/9/18/3715/htm

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A10 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

a20(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A20 index (A20): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• a20-index evaluated metric by showing the number of samples that fit the prediction values with a deviation of ±20% compared to experimental values

• https://www.mdpi.com/2076-3417/9/18/3715/htm

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A20 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

a20_index(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A20 index (A20): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• a20-index evaluated metric by showing the number of samples that fit the prediction values with a deviation of ±20% compared to experimental values

• https://www.mdpi.com/2076-3417/9/18/3715/htm

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A20 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

a30(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A30 index (A30): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Note: a30-index evaluated metric by showing the number of samples that fit the prediction values with a deviation of ±30% compared to experimental values

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A30 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

a30_index(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

A30 index (A30): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Note: a30-index evaluated metric by showing the number of samples that fit the prediction values with a deviation of ±30% compared to experimental values

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

A30 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

absolute_pearson_correlation_coefficient(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Absolute Pearson’s Correlation Coefficient (APCC or AR): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AR metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

acod(y_true=None, y_pred=None, X_shape=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Adjusted Coefficient of Determination (ACOD/AR2): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• https://dziganto.github.io/data%20science/linear%20regression/machine%20learning/python/Linear-Regression-101-Metrics/

• Scikit-learn and other websites denoted COD as R^2 (or R squared), it leads to the misunderstanding of R^2 in which R is PCC.

• We should denote it as COD or R2 only.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• X_shape (tuple, list, np.ndarray) – The shape of X_train dataset

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AR2 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

adjusted_coefficient_of_determination(y_true=None, y_pred=None, X_shape=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Adjusted Coefficient of Determination (ACOD/AR2): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• https://dziganto.github.io/data%20science/linear%20regression/machine%20learning/python/Linear-Regression-101-Metrics/

• Scikit-learn and other websites denoted COD as R^2 (or R squared), it leads to the misunderstanding of R^2 in which R is PCC.

• We should denote it as COD or R2 only.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• X_shape (tuple, list, np.ndarray) – The shape of X_train dataset

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AR2 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

ae(y_true=None, y_pred=None, decimal=None, non_zero=False, positive=False)

Absolute Error (AE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the absolute error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AE metric

Return type

result (np.ndarray)

apcc(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Absolute Pearson’s Correlation Coefficient (APCC or AR): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AR metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

ar(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Absolute Pearson’s Correlation Coefficient (APCC or AR): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AR metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

ar2(y_true=None, y_pred=None, X_shape=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Adjusted Coefficient of Determination (ACOD/AR2): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• https://dziganto.github.io/data%20science/linear%20regression/machine%20learning/python/Linear-Regression-101-Metrics/

• Scikit-learn and other websites denoted COD as R^2 (or R squared), it leads to the misunderstanding of R^2 in which R is PCC.

• We should denote it as COD or R2 only.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• X_shape (tuple, list, np.ndarray) – The shape of X_train dataset

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AR2 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

ce(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Cross Entropy (CE) or Entropy (E): Range = (-inf, 0]. Can’t give comment about this one

Notes

• Greater value of Entropy, the greater the uncertainty for probability distribution and smaller the value the less the uncertainty

• https://datascience.stackexchange.com/questions/20296/cross-entropy-loss-explanation

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

CE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

ci(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Confidence Index (or Performance Index): CI (PI): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• > 0.85, Excellent

• 0.76-0.85, Very good

• 0.66-0.75, Good

• 0.61-0.65, Satisfactory

• 0.51-0.60, Poor

• 0.41-0.50, Bad

• ≤ 0.40, Very bad

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

CI (PI) metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

cod(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Coefficient of Determination (COD/R2): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• https://scikit-learn.org/stable/modules/model_evaluation.html#r2-score

• Scikit-learn and other websites denoted COD as R^2 (or R squared), it leads to the misunderstanding of R^2 in which R is PCC.

• We should denote it as COD or R2 only.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R2 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

coefficient_of_determination(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Coefficient of Determination (COD/R2): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• https://scikit-learn.org/stable/modules/model_evaluation.html#r2-score

• Scikit-learn and other websites denoted COD as R^2 (or R squared), it leads to the misunderstanding of R^2 in which R is PCC.

• We should denote it as COD or R2 only.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R2 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

confidence_index(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Confidence Index (or Performance Index): CI (PI): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• > 0.85, Excellent

• 0.76-0.85, Very good

• 0.66-0.75, Good

• 0.61-0.65, Satisfactory

• 0.51-0.60, Poor

• 0.41-0.50, Bad

• ≤ 0.40, Very bad

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

CI (PI) metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

cross_entropy(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Cross Entropy (CE) or Entropy (E): Range = (-inf, 0]. Can’t give comment about this one

Notes

• Greater value of Entropy, the greater the uncertainty for probability distribution and smaller the value the less the uncertainty

• https://datascience.stackexchange.com/questions/20296/cross-entropy-loss-explanation

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

CE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

deviation_of_runoff_volume(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Deviation of Runoff Volume (DRV): Best possible score is 1.0, smaller value is better. Range = [1, +inf) Link: https://rstudio-pubs-static.s3.amazonaws.com/433152_56d00c1e29724829bad5fc4fd8c8ebff.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

DRV metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

drv(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Deviation of Runoff Volume (DRV): Best possible score is 1.0, smaller value is better. Range = [1, +inf) Link: https://rstudio-pubs-static.s3.amazonaws.com/433152_56d00c1e29724829bad5fc4fd8c8ebff.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

DRV metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

evs(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Explained Variance Score (EVS). Best possible score is 1.0, greater value is better. Range = (-inf, 1.0]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in denominator (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

EVS metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

explained_variance_score(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Explained Variance Score (EVS). Best possible score is 1.0, greater value is better. Range = (-inf, 1.0]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in denominator (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

EVS metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

get_processed_data(y_true=None, y_pred=None, decimal=None)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• clean (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred)

• decimal (int) – The number of fractional parts after the decimal point

Returns

y_true used in evaluation process. y_pred_final: y_pred used in evaluation process one_dim: is y_true has 1 dimensions or not decimal: The number of fractional parts after the decimal point

Return type

y_true_final

gini(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Gini coefficient (Gini): Best possible score is 1, bigger value is better. Range = [0, 1]

Notes

• This version is based on below repository matlab code.

• https://github.com/benhamner/Metrics/blob/master/MATLAB/metrics/gini.m

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

Gini metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

gini_coefficient(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Gini coefficient (Gini): Best possible score is 1, bigger value is better. Range = [0, 1]

Notes

• This version is based on below repository matlab code.

• https://github.com/benhamner/Metrics/blob/master/MATLAB/metrics/gini.m

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

Gini metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

gini_coefficient_wiki(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Gini coefficient (Gini): Best possible score is 1, bigger value is better. Range = [0, 1]

Notes

• This version is based on wiki page, may be is the true version

• https://en.wikipedia.org/wiki/Gini_coefficient

• Gini coefficient can theoretically range from 0 (complete equality) to 1 (complete inequality)

• It is sometimes expressed as a percentage ranging between 0 and 100.

• If negative values are possible, then the Gini coefficient could theoretically be more than 1.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

Gini metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

gini_wiki(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Gini coefficient (Gini): Best possible score is 1, bigger value is better. Range = [0, 1]

Notes

• This version is based on wiki page, may be is the true version

• https://en.wikipedia.org/wiki/Gini_coefficient

• Gini coefficient can theoretically range from 0 (complete equality) to 1 (complete inequality)

• It is sometimes expressed as a percentage ranging between 0 and 100.

• If negative values are possible, then the Gini coefficient could theoretically be more than 1.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

Gini metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

jensen_shannon_divergence(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Jensen-Shannon Divergence (JSD): Best possible score is 0.0 (identical), smaller value is better . Range = [0, +inf) Link: https://machinelearningmastery.com/divergence-between-probability-distributions/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

JSD metric (bits) for single column or multiple columns

Return type

result (float, int, np.ndarray)

jsd(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Jensen-Shannon Divergence (JSD): Best possible score is 0.0 (identical), smaller value is better . Range = [0, +inf) Link: https://machinelearningmastery.com/divergence-between-probability-distributions/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

JSD metric (bits) for single column or multiple columns

Return type

result (float, int, np.ndarray)

kge(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Kling-Gupta Efficiency (KGE): Best possible score is 1, bigger value is better. Range = (-inf, 1] Link: https://rstudio-pubs-static.s3.amazonaws.com/433152_56d00c1e29724829bad5fc4fd8c8ebff.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

KGE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

kld(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Kullback-Leibler Divergence (KLD): Best possible score is 0.0 . Range = (-inf, +inf) Link: https://machinelearningmastery.com/divergence-between-probability-distributions/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

KLD metric (bits) for single column or multiple columns

Return type

result (float, int, np.ndarray)

kling_gupta_efficiency(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Kling-Gupta Efficiency (KGE): Best possible score is 1, bigger value is better. Range = (-inf, 1] Link: https://rstudio-pubs-static.s3.amazonaws.com/433152_56d00c1e29724829bad5fc4fd8c8ebff.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

KGE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

kullback_leibler_divergence(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Kullback-Leibler Divergence (KLD): Best possible score is 0.0 . Range = (-inf, +inf) Link: https://machinelearningmastery.com/divergence-between-probability-distributions/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

KLD metric (bits) for single column or multiple columns

Return type

result (float, int, np.ndarray)

maape(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Arctangent Absolute Percentage Error (MAAPE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://support.numxl.com/hc/en-us/articles/115001223463-MAAPE-Mean-Arctangent-Absolute-Percentage-Error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAAPE metric for single column or multiple columns (radian values)

Return type

result (float, int, np.ndarray)

mae(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Absolute Error (MAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mape(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Absolute Percentage Error (MAPE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mase(y_true=None, y_pred=None, m=1, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Absolute Scaled Error (MASE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://en.wikipedia.org/wiki/Mean_absolute_scaled_error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• m (int) – m = 1 for non-seasonal data, m > 1 for seasonal data

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MASE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

max_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Max Error (ME): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

ME metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mbe(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Bias Error (MBE): Best possible score is 0.0. Range = (-inf, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MBE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

me(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Max Error (ME): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

ME metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_absolute_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Absolute Error (MAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_absolute_percentage_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Absolute Percentage Error (MAPE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_absolute_scaled_error(y_true=None, y_pred=None, m=1, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Absolute Scaled Error (MASE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://en.wikipedia.org/wiki/Mean_absolute_scaled_error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• m (int) – m = 1 for non-seasonal data, m > 1 for seasonal data

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MASE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_arctangent_absolute_percentage_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Arctangent Absolute Percentage Error (MAAPE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://support.numxl.com/hc/en-us/articles/115001223463-MAAPE-Mean-Arctangent-Absolute-Percentage-Error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MAAPE metric for single column or multiple columns (radian values)

Return type

result (float, int, np.ndarray)

mean_bias_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Bias Error (MBE): Best possible score is 0.0. Range = (-inf, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MBE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_percentage_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Percentage Error (MPE): Best possible score is 0.0. Range = (-inf, +inf) Link: https://www.dataquest.io/blog/understanding-regression-error-metrics/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_relative_bias(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Relative Error (MRE) - Mean Relative Bias (MRB): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MRE (MRB) metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_relative_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Relative Error (MRE) - Mean Relative Bias (MRB): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MRE (MRB) metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_squared_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Squared Error (MSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mean_squared_log_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Mean Squared Log Error (MSLE): Best possible score is 0.0, smaller value is better. Range = [0, +inf) Link: https://peltarion.com/knowledge-center/documentation/modeling-view/build-an-ai-model/loss-functions/mean-squared-logarithmic-error-(msle)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

MSLE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

medae(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Median Absolute Error (MedAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MedAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

median_absolute_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Median Absolute Error (MedAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MedAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mpe(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Percentage Error (MPE): Best possible score is 0.0. Range = (-inf, +inf) Link: https://www.dataquest.io/blog/understanding-regression-error-metrics/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mrb(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Relative Error (MRE) - Mean Relative Bias (MRB): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MRE (MRB) metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mre(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=False)

Mean Relative Error (MRE) - Mean Relative Bias (MRB): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MRE (MRB) metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

mse(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Mean Squared Error (MSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

MSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

msle(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=True, positive=True)

Mean Squared Log Error (MSLE): Best possible score is 0.0, smaller value is better. Range = [0, +inf) Link: https://peltarion.com/knowledge-center/documentation/modeling-view/build-an-ai-model/loss-functions/mean-squared-logarithmic-error-(msle)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

MSLE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

nash_sutcliffe_efficiency(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Nash-Sutcliffe Efficiency (NSE): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Link: https://agrimetsoft.com/calculators/Nash%20Sutcliffe%20model%20Efficiency%20coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

nnse(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Normalize Nash-Sutcliffe Efficiency (NNSE): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Link: https://agrimetsoft.com/calculators/Nash%20Sutcliffe%20model%20Efficiency%20coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

normalized_nash_sutcliffe_efficiency(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Normalize Nash-Sutcliffe Efficiency (NNSE): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Link: https://agrimetsoft.com/calculators/Nash%20Sutcliffe%20model%20Efficiency%20coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

normalized_root_mean_square_error(y_true=None, y_pred=None, model=0, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Normalized Root Mean Square Error (NRMSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://medium.com/microsoftazure/how-to-better-evaluate-the-goodness-of-fit-of-regressions-990dbf1c0091

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• model (int) – Normalize RMSE by different ways, (Optional, default = 0, valid values = [0, 1, 2, 3]

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NRMSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

nrmse(y_true=None, y_pred=None, model=0, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Normalized Root Mean Square Error (NRMSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Link: https://medium.com/microsoftazure/how-to-better-evaluate-the-goodness-of-fit-of-regressions-990dbf1c0091

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• model (int) – Normalize RMSE by different ways, (Optional, default = 0, valid values = [0, 1, 2, 3]

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NRMSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

nse(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Nash-Sutcliffe Efficiency (NSE): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Link: https://agrimetsoft.com/calculators/Nash%20Sutcliffe%20model%20Efficiency%20coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

NSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

pcc(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Pearson’s Correlation Coefficient (PCC or R): Best possible score is 1.0, bigger value is better. Range = [-1, 1] .. rubric:: Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• Remember no absolute in the equations

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

pcd(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Prediction of Change in Direction (PCD): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

PCD metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

pearson_correlation_coefficient(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Pearson’s Correlation Coefficient (PCC or R): Best possible score is 1.0, bigger value is better. Range = [-1, 1] .. rubric:: Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• Remember no absolute in the equations

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

pearson_correlation_coefficient_square(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

(Pearson’s Correlation Index)^2 = R^2 = R2s (R square): Best possible score is 1.0, bigger value is better. Range = [0, 1] .. rubric:: Notes

• Do not misunderstand between R2s and R2 (Coefficient of Determination), they are different

• Most of online tutorials (article, wikipedia,…) or even scikit-learn library are denoted the wrong R2s and R2.

• R^2 = R2s = R squared should be (Pearson’s Correlation Index)^2

• Meanwhile, R2 = Coefficient of Determination

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R2s metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

prediction_of_change_in_direction(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Prediction of Change in Direction (PCD): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

PCD metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

r(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Pearson’s Correlation Coefficient (PCC or R): Best possible score is 1.0, bigger value is better. Range = [-1, 1] .. rubric:: Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• Remember no absolute in the equations

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

r2(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Coefficient of Determination (COD/R2): Best possible score is 1.0, bigger value is better. Range = (-inf, 1]

Notes

• https://scikit-learn.org/stable/modules/model_evaluation.html#r2-score

• Scikit-learn and other websites denoted COD as R^2 (or R squared), it leads to the misunderstanding of R^2 in which R is PCC.

• We should denote it as COD or R2 only.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R2 metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

r2s(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

(Pearson’s Correlation Index)^2 = R^2 = R2s (R square): Best possible score is 1.0, bigger value is better. Range = [0, 1] .. rubric:: Notes

• Do not misunderstand between R2s and R2 (Coefficient of Determination), they are different

• Most of online tutorials (article, wikipedia,…) or even scikit-learn library are denoted the wrong R2s and R2.

• R^2 = R2s = R squared should be (Pearson’s Correlation Index)^2

• Meanwhile, R2 = Coefficient of Determination

• https://en.wikipedia.org/wiki/Pearson_correlation_coefficient

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

R2s metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

rae(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Relative Absolute Error (RAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Notes

• https://stackoverflow.com/questions/59499222/how-to-make-a-function-of-mae-and-rae-without-using-librarymetrics

• https://www.statisticshowto.com/relative-absolute-error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

rb(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=False)

Relative Error (RE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the relative error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RE metric

Return type

result (np.ndarray)

re(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=False)

Relative Error (RE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the relative error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RE metric

Return type

result (np.ndarray)

relative_absolute_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Relative Absolute Error (RAE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Notes

• https://stackoverflow.com/questions/59499222/how-to-make-a-function-of-mae-and-rae-without-using-librarymetrics

• https://www.statisticshowto.com/relative-absolute-error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RAE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

residual_standard_error(y_true=None, y_pred=None, n_paras=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Residual Standard Error (RSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Links:

• https://www.statology.org/residual-standard-error-r/

• https://machinelearningmastery.com/degrees-of-freedom-in-machine-learning/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• n_paras (int) – The number of model’s parameters

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

rmse(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Root Mean Squared Error (RMSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RMSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

root_mean_squared_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Root Mean Squared Error (RMSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RMSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

rse(y_true=None, y_pred=None, n_paras=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Residual Standard Error (RSE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Links:

• https://www.statology.org/residual-standard-error-r/

• https://machinelearningmastery.com/degrees-of-freedom-in-machine-learning/

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• n_paras (int) – The number of model’s parameters

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RSE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

se(y_true=None, y_pred=None, decimal=None, non_zero=False, positive=False)

Squared Error (SE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Note: Computes the squared error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

SE metric

Return type

result (np.ndarray)

single_absolute_error(y_true=None, y_pred=None, decimal=None, non_zero=False, positive=False)

Absolute Error (AE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the absolute error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

AE metric

Return type

result (np.ndarray)

single_relative_bias(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=False)

Relative Error (RE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the relative error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RE metric

Return type

result (np.ndarray)

single_relative_error(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=False)

Relative Error (RE): Best possible score is 0.0, smaller value is better. Range = (-inf, +inf)

Note: Computes the relative error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

RE metric

Return type

result (np.ndarray)

single_squared_error(y_true=None, y_pred=None, decimal=None, non_zero=False, positive=False)

Squared Error (SE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Note: Computes the squared error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

SE metric

Return type

result (np.ndarray)

single_squared_log_error(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=True)

Squared Log Error (SLE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Note: Computes the squared log error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

SLE metric

Return type

result (np.ndarray)

sle(y_true=None, y_pred=None, decimal=None, non_zero=True, positive=True)

Squared Log Error (SLE): Best possible score is 0.0, smaller value is better. Range = [0, +inf)

Note: Computes the squared log error between two numbers, or for element between a pair of list, tuple or numpy arrays.

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = True)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = True)

Returns

SLE metric

Return type

result (np.ndarray)

smape(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Symmetric Mean Absolute Percentage Error (SMAPE): Best possible score is 0.0, smaller value is better. Range = [0, 1] If you want percentage then multiply with 100%

Link: https://en.wikipedia.org/wiki/Symmetric_mean_absolute_percentage_error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

SMAPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

symmetric_mean_absolute_percentage_error(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Symmetric Mean Absolute Percentage Error (SMAPE): Best possible score is 0.0, smaller value is better. Range = [0, 1] If you want percentage then multiply with 100%

Link: https://en.wikipedia.org/wiki/Symmetric_mean_absolute_percentage_error

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

SMAPE metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

vaf(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Variance Accounted For between 2 signals (VAF): Best possible score is 100% (identical signal), bigger value is better. Range = (-inf, 100%]

Link: https://www.dcsc.tudelft.nl/~jwvanwingerden/lti/doc/html/vaf.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

VAF metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

variance_accounted_for(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Variance Accounted For between 2 signals (VAF): Best possible score is 100% (identical signal), bigger value is better. Range = (-inf, 100%]

Link: https://www.dcsc.tudelft.nl/~jwvanwingerden/lti/doc/html/vaf.html

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

VAF metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

wi(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Willmott Index (WI): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• https://www.researchgate.net/publication/319699360_Reference_evapotranspiration_for_Londrina_Parana_Brazil_performance_of_different_estimation_methods

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

WI metric for single column or multiple columns

Return type

result (float, int, np.ndarray)

willmott_index(y_true=None, y_pred=None, multi_output='raw_values', decimal=None, non_zero=False, positive=False)

Willmott Index (WI): Best possible score is 1.0, bigger value is better. Range = [0, 1]

Notes

• Reference evapotranspiration for Londrina, Paraná, Brazil: performance of different estimation methods

• https://www.researchgate.net/publication/319699360_Reference_evapotranspiration_for_Londrina_Parana_Brazil_performance_of_different_estimation_methods

Parameters

• y_true (tuple, list, np.ndarray) – The ground truth values

• y_pred (tuple, list, np.ndarray) – The prediction values

• multi_output – Can be “raw_values” or list weights of variables such as [0.5, 0.2, 0.3] for 3 columns, (Optional, default = “raw_values”)

• decimal (int) – The number of fractional parts after the decimal point (Optional, default = 5)

• non_zero (bool) – Remove all rows contain 0 value in y_pred (some methods have denominator is y_pred) (Optional, default = False)

• positive (bool) – Calculate metric based on positive values only or not (Optional, default = False)

Returns

WI metric for single column or multiple columns

Return type

result (float, int, np.ndarray)