Model Evaluation¶
This module provides functionality to evaluate the performance of a machine learning model. Each method compares the predicted data to true data.
Evaluate Accuracy¶
- evaluation_metrics.evaluate_accuracy(predicted_output, true_output)[source]¶
Calculate the proportion of labels a classifier correctly predicts.
- Parameters
predicted_output (numpy.ndarray) – The predictions made by the classifier.
true_output (int) – The true labels.
- Returns
accuracy – Proportion correctly predicted, between 0 and 1 (inclusive).
- Return type
float
Evaluate Confusion Matrix¶
- evaluation_metrics.confusion_matrix(number_labels, predicted_output, true_output)[source]¶
Calculate contingency table of predicted labels and true labels.
If there are L labels, then for 0 <= i, j <= L - 1, the (i, j) entry contains the number of times we predicted j when the true class is i.
- Parameters
number_labels (int) – The number of possible labels in the data.
predicted_output (numpy.ndarray) – The predictions made by the classifier.
true_output (int) – The true labels .
- Returns
confusion_matrix – Square [confusion] matrix of size number_labels by number_labels.
- Return type
numpy.ndarray
Notes
The true output and predicted output row vectors are stacked over each other. This 2 by sample_size numpy array is stored as output_combined. To identify how often we predict j when the truth is i, we count the number of times the column vector [i, j] appears in output_combined. I consulted 1 to figure out the lattermost step.
References
Evaluate Regression Error¶
- evaluation_metrics.evaluate_regression_error(predicted_output, true_output, norm=<function euclidean_2>)[source]¶
Calculate the error with respect to a norm of regression output.
- Parameters
predicted_output (numpy.ndarray) – The predictions made by the classifier.
true_output (int) – The true response values.
norm (func) – The choice of norm to use to measure error. Default is the Euclidean L_2 norm.
- Returns
error – Measurement of error of regression model. (squared norm)
- Return type
float