Understanding Confusion Matrices

A confusion matrix is a table that compares a classification model's predicted labels with the actual labels. For a yes/no problem it holds four counts: true positives (TP), false negatives (FN), false positives (FP) and true negatives (TN). Accuracy, precision, recall and F1 score are all calculated from those four numbers, and the off-diagonal cells show exactly which mistakes the model makes.

How do you read a confusion matrix?

Each row is an actual class and each column is a predicted class, so correct predictions sit on the diagonal (TP and TN) and every off-diagonal cell is an error: a false negative is a positive the model missed, and a false positive is a negative it flagged by mistake. Some textbooks and tools transpose this layout, so check the axis labels before reading off FP and FN.

                Predicted
                Positive  Negative
Actual Positive    TP        FN
       Negative    FP        TN

TP = True Positives (correctly predicted positive)
FN = False Negatives (missed positives)
FP = False Positives (false alarms)
TN = True Negatives (correctly predicted negative)

How do you create a confusion matrix in Python?

Pass the actual labels and then the predicted labels to scikit-learn's confusion_matrix(y_true, y_pred). It sorts the class labels, so with 0/1 labels the negative class comes first and the result is laid out as [[TN, FP], [FN, TP]], with true positives in the bottom-right rather than the top-left of the table above.

from sklearn.metrics import confusion_matrix

# Example: email spam classifier (1 = spam, 0 = not spam)
y_true = [1, 0, 1, 1, 0, 1, 0, 1, 1, 0]  # Actual labels
y_pred = [1, 0, 1, 0, 0, 1, 1, 1, 1, 0]  # Predicted labels

cm = confusion_matrix(y_true, y_pred)
print(cm)

# Output:
# [[3 1]
#  [1 5]]
# Row 1 (actual not spam): TN=3, FP=1
# Row 2 (actual spam):     FN=1, TP=5

tn, fp, fn, tp = cm.ravel()

How do you plot a confusion matrix?

Draw it as a heatmap: seaborn's heatmap() with annot=True and fmt='d' colors each cell and prints its count, which makes the largest error cells easy to spot. scikit-learn's ConfusionMatrixDisplay.from_predictions(y_true, y_pred) draws a similar chart in one line.

import seaborn as sns
import matplotlib.pyplot as plt

# Create heatmap (tick labels follow the sorted labels: 0, then 1)
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',
            xticklabels=['Not Spam', 'Spam'],
            yticklabels=['Not Spam', 'Spam'])
plt.ylabel('Actual')
plt.xlabel('Predicted')
plt.title('Spam Classifier Confusion Matrix')
plt.show()

Key Metrics from Confusion Matrix

Accuracy is the share of all predictions that are correct, precision is the share of predicted positives that are real, recall is the share of real positives the model catches, and F1 is the harmonic mean of precision and recall. The examples below use the spam matrix above (TP = 5, TN = 3, FP = 1, FN = 1); for a deeper comparison of the two most confused metrics, see precision vs recall.

Accuracy

TP, TN, FP, FN = 5, 3, 1, 1  # from the spam example

# Overall correct predictions
accuracy = (TP + TN) / (TP + TN + FP + FN)
print(f"Accuracy: {accuracy:.2%}")  # Accuracy: 80.00%

Precision (Positive Predictive Value)

# Of predictions labeled positive, how many were correct?
precision = TP / (TP + FP)
print(f"Precision: {precision:.2%}")  # Precision: 83.33%

# High precision = few false alarms

Recall (Sensitivity, True Positive Rate)

# Of actual positives, how many did we catch?
recall = TP / (TP + FN)
print(f"Recall: {recall:.2%}")  # Recall: 83.33%

# High recall = catch most positives

F1 Score (Harmonic Mean)

# Balance between precision and recall
f1 = 2 * (precision * recall) / (precision + recall)
print(f"F1 Score: {f1:.2%}")  # F1 Score: 83.33%

Specificity (True Negative Rate)

# Of actual negatives, how many did we correctly leave alone?
specificity = TN / (TN + FP)
print(f"Specificity: {specificity:.2%}")  # Specificity: 75.00%

# False positive rate = 1 - specificity (the x-axis of a ROC curve)

Using Sklearn for All Metrics

classification_report() prints precision, recall, F1 and support (the number of actual examples) for every class at once, plus accuracy and the macro and weighted averages. Each class row treats that class as the positive one, so the "Not Spam" row's precision is TN / (TN + FN) = 3 / 4.

from sklearn.metrics import classification_report

y_true = [1, 0, 1, 1, 0, 1, 0, 1, 1, 0]
y_pred = [1, 0, 1, 0, 0, 1, 1, 1, 1, 0]

print(classification_report(y_true, y_pred,
                          target_names=['Not Spam', 'Spam']))

# Output:
#               precision    recall  f1-score   support
#
#     Not Spam       0.75      0.75      0.75         4
#         Spam       0.83      0.83      0.83         6
#
#     accuracy                           0.80        10
#    macro avg       0.79      0.79      0.79        10
# weighted avg       0.80      0.80      0.80        10

Multi-Class Confusion Matrix

With more than two classes the matrix becomes N × N: the cell in row i, column j counts examples of actual class i that were predicted as class j. The diagonal still holds the correct predictions, and the largest off-diagonal cells show which pairs of classes the model confuses.

# Example: iris flower classifier (3 classes)
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier

iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
    iris.data, iris.target, test_size=0.3, random_state=42
)

model = RandomForestClassifier(random_state=42)
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

cm = confusion_matrix(y_test, y_pred)
print(cm)

# Visualize 3x3 matrix
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',
            xticklabels=iris.target_names,
            yticklabels=iris.target_names)
plt.title('Iris Classification')
plt.show()

Precision-Recall Tradeoff

Every decision threshold produces a different confusion matrix. Most classifiers output a probability and predict() effectively cuts it at 0.5 for two classes; lowering the threshold labels more cases positive, which raises recall and usually lowers precision, and raising it does the reverse.

from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import precision_recall_curve

# Binary problem: precision_recall_curve needs 0/1 labels
X, y = make_classification(n_samples=1000, weights=[0.9], random_state=42)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=42, stratify=y
)
model = LogisticRegression().fit(X_train, y_train)

# Probability of the positive class
y_scores = model.predict_proba(X_test)[:, 1]

# Same model, two thresholds, two confusion matrices
for threshold in (0.5, 0.3):
    y_pred = (y_scores >= threshold).astype(int)
    print(threshold, confusion_matrix(y_test, y_pred).ravel())  # tn, fp, fn, tp

# Precision and recall at every threshold
precision, recall, thresholds = precision_recall_curve(y_test, y_scores)

plt.plot(recall, precision)
plt.xlabel('Recall')
plt.ylabel('Precision')
plt.title('Precision-Recall Curve')
plt.show()

# Lower threshold → higher recall, usually lower precision
# Higher threshold → lower recall, usually higher precision

When to Optimize for What

Optimize for recall when missing a positive is the costly mistake, and for precision when a false alarm is. When both errors matter, especially with imbalanced classes, track F1 or choose the threshold from the real cost of each error.

Scenario	Optimize For
Medical diagnosis	Recall (don't miss sick patients)
Spam filter	Precision (don't block important emails)
Fraud detection	Both (F1 score)
Recommend products	Precision (only relevant suggestions)

Interpreting Confusion Matrix Errors

A false positive is a Type I error (the model raises an alarm that should not have been raised) and a false negative is a Type II error (the model misses a real positive). Which one costs more decides whether you tune for precision or recall; Type I vs Type II errors covers the same idea in hypothesis testing.

False Positives (Type I Error)

# Predicted positive, actually negative
# Example: Flagging legitimate email as spam
# Cost: User misses important email

False Negatives (Type II Error)

# Predicted negative, actually positive
# Example: Missing spam, letting it through
# Cost: User sees spam in inbox

Complete Example: Customer Churn

In this churn model (1 = customer churns), the matrix shows 6 churners caught, 2 missed and 1 loyal customer wrongly flagged, which works out to 80% accuracy, 85.7% precision and 75% recall.

from sklearn.metrics import confusion_matrix

# Predictions
y_true = [1, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1]
y_pred = [1, 0, 0, 1, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1]

cm = confusion_matrix(y_true, y_pred)

# Extract values
tn, fp, fn, tp = cm.ravel()

print(f"True Negatives: {tn}")
print(f"False Positives: {fp}")
print(f"False Negatives: {fn}")
print(f"True Positives: {tp}")

# Calculate metrics
accuracy = (tp + tn) / (tp + tn + fp + fn)
precision = tp / (tp + fp)
recall = tp / (tp + fn)
f1 = 2 * precision * recall / (precision + recall)

print(f"\nAccuracy: {accuracy:.1%}")
print(f"Precision: {precision:.1%}")
print(f"Recall: {recall:.1%}")
print(f"F1 Score: {f1:.1%}")

# Output:
# True Negatives: 6
# False Positives: 1
# False Negatives: 2
# True Positives: 6
#
# Accuracy: 80.0%
# Precision: 85.7%
# Recall: 75.0%
# F1 Score: 80.0%

Why is accuracy misleading on imbalanced data?

When one class is rare, a model can reach high accuracy by always predicting the common class, and the confusion matrix exposes this immediately. With 1,000 card transactions of which 10 are fraud, a model that never flags anything scores 99% accuracy while catching none of the fraud.

Model	TP	FN	FP	TN	Accuracy	Precision	Recall	F1
Always "not fraud"	0	10	0	990	99.0%	undefined	0%	0%
Real model	8	2	40	950	95.8%	16.7%	80.0%	27.6%

The do-nothing model wins on accuracy but catches no fraud at all, while the real model catches 8 of 10 frauds at the price of 40 false alarms. That is why precision, recall and F1 are the numbers to report when classes are imbalanced. (scikit-learn reports the undefined precision as 0 and raises an UndefinedMetricWarning.)

What are the most common confusion matrix mistakes?

The most common mistake is reading the matrix the wrong way round, which silently swaps false positives with false negatives and precision with recall.

• Swapping the arguments: confusion_matrix(y_pred, y_true) returns the transpose, so FP and FN trade places and precision and recall swap. The actual labels always go first.
• Assuming TP is top-left: scikit-learn sorts the labels, so for 0/1 data TP is bottom-right. Pass labels=[1, 0] to put the positive class first, or unpack with cm.ravel().
• Comparing raw counts across test sets of different sizes: confusion_matrix(y_true, y_pred, normalize='true') divides each row by its total, so the diagonal reads as per-class recall. For the spam example, rounded to two decimals, that gives [[0.75 0.25] [0.17 0.83]].
• Judging the model at a single threshold: one matrix describes one cutoff. Look at the ROC curve and AUC or the precision-recall curve across all thresholds before you pick one.
• Building the matrix from training data: training-set matrices look far better than the model will perform. Use a held-out test set or cross-validation.