Confusion Matrix

Hook problem: evaluate a spam filter

You run a simple classifier that labels each email as spam or not-spam. You need to know not only how many emails were right, but what kind of errors were made.

This fixed fixture has 12 messages:

First naive idea

One score is the total number of correct predictions:

correct / total.

For these 12 examples, that number is 7 / 12 = 58.3%.

But it hides a critical split:

Where it hurts

e3 and e4 are both wrong, but they do not cost the same:

• e3: not-spam predicted as spam (false alarm),
• e4: spam predicted as not-spam (missed message).

Core invention

Separate the questions:

• Is the truth actual yes or no (relative to one chosen positive class)?
• Is the model output predicted yes or no (for the same positive class)?

That gives four buckets.

Interactive trace

Step through the same 12 examples and watch one cell increment each time.

The fixture is deterministic, so the same email always drives the same step.

Formal version

Let y be the true label and \hat{y} be the predicted label. In this node, the positive class is explicitly declared:

1 = spam, 0 = not spam.

Use rows for actual, columns for predicted:

Here 1 is a chosen positive class flag, not an intrinsic “goodness” label.

TP, FP, TN, and FN are counts in these four cells.

The one-cell invariant is:

n is the number of evaluated examples.

The shared example fixture has:

• TP = 3
• FP = 2
• TN = 4
• FN = 3
• correct = TP + TN = 7
• wrong = FP + FN = 5
• total = 12

Implementation sketch

Each email updates exactly one bucket:

if (actual === positiveLabel && predicted === positiveLabel) {
  tp += 1;
} else if (actual !== positiveLabel && predicted === positiveLabel) {
  fp += 1;
} else if (actual !== positiveLabel && predicted !== positiveLabel) {
  tn += 1;
} else {
  fn += 1;
}

Correctness intuition

The two boolean questions are exhaustive and mutually exclusive.

For a fixed positive label, each example is exactly one of these four combinations:

• actual positive + predicted positive
• actual negative + predicted positive
• actual negative + predicted negative
• actual positive + predicted negative

So each example is assigned to exactly one matrix cell. After 9 emails:

TP = 2, FP = 2, TN = 3, FN = 2, total = 9.

At the end:

TP = 3, FP = 2, TN = 4, FN = 3, total = 12.

You can read this as the invariance check for every trace prefix:

And for the full fixture, processed examples = 12.

Complexity

The scan is one pass over the examples:

Each example contributes one constant-time set of comparisons and one counter increment.

Common confusions

A useful quick exercise is a positive-class swap. Keep the same underlying predictions but set positiveLabel = "not-spam":

• TP = 4
• FP = 3
• TN = 3
• FN = 2

This does not make the dataset better or worse; it only changes what “positive” means.

Connections

This concept is the base for precision, recall, and f1-score. Those nodes are planned but not yet in the graph.

Static trace ledger (no-JS fallback)

The deterministic ledger below is visible without JavaScript and stays aligned with the interactive demo:

Invariant check: {TP + FP + TN + FN} = 12.

Exercises

1. Which cell does e9 reach?
2. Which cell changes if positiveLabel becomes "not-spam"?
3. In this node, if an email is predicted as spam but is actually not-spam, what should happen to precision later?
4. Why is a single wrong-rate number less informative than the four-cell table?

Graph connections : Confusion Matrix