Basics
Condensed to its core, the confusion matrix cross-tabulates two binary dimensions and classifies each individual case into one of 4 possible categories that result from combining the two binary variables (e.g., the condition and decision of each case) with each other. This may still sound complicated, but looks like this:
| Condition | ||
|---|---|---|
| Decision | present (TRUE): |
absent (FALSE): |
positive (TRUE): |
hi |
fa |
negative (FALSE): |
mi |
cr |
Fortunately, this is not so confusing any more. And, perhaps surprisingly, all other metrics follow from this simple core in a straightforward way. In the following, we illustrate how the other metrics can be constructed from the 4 essential frequencies in the core of the matrix.
Adopting 2 perspectives on a population
Essentially, the confusion matrix views a population of N individuals in different ways by adopting different perspectives. “Adopting a perspective” means that we can distinguish between individuals on the basis of some criterion. The 2 primary criteria used here are:
- each individual’s condition, which can either be present (
TRUE) or absent (FALSE), and
- each individual’s decision, which can either be
positive(TRUE) ornegative(FALSE).
Numerically, the adoption of each of these two perspectives splits the population into two subgroups.1 Applying two different splits of a population into two subgroups results in \(2 \times 2 = 4\) cases, which form the core of the confusion matrix:
hirepresents hits (or true positives): condition present (TRUE) & decision positive (TRUE).mirepresents misses (or false negatives): condition present (TRUE) & decision negative (FALSE).farepresents false alarms (or false positives): condition absent (FALSE) & decision positive (TRUE).crrepresents correct rejections (or true negatives): condition absent (FALSE) & decision negative (FALSE).
Importantly, all frequencies required to understand and compute various metrics are combinations of these four frequencies — which is why we refer to them as the four essential frequencies (see the vignette on Data formats). For instance, adding up the columns and rows of the matrix yields the frequencies of the two subgroups that result from adopting our two perspectives on the population N (or splitting N into subgroups by applying two binary criteria):
- by condition (corresponding to the two columns of the confusion matrix):
\[ \begin{aligned} \texttt{N} \ &= \ \texttt{cond_true} & +\ \ \ \ \ &\texttt{cond_false} & \textrm{(a)} \\ \ &= \ (\texttt{hi} + \texttt{mi}) & +\ \ \ \ \ &(\texttt{fa} + \texttt{cr}) \\ \end{aligned} \]
- by decision (corresponding to the two rows of the confusion matrix):
\[ \begin{aligned} \texttt{N} \ &= \ \texttt{dec_pos} & +\ \ \ \ \ &\texttt{dec_neg} & \ \ \ \ \textrm{(b)} \\ \ &= \ (\texttt{hi} + \texttt{fa}) & +\ \ \ \ \ &(\texttt{mi} + \texttt{cr}) \\ \end{aligned} \]
To reflect these two perspectives in the confusion matrix, we only need to add the sums of columns (i.e., by condition) and rows (by decision):
| Condition | ||||
|---|---|---|---|---|
| Decision | present (TRUE): |
absent (FALSE): |
Sum: | |
positive (TRUE): |
hi |
fa |
dec_pos |
|
negative (FALSE): |
mi |
cr |
dec_neg |
|
| Sum: | cond_true |
cond_false |
N |
Example
To view a 2x2 confusion table in riskyr, use the plot_tab function or plot an existing scenario as type = "tab":
# Plot table from basic input parameters: -----
plot_tab(prev = .05, sens = .75, spec = .66, N = 1000,
p_lbl = "def", title_lbl = "Scenario 1")
## Plot an existing riskyr scenario: -----
# s <- scenarios$n1
# plot(s, type = "tab", p_lbl = "def")
Example of a 2x2 confusion table in riskyr.
Accuracy as a 3rd perspective
A 3rd way of grouping the four essential frequencies results from asking the question: Which of the four essential frequencies are correct decisions and which are erroneous decisions? Crucially, this question about decision accuracy can neither be answered by only considering each individual’s condition (i.e., the columns of the matrix), nor can it be answered by only considering each individual’s decision (i.e., the rows of the matrix). Instead, answering the question about accuracy requires that the other dimensions have been determined and then considering the correspondence between condition and decision. Checking the correspondence between rows and columns for the four essential frequencies yields an important insight: The confusion matrix contains two types of correct decisions and two types of errors:
A decision is correct, when it corresponds to the condition. This is the case for two cells in (or the “" diagonal of) the confusion matrix:
hi: condition present (TRUE) & decision positive (TRUE)cr: condition absent (FALSE) & decision negative (FALSE)
A decision is incorrect or erroneous, when it does not correspond to the condition. This also is the case for two cells in (or the “/” diagonal of) the confusion matrix:
mi: condition present (TRUE) & decision negative (FALSE)fa: condition absent (FALSE) & decision positive (TRUE)
Splitting all N individuals into two subgroups of those with correct vs. those with erroneous decisions yields a third perspective on the population:
- by the correspondence of decisions to conditions (corresponding to the two diagonals of the confusion matrix):
\[ \begin{aligned} \texttt{N} \ &= \ \texttt{dec_cor} & +\ \ \ \ \ &\texttt{dec_err} & \ \ \textrm{(c)} \\ \ &= \ (\texttt{hi} + \texttt{cr}) & +\ \ \ \ \ &(\texttt{mi} + \texttt{fa}) \\ \end{aligned} \]
Example
Re-arranging the cells of the 2x2 confusion table allows illustrating accuracy as a 3rd perspective (e.g., by specifying the perspective by = "cdac"):
plot_tab(prev = .05, sens = .75, spec = .66, N = 1000,
by = "cdac", p_split = "h",
p_lbl = "def", title_lbl = "Scenario 2")
Arranging a 2x2 confusion table by condition and by accuracy.
Avoiding common sources of confusion
It may be instructive to point out two possible sources of confusion, so that they can be deliberately avoided:
Beware of alternative terms for
miandcr:Misses
miare often called “false negatives” (FN), but are nevertheless cases for which the condition isTRUE(i.e., in thecond_truecolumn of the confusion table).Correct rejections
crare often called “true negatives” (TN), but are nevertheless cases for which the condition isFALSE(i.e., in thecond_falsecolumn of the confusion table).
Thus, the terms “true” and “false” are sometimes ambiguous by switching their referents. When used to denote the four essential frequencies (e.g., describing mi as “false negatives” and cr as “true negatives”) the terms refer to the correspondence of a decision to the condition, rather than to their condition. To avoid this source of confusion, we prefer the terms mi and cr over “false negatives” (FN) and “true negatives” (TN), respectively, but offer both options as pre-defined lists of text labels (see txt_org and txt_TF).
- Beware of alternative terms for
dec_coranddec_err:
Similarly, it may be tempting to refer to instances ofdec_coranddec_erras “true decisions” and “false decisions”. However, this would also invite conceptual confusion, as “true decisions” would includecond_falsecases (cror TN cases) and “false decisions” would includecond_truecases (mior FN cases). Again, we prefer the less ambiguous terms “correct decisions” vs. “erroneous decisions”.
