Hume's Likelihood Ratios
A wise man, therefore, proportions his belief to the evidence
We have taken a good look at the Base Rate BR.
Let’s now examine the other component of Bayes Theorem: the Likelihood pair TPR and FPR.
The True Positive Rate TPR – the probability of the evidence if the hypothesis is true – and the False Positive Rate FPR – the probability of the evidence if the hypothesis is false – provide the weight of evidence that drives the update from BR to the posterior probability PP.
Remember: Evidence is confirming if it is more likely when H is true: TPR>FPR, hence PP>BR. It is disconfirming if it is more likely when H is false: TPR<FPR, hence PP<BR. And it is neutral – uninformative, irrelevant – if it as likely when H is true as when H is false: TPR=FPR, hence PP=BR.
The gap between TPR and FPR determines the size of the update. This can be measured in several ways. One of them is the Likelihood Ratio LR=TPR/FPR, where we have:
Confirming evidence: LR>1
Disconfirming evidence: LR<1
Neutral evidence: LR=1.
The dynamics of the updating process is best seen when Bayes Theorem is written in odds form (mathophobes – you can do it if you try):
PO = LR ∙ BO
where PO=PP/(1-PP) are Posterior Odds and BO=BR/(1-BR) are Prior Odds.1
This says that Posterior Odds are a linear function of Prior Odds, with slope LR. The formula helps us to visualise what we have called the evidential tug of war – the iterative accumulation of evidence that drives the search for Truth:
The updating process unfolds through multiplicative accumulation. Starting from initial prior odds BO0, evidence E1 generates Likelihood Ratio LR1, thus producing posterior odds PO1. These then become the new prior odds for the next step: PO1=BO1. Then a second, conditionally independent piece of evidence E2 provides LR2, leading to PO2, which become BO2, and so on.
As we saw at the beginning of our journey, the tug of war can lead to distinct outcomes, which we are now equipped to describe more precisely.
Interestingly, they are closely echoed in the note on Locke which opens Section VI of Hume’s Enquiries Concerning Human Understanding, entitled Of Probability:2
Mr. Locke divides all arguments into demonstrative and probable. In this view, we must say, that it is only probable that all men must die, or that the sun will rise to-morrow. But to conform our language more to common use, we ought to divide arguments into demonstrations, proofs, and probabilities. By proofs meaning such arguments from experience as leave no room for doubt or opposition.
Demonstrations are based on what we have called Faith: a prior certainty in the truth or falsity of a hypothesis. Faith requires no tug of war as no amount of evidence can change it.
Proofs, on the other hand, are entirely based on evidence, and occur when the tug of war has a winner. This can happen in two ways:
1. A single piece of conclusive evidence yields de jure certainty
Multiplicative accumulation implies that even a single piece of conclusive evidence can stop the update and immediately drive Posterior Odds all the way to infinity or to zero. Namely, a Smoking Gun (FPR=0) is sufficient to yield an infinite LR, thus driving PO to infinity (a vertical line) and PP to 1. Conversely, a single Alibi (TPR=0) is all we need to yield a LR of 0, thus collapsing PO to 0 (a horizontal line), and PP to 0. At any stage of the update, the acquisition of conclusive evidence implies that the hypothesis is certainly true or certainly false. A Smoking Gun is enough to prove that the hypothesis “The suspect is guilty” must be true. An Alibi is enough to prove that the hypothesis must be false. Or, to use another famous image, one black swan is sufficient to prove that the hypothesis “All swans are white” must be false. Such is the allure of conclusive evidence: it provides irrefutable, de jure certainty, thus freeing our beliefs from subjective priors.
2. Accumulation of inconclusive evidence yields de facto certainty
Multiplicative accumulation also implies that, if Likelihood Ratios are consistently confirming (LR>1) or disconfirming (LR<1), Posterior Odds tend to infinity or to zero. Hence, posterior probabilities converge towards certainty. While each piece of evidence falls short of proof, they collectively carry overwhelming weight in one direction. PP does not reach 0 or 1 exactly, yet it approximates one or the two extremes, thus allowing us to consider the hypothesis true or false beyond reasonable doubt. We cannot demonstrate that all men must die, or that the sun must rise tomorrow. We can only expect it, based on an overwhelming accumulation of confirming evidence. As they converge to one of the two boundaries of the probability spectrum, posterior probabilities again cease to depend on Base Rates: whatever the initial non-zero priors (remember Cromwell’s Rule), convergence proves that the hypothesis is true or false. This happens to everyone’s satisfaction, leaving no room for doubt or opposition. However, such certainty is not the inescapable consequence of conclusive evidence, but merely the limit of a convergent accumulation of inconclusive evidence. It is virtual, de facto certainty, which again frees us from subjective priors but remains open to refutation.
Finally, Probabilities are the result of a tug of war where evidence remains in the balance, as neither side manages to prevail on the other. Evidence may shift probabilities away from initial priors, but neither conclusively nor overwhelmingly. As competing LR offset one another, the tug of war produces no winner, leaving us with the belief that the hypothesis is probably true and, by complement, also probably false.
Crucially – as we have seen – unlike proofs, probabilities fail to free our beliefs from priors. Hence the illegitimate temptation, driven by our craving for certainty, to escape the burden of priors by falling into Prior indifference.
In fact, notice that, under Prior indifference, we have BO=1, and therefore PO=LR, i.e. PP=LR/(1+LR): confirming evidence=supportive evidence, and neutral evidence=coin toss evidence.
Simple algebra3 gives us the relationship between PP and BR as a function of LR:
Graphically:
The graph is a visual compendium of all the Bayes Theorem properties we have encountered so far:
Faith: BR=0 hence PP=0, BR=1 hence PP=1, regardless of LR.
Cromwell’s Rule: 0<BR<1.
Confirming evidence: LR>1, hence PP>BR. Notice the increasing concavity of the curve as the weight of evidence increases.
Disconfirming evidence: LR<1, hence PP<BR. Notice the increasing convexity of the curve as the weight of evidence increases.
Neutral evidence: LR=1, hence PP=BR. No update.
Conclusive evidence: De jure certainty
Smoking Gun: FPR=0, hence LR=∞. Maximum concavity (coinciding with the vertical axis), yielding PP=1 for all values of BR.
Alibi: TPR=0, hence LR=0. Maximum convexity (coinciding with the horizontal axis), yielding PP=0 for all values of BR.
Converging evidence: De facto certainty
Iterative accumulation of confirming evidence pushing LR towards the vertical axis, or disconfirming evidence pushing LR towards the horizontal axis.
Laplace’s dictum: The smaller BR, the greater the LR required to achieve a given PP.
Prior Indifference: Shifting a small BR to Prior indifference (BR=50%) is a faster but illegitimate way to boost PP in lieu of increasing the weight of evidence.
Amazingly, this is David Hume 15 years before the appearance of Bayes Theorem:
A wise man, therefore, proportions his belief to the evidence. In such conclusions as are founded on an infallible experience, he expects the event with the last degree of assurance, and regards his past experience as a full proof of the future existence of that event. In other cases, he proceeds with more caution: He weighs the opposite experiments: He considers which side is supported by the greater number of experiments: To that side he inclines, with doubt and hesitation; and when at last he fixes his judgment, the evidence exceeds not what we properly call probability. All probability, then, supposes an opposition of experiments and observations, where the one side is found to overbalance the other, and to produce a degree of evidence, proportioned to the superiority. A hundred instances or experiments on one side, and fifty on another, afford a doubtful expectation of any event; though a hundred uniform experiments, with only one that is contradictory, reasonably beget a pretty strong degree of assurance. In all cases, we must balance the opposite experiments, where they are opposite, and deduct the smaller number from the greater, in order to know the exact force of the superior evidence.4
Odds are the ratios between probabilities and their complements. E.g. if P(H)=80%, Odds are 80/20=4. If P(H)=50%, Odds are 50/50=1. If P(H)=20%, Odds are 20/80=0.25, and so on. These are known as Odds in favour. An alternative notation is the reciprocal Odds against, respectively 20/80, 50/50 and 80/20. These are what we mean when we use the term Long Odds for low probabilities.
David Hume, Enquiries Concerning Human Understanding, p. 56
Notice PP=PO/(1+PO) and replace PO.
Enquiries Concerning Human Understanding, Section X, Of Miracles, p. 110