We start with P
Today we begin the journey that will lead us to Bayes Theorem
Today we begin the journey that will lead us to Bayes Theorem.
We start with P.
P stands for Probability. We all know what that means. When we say that something is probable, we mean that it is possible, it may happen, it may be true. P measures the degree to which that is the case. It ranges from 0 to 1. P=0 (or 0%) means that the thing is impossible, it cannot happen, it can’t be true. P=1 (or 100%) means that it is certain, it must certainly happen, it is certainly true. In the middle between the two extremes, P=0.5 (or 50%) means that the thing is equally probable, as likely to happen or not happen, as likely to be true or false.
Along the scale from 0 to 1, we use a variety of terms to describe our perceptions of different degrees of probability. Building on a CIA memo written in 1964, the following graph (available here) lists many of them:
We are all familiar with these expressions. Yet, as the graph – known as a joyplot – shows, we do not attach the same numerical value to them. When we say, for instance, that something is “almost certainly” true, we clearly have a high probability in mind. But whether that means 80% or 90%, 99%, or some other value in the upper range, varies from person to person and depends on context and circumstances. This is entirely normal in ordinary language, where words – especially those referring to abstract concepts – exist within a halo of vagueness. What is a chair? What is love? What is green?
What is probability?
If you studied Probability Theory in any form, you have a ready answer: probability is the relative frequency with which something happens or is true in a large number of identical, repeatable trials. For example, if we toss a fair coin many times under identical conditions, the fraction of Heads converges to 0.5. Probability Theory started in the 17th century as a study of hard evidence, resulting from controlled, replicable experiments, leading to the measurement of undisputable objective probabilities, grounded on empirical frequencies.1
But this is not the kind of evidence we will encounter in Bayesland. So please take your coins, dice and card decks and put them aside.
Most of the evidence we will encounter in Bayesland is soft. Soft evidence does not arise from measurable regularities. Rather, it is shaped by the observer’s perception and filtered through his judgement and interpretation, giving rise to subjective probabilities such as those depicted in the joyplot. These probabilities are grounded in beliefs, confidence, and trust.
Probability in Bayesland is a primal concept which, since the dawn of civilisation, has been inherently tied to the evaluation of soft evidence.2 The Law is the original domain in which the concept of probability has been moulded through time. Law needs to reach a decision, i.e. separate true from false, guilt from innocence. To reach a verdict is, literally, verum dicere: to speak the truth. Over the centuries, different legal systems defined different rules of evidence but relied on the same underlying relationship between beliefs and trust. Now as then, our confidence rests on the judgement of reputable authorities. Indeed, the very word probability comes from the Latin probabilis, which means ‘worthy of approbation‘ 3. Approbation comes from the recognized probity of honest, honourable people, who are thus capable of influencing our beliefs by approving a hypothesis as true or false.
A tug of war between probi viri: this is probability in Bayesland.
Before we begin, let’s give some structure to what has so far been a deliberately vague introduction.