A rule for updating a belief when new evidence arrives. It is a short piece of algebra, it is not in any dispute, and misapplying it produces some of the most consequential errors in medicine and law.
The theorem relates the probability of a hypothesis given evidence to the probability of the evidence given the hypothesis. These are different quantities and confusing them is the source of most of the trouble.
Three ingredients. The prior: how likely the hypothesis was before the evidence. The likelihood: how probable the evidence is if the hypothesis is true. The evidence term: how probable the evidence is overall, whether or not the hypothesis holds.
The result is the posterior: how likely the hypothesis is now.
The essential content is that evidence updates a prior rather than replacing it. A test result does not tell you whether something is true; it tells you how much to adjust what you already believed.
Thomas Bayes, an English Presbyterian minister, worked on the problem and did not publish. Richard Price found the essay among his papers after his death and published it in 1763.
Pierre-Simon Laplace derived the result independently and developed it far more thoroughly, applying it to astronomy, to population estimates and to the reliability of testimony. For much of the nineteenth century the method was called inverse probability, and Laplace did most of the work now attached to Bayes' name.
The standard demonstration is worth working through, because it is not a curiosity: clinicians get it wrong routinely when tested.
Take a disease affecting one person in a thousand, and a test that is ninety nine percent accurate in both directions.
A person tests positive. The intuitive answer is that they almost certainly have the disease.
Consider ten thousand people. Ten have the disease and about ten of those test positive. Nine thousand nine hundred and ninety do not, and about one percent of them, roughly a hundred, test positive anyway.
So around a hundred and ten positive results, of which about ten are correct. The probability of having the disease given a positive test is roughly nine percent, not ninety nine.
The reason is the base rate. False positives from a large healthy population outnumber true positives from a small sick one, and no improvement in test accuracy removes that unless the disease is common. This is why screening asymptomatic populations for rare conditions produces mostly false alarms, and it is the mathematics underlying the capsule on overdiagnosis.
The same error in a courtroom has produced serious miscarriages of justice.
The fallacy is treating the probability of the evidence given innocence as the probability of innocence given the evidence. A match occurring in one in a million people is not a one in a million chance of innocence, because in a population of sixty million there are sixty matches.
Sally Clark was convicted in England in 1999 of murdering two of her children, partly on expert testimony that the chance of two cot deaths in one family was one in seventy three million. That figure was wrong on its own terms, because it assumed independence between the deaths when a shared genetic or environmental cause makes a second death more likely after a first. It was also the wrong comparison: the relevant question is how the probability of two cot deaths compares with the probability of two murders, which is also rare. Her conviction was quashed in 2003 and the case prompted a review of similar convictions.
The theorem is a consequence of the axioms of probability and nobody disputes it. What is disputed is how widely the framework should be used.
The frequentist position treats probability as a long-run frequency, so a fixed parameter has no probability distribution, and inference proceeds through significance tests and confidence intervals. This is the tradition most scientists are trained in.
The Bayesian position treats probability as a degree of belief, so a parameter can have a distribution and prior information can be incorporated explicitly. This is more flexible and requires choosing a prior, which critics regard as introducing subjectivity into what should be an objective procedure.
The argument was bitter for most of the twentieth century and has largely cooled. The practical resolution has been computational: methods for sampling from complex posterior distributions made Bayesian analysis feasible for realistic problems from the 1990s, and it is now standard in phylogenetics, in epidemiological modelling, in astronomy and in machine learning.
The strongest general defence of the framework is that it makes assumptions explicit. A prior must be stated and can be argued about. In frequentist analysis assumptions are equally present and are embedded in the choice of test and model, where they are harder to see and therefore harder to challenge.
