Extraordinary Claims Require Extraordinary Evidence

⬅️ Hub: Epistemology (MOC) · Formal tool: Bayes’ theorem · Historical source: David Hume · Neighbor: Deductive & Inductive Reasoning

Core idea

The less plausible a claim is in light of relevant background knowledge, the more discriminating the evidence must be before the claim becomes rationally credible. This is not a second logic for unusual claims. It is the ordinary logic of evidential updating applied to a low prior probability and live alternative explanations.

In one minute

• Ordinary claim: “I saw a dog in my neighbor’s yard.” Dogs and yards are common, so testimony from a normally reliable person may be enough.
• Extraordinary claim: “I saw a dog levitate ten feet into the air.” Error, illusion, exaggeration, deception, hidden support, or a manipulated recording are initially much more probable than unaided levitation.
• The same statement can therefore rationally produce different confidence because its content changes the prior probability and the plausibility of competing explanations.
• What matters is not whether evidence looks dramatic, but whether it is far more expected if the claim is true than if the claim is false.
• “Extraordinary” should mean strongly surprising relative to justified background knowledge, not merely unfamiliar, unpopular, offensive, or contrary to one’s ideology.

Most precise formulation

Posterior odds = prior odds × likelihood ratio.

A low prior is overcome only by evidence with a sufficiently large likelihood ratio—that is, evidence much more probable under the extraordinary claim than under its serious alternatives.

What the slogan does—and does not—mean

The familiar wording, “extraordinary claims require extraordinary evidence,” is commonly associated with Carl Sagan, who popularized it. Marcello Truzzi had used the close formulation “an extraordinary claim requires extraordinary proof” in 1975. The underlying epistemic idea is older: David Hume’s treatment of miracle testimony in Section X of An Enquiry Concerning Human Understanding is a classic precursor.¹

It is therefore more accurate to say:

1. the exact slogan is modern;
2. the balancing of prior credibility against testimony has important antecedents in Hume and other Enlightenment writers; and
3. Bayesianism provides a modern formal reconstruction of that logic—it is not the historical source of Hume’s argument, and scholars dispute whether Hume himself should be read as a mathematical Bayesian.²

The principle does not imply:

• two species of logic, one for ordinary claims and one for extraordinary claims;
• that unfamiliar claims may be rejected without investigation;
• that only an equally strange-looking object can count as evidence;
• that testimony is never evidence for an unusual event;
• that scientific consensus fixes truth permanently; or
• that a person may assign a probability of zero to whatever challenges their worldview.

The Bayesian structure

Let:

• $H$ = the claim or hypothesis;
• $E$ = the observed evidence;
• $P(H)$ = the prior probability of the claim, before incorporating $E$;
• $P(E\mid H)$ = how probable the evidence would be if the claim were true;
• $P(E\mid \neg H)$ = how probable the evidence would be if the claim were false; and
• $P(H\mid E)$ = the posterior probability, after incorporating the evidence.

Bayes’ theorem is:

$$P(H\mid E)=\frac{P(E\mid H)P(H)}{P(E\mid H)P(H)+P(E\mid \neg H)P(\neg H)}$$

For comparing a claim with its negation, the odds form is more revealing:

$$\underset{\text{posterior odds}}{\underbrace{\frac{P(H\mid E)}{P(\neg H\mid E)}}}=\underset{\text{prior odds}}{\underbrace{\frac{P(H)}{P(\neg H)}}}\times \underset{\text{likelihood ratio / Bayes factor}}{\underbrace{\frac{P(E\mid H)}{P(E\mid \neg H)}}}$$

This corrects the loose shorthand “posterior ∝ prior × strength of evidence.” The shorthand is directionally right, but evidential strength is comparative. Evidence supports $H$ to the extent that it is more expected under $H$ than under a competitor.

Diagnosticity, not spectacle

A vivid video may be weak evidence for levitation if convincing-looking edited videos are common. A dull instrument reading may be powerful evidence if it would be extraordinarily unlikely under every non-levitation explanation. “Extraordinary evidence” means a large evidential advantage over the alternatives, not evidence that looks theatrical.

Worked example: the same witness, two priors

Suppose a witness reports $H$, and, for illustration:

$$P(\text{report }H\mid H)=0.99$$ $$P(\text{report }H\mid \neg H)=0.01$$

The report has a likelihood ratio of $99$: it is 99 times more likely if $H$ is true than if $H$ is false.

The testimony is strong evidence in both cases: it multiplies the odds by 99. But the extraordinary claim began with such low odds that one report still leaves it improbable. Strong evidence can greatly raise a probability without making the claim probable overall.

How strong would the evidence have to be?

If the acceptance target were 95%, then the target posterior odds would be $19:1$.

• Starting from a 50% prior, a Bayes factor of 19 reaches 95%.
• Starting from a 0.001% prior, the required Bayes factor is roughly 1.9 million.

There is nothing sacred about a 95% threshold; it is only an illustration. The appropriate threshold depends on whether one is forming a tentative belief, publishing a scientific claim, convicting a defendant, prescribing a risky treatment, or taking a reversible precaution.

Why ordinary testimony sometimes suffices

Testimony is not evidentially weightless. We routinely and rationally learn from other people. For a mundane claim, several facts may already favor it:

• the event has a high base rate;
• the witness had a good opportunity to observe it;
• the report is ordinary enough that elaborate deception is unlikely;
• normal perceptual error would not radically change the type of event reported; and
• nothing important turns on perfect certainty.

For an extraordinary report, the comparison changes. The relevant contest is rarely just:

miracle versus lying.

It is usually:

unusual event versus misunderstanding or memory distortion or illusion or fraud or instrument failure or selection effects or an unknown but natural mechanism.

A sound evaluation compares the claim with the best developed alternatives, not with a single caricatured explanation.

What can count as extraordinary evidence?

There is no special substance called “extraordinary evidence.” The evidence can be ordinary in kind while exceptional in its reliability, independence, quantity, or explanatory specificity.

No single item is automatically decisive. A video, a credentialed witness, a statistically significant result, or many agreeing reports can each remain weak if the same evidence is also likely under error, fraud, bias, or dependence.

Independence matters

If genuinely independent evidence items have Bayes factors $B{F}_1,B{F}_2,\dots ,B{F}_n$, their combined force can sometimes be represented by multiplication:

$$\text{posterior odds}=\text{prior odds}\times B{F}_1\times B{F}_2\times \cdots \times B{F}_n$$

Using the witness illustration above and assuming—unrealistically strongly—that each witness has a Bayes factor of 99 and their errors are conditionally independent:

• one witness moves a 0.001% prior to about 0.099%;
• two move it to about 8.9%; and
• three move it to about 90.7%.

But witnesses often share news reports, expectations, group pressures, environmental conditions, or the same mistaken source. Ten retellings of one story are not ten independent confirmations. Naively multiplying correlated evidence double-counts it.

Hume on testimony and miracles

Hume’s discussion appears in Section X, “Of Miracles,” of An Enquiry Concerning Human Understanding. He begins from the general rule that a wise person “proportions his belief to the evidence.”³

His reasoning can be reconstructed as follows:

1. We trust testimony because experience shows that testimony is often connected with facts.
2. That trust is defeasible: witnesses can misperceive, misremember, exaggerate, deceive, or transmit rumors.
3. A miracle, on Hume’s definition, conflicts with the regular course of experience that supports a law of nature.
4. Rational judgment must weigh the testimonial evidence against the evidence supporting the regularity and against the known fallibility of testimony.
5. Testimony establishes a miracle only when the falsehood of that testimony would be even less credible than the event reported.³

This is structurally close to asking which hypothesis better predicts all the evidence:

• $H_M$: the miracle occurred;
• $H_F$: the report arose through some combination of error, deception, distortion, or another non-miraculous cause.

Hume’s point is not simply that “nature always wins by definition.” His maxim tells us to compare rival explanations and accept only the stronger balance of evidence. His own application is highly skeptical because he judges the historical miracle reports available to him to fall far short.

A qualification about Hume

Hume’s argument remains contested. Critics argue that if “law of nature” means an exceptionless regularity and that regularity is treated as probability 1, the conclusion becomes question-begging: no finite testimony could ever update a zero prior. Other interpreters reply that Hume does not assign literal probability zero; he compares degrees of experiential support, and even gives a hypothetical case in which extensive, uniform testimony could establish an extraordinary worldwide darkness.²

A careful Bayesian version avoids dogmatism:

• give the miracle or anomaly a low but nonzero prior if it is logically possible;
• state the background assumptions openly;
• model the strongest relevant alternatives;
• permit sufficiently diagnostic evidence to overcome the prior; and
• revise the supposed “law” if reproducible contrary evidence accumulates.

The most important qualifications

“Extraordinary” is not a synonym for “I dislike it”

A rational prior should be informed by relevant base rates, established observations, explanatory success, and well-supported theory. It should not merely encode political loyalty, religious commitment, cultural familiarity, aesthetic preference, or deference to prestige.

Novel is not the same as antecedently improbable

A claim may be new without conflicting with much prior evidence. A newly discovered species in a poorly surveyed ecosystem is unusual news, but it is not comparable to a violation of conservation laws. Conversely, a familiar rumor can still be antecedently unlikely.

A low prior is not a veto

Assigning $P(H)=0$ makes updating impossible: multiplying by any finite Bayes factor still gives zero. Except for logical contradictions or stipulations, extreme confidence should ordinarily remain revisable.

The prior must not already contain the new evidence

If a surprising measurement is used to lower the prior and then used again as the likelihood evidence, it is counted twice. Priors should reflect the background information available before the focal evidence is incorporated.

Evidence against the old view can be evidence for the new claim

The evidential burden is not paid only by directly proving every part of a novel theory. Repeated failures of the established model, successful novel predictions, and the elimination of rival explanations can substantially shift the posterior.

“Unexplained” does not identify the explanation

Evidence that defeats one mundane explanation does not automatically establish a specific extraordinary one. “The light was not an aircraft” does not by itself entail “the light was extraterrestrial.” The hypothesis must predict the evidence better than all serious competitors.

More evidence is not necessarily better evidence

One highly discriminating observation can outweigh a thousand weak or correlated observations. The crucial question is not simply “How much evidence is there?” but “How large is the likelihood ratio, and have dependencies been handled?”

Belief thresholds and action thresholds differ

One can rationally take precautions against a low-probability catastrophe when the downside is enormous, even while withholding belief that the catastrophe is likely. Decision-making depends on probability and consequences; the ECREE maxim concerns evidential support, not the whole theory of action.

The burden of proof is related but not identical

Bayesian updating describes how evidence changes rational credence. A burden of proof is a rule governing who must support a claim and how much support a context demands. Legal, scientific, historical, and everyday contexts use different thresholds. The person advancing a departure from a shared background often bears a practical burden, but that convention is not itself Bayes’ theorem.

Strong objections and the best replies

Objection: “All claims require the same evidence”

Concession: All claims should be evaluated by the same rules. There is no magical category of evidence reserved for anomalies.

Reply: The same rule yields different burdens because the prior odds and alternative explanations differ. Equal treatment of the evidence does not imply equal posterior confidence from the same testimony.

Objection: “The maxim protects orthodoxy and suppresses discovery”

Concession: The word “extraordinary” is easily abused. A consensus can be wrong, and gatekeepers can disguise status-quo preference as a prior.

Reply: Proper Bayesian reasoning requires explicit, evidence-based priors and real updating. Reproducible anomalies must raise the probability of revision. Mere novelty or social unpopularity is not a legitimate reason for an extremely low prior. David Deming’s critique is useful here: ECREE becomes anti-scientific when it functions as a label rather than an analysis.⁴

Objection: “Priors are subjective”

Concession: Reasonable people can assign different priors, especially when data are sparse or background theories are disputed.

Reply: That is a reason for transparency and sensitivity analysis, not for pretending priors do not exist. Ask whether a conclusion survives across a defensible range of priors and whether disputants agree on the likelihood ratio even when they disagree about the starting point.

Objection: “Hume assumes miracles cannot occur”

Concession: A rigid reading that gives miracles a zero prior would beg the question.

Reply: The more defensible principle gives unusual claims low but revisable priors and asks whether the total evidence favors the event over testimonial failure and other alternatives. This preserves Hume’s balancing insight without making counterevidence impossible in principle.

Objection: “A true extraordinary event may leave only ordinary evidence”

Concession: Reality does not guarantee that every truth will be knowable. A rare event may occur without leaving enough evidence to establish it.

Reply: The principle concerns when belief is justified, not what can happen. “It occurred” and “we now have sufficient reason to believe it occurred” are different claims.

A practical evaluation checklist

When someone makes an extraordinary claim, ask:

1. What exactly is the hypothesis? Make it specific enough to test.
2. Extraordinary relative to what? Identify the background knowledge and base rates doing the work.
3. What is the strongest rival explanation? Avoid comparing the claim only with a straw man.
4. How likely is this evidence if the claim is true? Estimate $P(E\mid H)$.
5. How likely is the same evidence if the claim is false? Estimate $P(E\mid \neg H)$.
6. Are the evidence sources genuinely independent? Trace common sources, incentives, methods, and failure modes.
7. Was the prediction made before the result? Distinguish forecast from accommodation after the fact.
8. Could selection effects explain the report? Look for unpublished failures, cherry-picking, and survivor bias.
9. What evidence would change each side’s mind? A position insulated from every possible observation is not updating.
10. Does the conclusion survive plausible priors? Run a sensitivity check rather than hiding the assumptions.
11. What level of confidence is actually needed? Separate tentative belief, public assertion, and high-stakes action.
12. Am I applying the standard symmetrically? Demand proportionate evidence from congenial and uncongenial claims alike.

Compact formulations

Evidence should be proportioned to the prior improbability of the claim.

More precisely:

The lower the prior odds of a claim, the larger the likelihood ratio required to reach a given posterior confidence.

And in plain language:

Ask not whether the evidence is impressive, but whether it would be much harder to explain if the claim were false.

Related

• Bayes’ theorem — the formal update rule and common base-rate errors
• David Hume — testimony, induction, empiricism, and religious skepticism
• An Enquiry Concerning Human Understanding — Section X, “Of Miracles”
• Deductive & Inductive Reasoning — where probabilistic inference sits among forms of reasoning
• Skepticism — doubt as an epistemic method rather than a fixed conclusion
• Logical Fallacies — recurring failures in reasoning about evidence

References

• Hume, David. “Of Miracles,” Section X of An Enquiry Concerning Human Understanding (1748; 1777 text), Hume Texts Online.
• Coventry, Angela M. “David Hume,” especially §5.1, “Miracles,” Stanford Encyclopedia of Philosophy (substantive revision 2026), entry.
• Levine, Michael. “Miracles,” Stanford Encyclopedia of Philosophy (substantive revision 2024), entry.
• Deming, David. “Do Extraordinary Claims Require Extraordinary Evidence?” Philosophia 44 (2016): 1319–1331, doi:10.1007/s11406-016-9779-7.
• National Academies of Sciences, Engineering, and Medicine. Reproducibility and Replicability in Science, Appendix D, “Using Bayes Analysis for Hypothesis Testing” (2019), NCBI Bookshelf.

Footnotes

1. Deming documents Sagan’s popularization, Truzzi’s earlier near-formulation, and the Enlightenment background. The genealogy of the maxim is distinct from the mathematical justification developed here. ↩

2. See the interpretive discussion in §5.1 of the Stanford Encyclopedia of Philosophy entry on Hume and §3 of its entry on miracles. ↩ ↩²

3. David Hume, An Enquiry Concerning Human Understanding, §10.4 and §10.13, in Hume Texts Online. ↩ ↩²

4. David Deming, “Do Extraordinary Claims Require Extraordinary Evidence?”, argues that the slogan is often misused when “extraordinary” is left undefined or equated with novelty. ↩