Replication Crisis

⬅️ 60.21 Psychology · Belief and bias: Psychology of Belief (MOC) · Judgment: Why Smart People Get Things Wrong - Daniel Kahneman and Sam Harris · Practice: Critical Thinking · Epistemology: Extraordinary Claims Require Extraordinary Evidence

Core idea

The replication crisis is not the discovery that “science is fake.” It is the discovery that the published literature can be much more confident, positive, and precise than the underlying evidence warrants. Small samples, low statistical power, flexible analysis, selective publication, weak theory, and occasionally fabrication all produce the same surface symptom: a striking result that weakens or disappears when tested again. The response has been a credibility revolution—preregistration, Registered Reports, open data and code, larger multi-lab studies, explicit robustness checks, and a sharper separation between exploration and confirmation.

Evidence and attribution boundary

This note is current through September 16, 2026. It distinguishes a failed replication, a retraction, an allegation, an institutional misconduct finding, and a court ruling. Those are not interchangeable. A failed replication does not prove fraud; co-authorship does not identify who altered data; and a retraction establishes that a paper should no longer be relied upon, not necessarily who was responsible or what their intent was.

The Righteous Mind, Chapter 3 — an endnote-level audit queue, not a retrospective verdict

Haidt’s 2012 synthesis preserves four cautions that are easy to lose when its examples are retold: the hypnosis study selected highly hypnotizable students; the foul-odor study’s four outcomes moved in the predicted direction but not every individual comparison was significant (the combined result was reported as p < .001); the infant looking-time effect appeared among ten-month-olds but not six-month-olds while reaching favored the helper at both ages; and the original trolley reaction-time result was later challenged as an artifact of the selected stories.

These details illustrate why an evidence audit must travel below a book’s headline. They do not by themselves tell us which findings now replicate. The odor, bitter-taste, handwashing, transgression-cleansing, and sanitizer studies should be checked claim by claim against current direct replications, preregistrations, effect sizes, and corrections before being used as robust evidence. Until that review, label them source-dated findings rather than either settled facts or “debunked” effects. Hypnosis selection, PDF p. 370 (file not included in this release) · Odor and cleanliness endnotes, PDF pp. 372-373 (file not included in this release) · Infant qualification, PDF p. 373 (file not included in this release) · Trolley correction, PDF p. 374 (file not included in this release) · Permanent concept note

Executive synthesis

1. Replication asks whether a result survives new data. Reproducibility, under the National Academies’ terminology, asks whether the same data and analysis produce the same computational result.
2. The crisis is a selection problem as much as a conduct problem. If journals favor surprising positive findings while researchers have many defensible analytic choices, the published record becomes a biased sample of all the analyses and studies that were attempted.
3. P-hacking need not look like cheating from the inside. Repeatedly checking, excluding, transforming, subgrouping, or switching outcomes can feel like reasonable problem-solving. If those choices were guided by the observed results and then presented as if prespecified, the nominal p value no longer means what readers think it means.
4. Fraud is real but is not the default explanation for nonreplication. Honest sampling error, low power, measurement differences, hidden context, coding mistakes, and overgeneralized theory can all produce inconsistent studies. Fabrication and falsification require separate evidence.
5. “Did it replicate?” is not a single binary statistic. Significance, effect direction, effect-size compatibility, prediction intervals, Bayesian evidence, and whether the combined evidence favors an effect can yield different answers.
6. Several famous claims were badly downgraded, not all erased. The hormonal and risk-taking claims in “power posing,” a large ego-depletion effect, signing first to induce honesty, and the Ten Commandments honesty prime did not survive their strongest direct tests. Narrower or modified claims sometimes remain live.
7. Science’s correction mechanisms worked—but often late and at personal cost. Replicators, junior scholars, whistleblowers, data sleuths, journals, and institutional investigators exposed errors that ordinary peer review had missed. The delay shows both the resilience and the weakness of the system.

What exactly is being replicated?

The words reproducibility and replicability are used differently across disciplines. This note follows the definitions in the U.S. National Academies’ 2019 consensus report, Reproducibility and Replicability in Science:

Direct and conceptual replication

• A direct replication holds the original protocol as constant as practical. It is the clearest test of whether a particular result can recur under similar conditions.
• A conceptual replication tests the same theoretical claim with a different manipulation or measure. It may support a theory even if the precise original effect fails—but it also gives researchers more freedom to reinterpret what the theory predicted.
• A multi-lab replication runs a coordinated protocol at many sites. It can estimate the average effect and between-site variation while reducing dependence on one laboratory’s idiosyncrasies.

A single failed replication is evidence, not a verdict

The National Academies emphasizes that a successful replication does not prove the original interpretation correct and one failed replication does not conclusively refute it. Repeated, well-powered, preregistered failures are much stronger evidence than one small null result; likewise, repeated success cannot rescue a design whose measure does not identify the claimed construct.

How large is the problem?

There is no scientifically honest universal answer such as “only 36% of psychology is true.” Different projects sample different journals, periods, fields, claim types, and feasibility levels; they also use different definitions of success.

The strongest general conclusion is therefore not a magic failure percentage. It is that published effects are often inflated, many positive claims are less stable than the literature made them appear, and the answer depends materially on how replication is defined and sampled.

The causal anatomy of the crisis

Career and journal rewards favor novelty, clarity, and positive results
                              ↓
Small samples + noisy measures + weakly constrained theories
                              ↓
Many reasonable-looking choices in collection, cleaning, and analysis
                              ↓
Only the striking path is written up; null paths and studies disappear
                              ↓
Published effects are selected upward (the winner's curse)
                              ↓
Larger, preregistered, or independent studies estimate smaller effects
                              ↓
Replication failure, controversy, correction, or retraction
 
Separate branch: fabrication or falsification → apparently clean result → forensic or institutional investigation

1. Low power and the winner’s curse

Small samples make estimates unstable. A real but modest effect will be detected only when random noise happens to push the estimate upward; therefore, the subset of small studies that cross the significance threshold tends to exaggerate the effect. Low power also makes true effects easier to miss. Button and colleagues’ review, “Power failure”, explains why underpowered neuroscience produces both low discovery probability and inflated published estimates.

2. Publication and citation bias

Researchers may not write up null studies; reviewers may find them uninteresting; editors may prefer a clean positive story; and later authors may preferentially cite the exciting result. The published literature is then not a neutral census of the evidence but a filtered subset. In a comparison of psychology articles, Scheel, Schijen, and Lakens (2021) found that 96% of standard reports supported their first hypothesis versus 44% of Registered Reports. This is not a randomized trial of publishing formats, but the enormous difference is consistent with strong selection and/or analytic inflation in the ordinary literature.

3. Researcher degrees of freedom

Real data require judgment: Which observations count? Which outcome best measures the construct? Which covariates belong in the model? When is the sample large enough? The problem is not discretion itself. The problem is undisclosed, result-contingent discretion combined with a statistical test that assumes the tested analysis was not selected because it looked favorable.

4. HARKing and theory that can explain everything

HARKing means hypothesizing after the results are known and writing the result as an a priori prediction. Post hoc theorizing is indispensable for discovery. It becomes misleading when readers are not told that the hypothesis was generated by the same data said to confirm it. A theory that can absorb every outcome also risks being unfalsifiable: a failure becomes an unspecified “boundary condition,” while success is treated as confirmation.

5. Measurement and hidden context

A manipulation may not affect the intended construct; an outcome may be unreliable; translations, participant pools, experimenter behavior, or historical context may matter. “The effect is context-sensitive” can be a real scientific discovery—but it must generate a testable account of which contexts and why, not serve as an all-purpose escape hatch after failure.

6. Incentives and prestige

Hiring, tenure, grants, consulting, books, and media attention reward a stream of surprising, accessible findings. Peer review normally inspects a polished manuscript, not the complete history of abandoned analyses, raw files, and laboratory decisions. Prestige can then weaken skepticism: “too good to be true” data may be read as evidence of exceptional skill rather than a reason for audit.

7. Error and misconduct

Coding mistakes, mislabeled conditions, duplicate rows, spreadsheet errors, and data loss can invalidate a result without deceptive intent. Fabrication means making up data or results; falsification means manipulating materials, processes, or data so the record is inaccurate. Intent and responsibility require evidence beyond a failed replication.

P-hacking in depth

What it is

P-hacking is the use of data-dependent analytic choices to obtain, retain, or foreground a conventionally significant result—usually p < .05—without accounting for the search process. The term covers deliberate fishing, but it also covers motivated exploration by researchers who sincerely believe each decision is sensible.

A p value is approximately the probability, assuming a specified null model and the test’s other assumptions, of obtaining a test statistic at least as incompatible with that model as the one observed. It is not:

• the probability that the null hypothesis is true;
• the probability that the result occurred “by chance”;
• the probability that the finding will replicate;
• the size or practical importance of the effect; or
• the false-positive rate for a conclusion selected after many hidden analyses.

The American Statistical Association’s statement on p values stresses that scientific decisions should not rest on whether a number crosses a threshold and that full reporting and effect size matter.

Why flexibility changes the false-positive rate

If a researcher conducts one prespecified valid test under a true null at α = .05, its long-run false-positive rate is 5%. But suppose 20 independent outcomes, subgroups, or specifications are tried and only the most favorable is reported. The probability that at least one crosses .05 is:

Real analyses are often correlated, so this toy calculation is not a literal estimate for a particular study. It shows the logic: the relevant error rate belongs to the whole search procedure, not merely the final line printed in the paper.

In “False-Positive Psychology”, Joseph Simmons, Leif Nelson, and Uri Simonsohn simulated common researcher choices. Individually, optional stopping, selecting between dependent variables, adding a covariate, or selectively reporting conditions raised the false-positive rate above 5%. Combining their four simulated freedoms produced a false-positive rate of 60.7%. Their deliberately absurd demonstration appeared to show that listening to “When I’m Sixty-Four” made participants younger—using practices then accepted in mainstream research.

The common “garden of forking paths”

Andrew Gelman and Eric Loken’s “garden of forking paths” adds an important point: researchers do not have to run every possible model. Knowledge of the observed data can silently guide a path that would have been different had the data looked different. The analysis is therefore selected even if only one final model was executed.

P-hacking is not the same as fraud

The ethical boundary

Exploration becomes reliable science when the search is disclosed and the resulting hypothesis faces a genuinely new test. The sin is not curiosity; it is presenting the destination as if the route had been fixed before the journey began.

Famous findings that were retracted, weakened, or disputed

“Refuted” is often too blunt. The ledger below records what the original claim was, what later evidence actually tested, and the best current verdict.

1. Signing first makes people more honest

Original claim. A 2012 PNAS paper reported that signing a declaration at the beginning rather than the end of a form made ethics salient and reduced dishonest reporting. The idea was adopted in some real-world forms and policy discussions.

What happened next. Kristal and colleagues ran five conceptual replications (combined n = 4,559) and one high-powered, preregistered direct replication (n = 1,235) designed with authors of the original paper. They found no effect of signing first on honest reporting (2020 replication). A 2026 correction fixed errors in that replication paper but left the direct-replication conclusion unchanged. Separately, forensic evidence showed that the insurance-company data in Study 3 of the 2012 paper were fabricated, and PNAS retracted the article in 2021.

Verdict. The paper is retracted, its flagship intervention failed a strong direct test, and it should not be used as evidence that moving a signature line reduces dishonesty. This does not establish that every advance honesty pledge is ineffective; it invalidates the specific paper and simple signing-first prescription.

2. Recalling the Ten Commandments reduces cheating

Original claim. In Experiment 1 of Mazar, Amir, and Ariely (2008), participants who recalled the Ten Commandments reportedly cheated less than participants who recalled books; the reported difference corresponded to about d = 0.48.

What happened next. A Registered Replication Report coordinated 25 direct replications with a total N = 5,786. In the primary meta-analysis of 19 labs (n = 4,674), the estimated effect was essentially zero and slightly opposite the original direction (d ≈ −0.04; the raw difference was 0.11 matrices, 95% CI [−0.09, 0.31]) (Verschuere et al., 2018). The original authors reviewed the common protocol. The original article received a 2024 expression of concern.

Verdict. The original Ten Commandments effect was not supported by the large direct-replication project. The result does not prove that all moral reminders in all contexts are useless; it sharply lowers confidence in this iconic demonstration and in broad claims built from it.

3. “Power poses” alter hormones, risk tolerance, and confidence

Original claim. Carney, Cuddy, and Yap’s 2010 study (N = 42) reported that brief expansive versus contractive postures changed testosterone, cortisol, risk tolerance, and feelings of power.

What happened next. A larger replication (N = 200) found no effect on hormones or risk tolerance, although participants in expansive poses reported feeling more powerful (Ranehill et al., 2015). A later special issue of preregistered studies found little support for downstream behavioral benefits; debate continued over a narrower postural-feedback effect on subjective feelings (2017 overview; Bayesian meta-analysis). Original first author Dana Carney publicly withdrew confidence in the original effect package, while Amy Cuddy continued to defend narrower affective claims.

Verdict. The famous hormonal and risk-taking package is not robust. “Standing expansively changes your hormones and makes you take consequential risks” is not supported by the best direct evidence. Small effects on self-reported feelings or closely related outcomes remain debated and should not be conflated with the original headline.

4. Self-control is a depletable resource (“ego depletion”)

Original claim. Exerting self-control on one task was said to consume a limited resource, impairing performance on a second task. Early meta-analysis suggested a medium effect.

What happened next. A 23-lab Registered Replication Report (N = 2,141) estimated d = 0.04, 95% CI [−0.07, 0.15] (Hagger et al., 2016). A different 36-lab preregistered paradigmatic test (N = 3,531) found a nonsignificant confirmatory effect of d = 0.06, with the data about four times more likely under the specified null than the informed alternative (Vohs et al., 2021). Other preregistered work has found small effects under some protocols, including a 12-lab estimate of d = 0.10 (Dang et al., 2021).

Verdict. The large, general, resource-like depletion effect was strongly downgraded. A small, task-dependent fatigue, motivation, or control effect may exist, but the simple “willpower is a fuel tank” model is not established by the once-celebrated evidence.

5. Priming the elderly stereotype makes students walk more slowly

Original claim. Bargh, Chen, and Burrows (1996) reported that participants unobtrusively exposed to words related to old age walked more slowly after leaving the experiment.

What happened next. Doyen and colleagues used automated timing and did not reproduce the basic effect. They obtained slower walking only when experimenters expected the prime to slow participants, implicating experimenter expectancy (Doyen et al., 2012). Other social-priming results and conceptual replications have been mixed, and defenders argue that linguistic, cultural, procedural, or awareness differences can moderate priming.

Verdict. The iconic walking-speed demonstration is not a secure, general effect. The larger proposition that prior stimuli can influence later cognition or behavior is far broader and is not refuted by the failure of this one dramatic paradigm.

6. Evenly spaced deadlines reduce procrastination and improve performance

Original claim. Ariely and Wertenbroch (2002) reported that externally imposed, evenly spaced deadlines improved performance relative to self-set deadlines or one final deadline.

What happened next. A preregistered replication did not reproduce the central result (Hyndman & Bisin, 2026). Data Colada then published forensic analyses alleging that the data files for both main studies had been tampered with. On September 2, 2026, Psychological Science retracted the paper, citing the failed replication, inability to confirm the authenticity and completeness of the underlying data, and the Data Colada investigation. Both authors agreed to the retraction.

Verdict. The paper is retracted and its findings are not reliable. This does not show that deadlines never help; it removes this influential article as trustworthy evidence for the particular pattern it reported.

7. Brief canvassing conversations durably change attitudes toward same-sex marriage

Original claim. LaCour and Green (2014) reported unusually large, persistent, and contagious attitude change after conversations with gay canvassers.

What happened next. Science retracted the paper after confirming that participant incentives and funding were misrepresented, independent analysts identified statistical irregularities, and lead author Michael LaCour did not produce original survey data for verification. Co-author Donald Green concurred with retraction; LaCour did not. Later canvassing studies using verifiable data have found persuasion effects under other designs, so the general possibility of conversational attitude change is separate from this invalid article.

Verdict. The 2014 paper is unusable evidence. The journal’s documented reasons are stronger than merely “someone failed to replicate it,” but they do not by themselves settle every broader contact or canvassing theory.

The dishonesty-research scandal: one paper, two compromised datasets

Gideon Lewis-Kraus’s 2023 New Yorker investigation is valuable because it reconstructs how behavioral ethics became both a showcase for clever, policy-friendly findings and a case study in the system’s vulnerabilities.

The 2012 signing-first paper combined studies overseen by different co-authors:

• Study 3, the automobile-insurance field data. Data Colada’s 2021 analysis documented duplicated and altered records, generated-looking values, inconsistent formatting, and other evidence that the posted data were fabricated. Lewis-Kraus later reported comparisons with records from The Hartford indicating that large portions of the dataset were invented or changed. Dan Ariely had supplied the file to the research team and its metadata named him, but he denied fabricating or manipulating it and said the data had come from the insurer. The Hartford denied responsibility. The existence of fabricated data is established by the retraction; who performed the fabrication was not established in a public institutional finding against Ariely.
• Study 1, the laboratory signing experiment. Data Colada’s 2023 analysis identified row movements, clustered observations, and other signs of alteration. Harvard’s later investigation attributed research misconduct involving this and three other papers to Francesca Gino.

This is more disturbing than a single contaminated file: evidence indicated that two distinct datasets in the same article were separately compromised, apparently through different mechanisms. It also illustrates why the unit of trust cannot simply be “the author” or even “the paper.” Each study needs a provenance trail: who collected it, who had custody, which file is raw, what transformations occurred, and which author verified the analysis.

People and cases: an evidence-status ledger

How to read this section

Inclusion means that a case became central to the modern replication or research-integrity debate. It is not a claim that every listed person committed the same conduct. The status column is controlling: allegation, retraction, institutional finding, and unresolved attribution are intentionally kept separate.

What the cases teach

• A replication failure and a forensic anomaly are different kinds of evidence. Null results challenge an effect; duplicated rows, impossible distributions, file metadata, or mismatched raw records challenge the integrity of a dataset.
• Author order is not a chain of custody. Multi-author papers need explicit contribution statements and verifiable data provenance.
• Peer review is not an audit. Reviewers generally do not inspect raw-data histories or independently rerun every analysis.
• Junior researchers face asymmetric risk. Stapel’s whistleblowers and Zoé Ziani’s challenge to Gino’s networking study show why protected channels and independent review matter.
• Retraction notices vary in informativeness. “Retracted” tells the reader to stop relying on a paper; the notice and institutional record tell the reader why.
• Institutional secrecy can leave attribution unresolved. Confidentiality protects fairness and privacy, but it can also leave the public dependent on one party’s characterization of an investigation.

Why replication failure does not automatically mean fraud—or even falsity

Suppose an original study estimates an effect of +0.40 and a replication estimates +0.10. Several explanations are possible:

1. The true effect is zero, and the original was a false positive.
2. The true effect is small, and the original was selected upward.
3. Both estimates are noisy, and their uncertainty intervals overlap substantially.
4. The populations or implementation differ in a way the theory should eventually explain.
5. The manipulation failed or the outcome did not measure the intended construct in one study.
6. The original or replication contains error in data, code, or reporting.
7. The original data were fabricated or falsified—a serious claim requiring affirmative evidence.

The scientifically useful response is not to choose the most dramatic story. It is to compare protocols, manipulation checks, effect sizes and intervals, data quality, preregistration, power, and all available replications. A good meta-analysis asks whether variation is compatible with sampling error, whether moderators were specified in advance, and whether missing studies could distort the record.

Statistical significance is not a replication criterion by itself

“Original p = .04; replication p = .07” can occur even when the estimated effects are nearly identical. Conversely, two studies can both be significant but estimate meaningfully different effects. Better assessments include:

• direction and magnitude of the replication estimate;
• whether the replication interval contains the original estimate and substantively important effects;
• a prediction interval derived from prior evidence;
• a meta-analysis of original and replication;
• equivalence tests or a smallest effect size of interest;
• Bayesian evidence for prespecified alternatives; and
• whether the operationalization actually tests the theoretical claim.

The credibility revolution

The crisis changed research practice. No single reform is sufficient; each blocks a different failure mode.

The Transparency and Openness Promotion Guidelines created graduated standards for citation, data, code, materials, design transparency, preregistration, and replication. Registered Reports move the crucial publication decision before outcomes are known. In an experiment with 353 reviewers, Registered Reports were rated higher on rigor, methods, analysis, and overall quality without being rated less novel or creative (Soderberg et al., 2021).

The better framing

The replication crisis is also evidence that science can learn about its own failure modes. The appropriate conclusion is neither blind trust nor blanket cynicism. It is calibrated trust in methods and converging evidence rather than prestige, novelty, or one beautiful result.

How to read a behavioral-science claim now

A twelve-question audit

1. What is the exact claim? Separate the measured result from the popular extrapolation.
2. Was the hypothesis and primary analysis preregistered? If not, treat the result as more exploratory.
3. How many outcomes, conditions, subgroups, and time points were available? Look beyond the one reported comparison.
4. Was the sample large enough for the effect that matters? A post hoc power claim based on the observed effect is not reassuring.
5. What is the effect size and uncertainty interval? Ask whether the interval includes effects too small to matter.
6. Were exclusions, stopping rules, transformations, and covariates fixed in advance? If they changed, are deviations disclosed and justified?
7. Are data, code, materials, and a clear provenance trail available? “Available on request” is weaker than an accessible archive.
8. Does the measure identify the construct? A manipulation check or self-report may not establish the behavioral, biological, or policy outcome used in the headline.
9. Has an independent, adequately powered replication been attempted? Distinguish direct from conceptual replication.
10. Is the result one study or a convergent program? Prefer multiple methods, teams, populations, and real-world outcomes.
11. Has the paper been corrected, flagged, or retracted? Check the journal page, not only a PDF or citation in a book.
12. What would lower confidence? If every possible outcome can be explained after the fact, the theory is not carrying much risk.

A practical evidence ladder

Common mistakes in talking about the crisis

“If it failed once, it is false”

Too strong. Ask how precise the replication was, whether it could detect the effect of interest, how faithfully it implemented the protocol, and what all studies estimate together.

“If it replicated once, it is true”

Also too strong. Both studies may share a confound, a biased measure, or an incorrect causal interpretation. Replication establishes consistency, not necessarily validity.

“P < .05 means a 95% chance the claim is true”

False. That posterior probability depends on prior plausibility, power, bias, the number of hypotheses searched, model assumptions, and the observed evidence.

“P-hacking is just fraud with statistics”

Sometimes it is deliberate deception; often it is a continuum of motivated and normalized practices. The remedy must therefore change workflows and incentives, not merely identify villains.

“Fraud explains the replication crisis”

Fraud produces spectacular cases but appears to account for a small share of published papers. Selection bias, low power, weak measurement, and undisclosed flexibility can distort an entire literature without anyone fabricating a row.

“Open data solves everything”

Open data allow checking, but a fabricated dataset can be openly shared, a reproducible analysis can answer the wrong question, and privacy restrictions can make openness inappropriate. Transparency is a means of scrutiny, not a truth certificate.

“The crisis means experts know nothing”

The failures are uneven. Some effects replicate reliably; some shrink; some are conditional; some collapse. The lesson is to weight claims by design and cumulative evidence instead of treating “peer reviewed” as a binary seal.

Timeline

Open questions

• What is the right success criterion when two estimates point the same way but one is nonsignificant?
• How much heterogeneity is a substantive boundary condition, and how much is noise or post hoc rescue?
• Which fields and claim types need routine independent replication before policy use?
• How can universities protect whistleblowers while preserving due process for the accused?
• Should journals require immutable raw-data deposits or cryptographic provenance for high-impact claims?
• How should co-authors divide responsibility when only one person possessed the raw data?
• How can corrective work receive enough career credit to compete with novelty?
• When is public naming justified before an institutional finding, and how should uncertainty be communicated?
• How much have reforms improved work published after the studies sampled by replication projects?
• What is an optimal replication rate? A field testing risky, novel ideas should not expect 100%, but it should reliably distinguish exploration from established knowledge.

Selected sources

Definitions, scale, and methods

• National Academies of Sciences, Engineering, and Medicine. Reproducibility and Replicability in Science (2019).
• Open Science Collaboration. “Estimating the Reproducibility of Psychological Science” (2015).
• Camerer et al. “Evaluating Replicability of Laboratory Experiments in Economics” (2016).
• Camerer et al. “Evaluating the Replicability of Social Science Experiments in Nature and Science” (2018).
• SCORE Collaboration. “Assessing Replicability of the Social and Behavioural Sciences” (2026).
• Miske et al. “Reproducibility of Social and Behavioural Science Research” (2026).
• Simmons, Nelson, and Simonsohn. “False-Positive Psychology” (2011).
• Ioannidis. “Why Most Published Research Findings Are False” (2005).
• John, Loewenstein, and Prelec. “Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling” (2012).
• Wasserstein and Lazar. “The ASA Statement on p-Values” (2016).
• Nosek et al. “Promoting an Open Research Culture” (2015).
• Munafò et al. “A Manifesto for Reproducible Science” (2017).

Cases and disputed findings

• Lewis-Kraus. “They Studied Dishonesty. Was Their Work a Lie?”, The New Yorker (2023). Narrative investigation and interviews; use the institutional reports and journal notices below for formal status.
• Kristal et al. “Signing at the Beginning Versus at the End Does Not Decrease Dishonesty” (2020), plus the 2026 correction.
• PNAS. Retraction of the 2012 signing-first paper (2021).
• Data Colada. Insurance-data analysis (2021) and Gino Study 1 analysis (2023). These are forensic allegations and analyses, not institutional rulings.
• Harvard Business School. Final Investigation Committee Report concerning Francesca Gino (2023; unsealed in litigation in 2024).
• Tilburg, Groningen, and Amsterdam committees. Flawed Science: The Fraudulent Research Practices of Social Psychologist Diederik Stapel (2012).
• Cornell University. Statement on the Brian Wansink academic-misconduct investigation (2018).
• U.S. Office of Research Integrity. 2012 Annual Report, including Marc Hauser findings.
• Science. Editorial retraction of LaCour and Green (2015).
• Psychological Science. Retraction of Ariely and Wertenbroch’s procrastination paper (2026).

Related notes

• Kahneman and Harris on motivated self-deception
• Psychology of Belief (MOC)
• Critical Thinking
• Skepticism 101
• Bayes’ theorem
• Extraordinary Claims Require Extraordinary Evidence