František Bartoš, Alexandra Sarafoglou, Balazs Aczel, Suzanne Hoogeveen, Christopher D. Chambers, Eric-Jan Wagenmakers

J. Robust. Rep. 0-Editorial (2025) · published 16 April 2025 | Toggle abstract · pdf

The vast majority of empirical research articles report a single primary analysis outcome that is the result of a single analysis plan, executed by a single analysis team (usually the team that also designed the experiment and collected the data). However, recent many-analyst projects have demonstrated that different analysis teams generally adopt a unique approach and that there exists considerable variability in the associated conclusions. There appears to be no single optimal statistical analysis plan, and different plausible plans need not lead to the same conclusion. A high variability in outcomes signals that the conclusions are relatively fragile and dependent on the specifics of the analysis plan. Crucially, without multiple teams analyzing the data, it is difficult to gauge the extent to which the conclusions are robust. We have recently proposed that empirical articles of particular scientific interest or societal importance are accompanied by two or three short reports that summarize the results of alternative analyses conducted by independent experts [F. Bartoš et al., Nat. Hum. Behav. (2025)]. In order to showcase the practical feasibility and epistemic benefits of this approach we have founded the Journal of Robustness Reports, which is dedicated to publishing short reanalyses of empirical findings. This editorial describes the scope and the workflow of the Journal of Robustness Reports including the type and format of the published articles. We hope that the Journal of Robustness Reports will help make reanalyses of published findings the norm across the empirical sciences.

Alexandra Sarafoglou

J. Robust. Rep. 0-Ex2 (2025) · published 16 April 2025 | Toggle abstract · pdf

Although participants' taste scores were influenced by their preferences, they still generally favored Chaudfontaine over tap water, which confirms the original claim.

Suzanne Hoogeveen

J. Robust. Rep. 0-Ex1 (2025) · published 16 April 2025 | Toggle abstract · pdf

Varying data inclusion criteria and prior settings, a Bayesian multiverse analysis failed to provide robust evidence for the claim that people prefer Chaudfontaine over tap water.

Hannah E. Fowles, Peter J. Allen

J. Robust. Rep. 3-rr1 (2026) · published 26 March 2026 | Toggle abstract · pdf

An objective, neurological effect of a non-deceptive placebo observed in [D. A. Guevarra et al., Nat. Commun. 11, 3785 (2020)] is robust to age sub-sampling and electrode selection. However, it seems likely to be a false positive.

Thomas S. A. Wallis, Joshua M. Martin

J. Robust. Rep. 1-rr1 (2026) · published 24 March 2026 | Toggle abstract · pdf

A Bayesian mixed-effects analysis found no evidence that patients treated with cataract surgery rely more on color cues for object recognition compared to controls.