The evaluation of scholarly work by others in the same field before publication. It is the central quality mechanism of academic publishing, it is much younger than most people assume, and its limitations are well documented.
An author submits a manuscript to a journal. An editor assesses whether it is within scope and of plausible quality, and rejects a substantial proportion at this stage without external review.
Surviving manuscripts are sent to two or more reviewers with relevant expertise, who assess the significance of the question, the soundness of the methods, whether the conclusions follow from the results, and whether the work is placed correctly in the existing literature.
Reviewers recommend acceptance, revision or rejection, and the editor decides. Most published papers go through at least one round of revision.
Reviewers are almost always unpaid, and reviewing is treated as a professional obligation in exchange for having one's own work reviewed.
Several models exist. Single-blind review conceals reviewer identity from authors. Double-blind conceals both. Open review discloses both and sometimes publishes the reports. Post-publication review makes the work available first and collects evaluation afterwards.
Peer review is often described as a timeless feature of science, and it is not.
Early scientific journals were edited by individuals or societies who selected what to publish largely on their own judgement. The Philosophical Transactions of the Royal Society, founded in 1665, used a form of review by council members, and this was unusual.
For most of the history of science, publication decisions rested with editors. Einstein published his 1905 papers in Annalen der Physik without external peer review, and reportedly objected strongly when an American journal sent one of his later papers to a referee.
Systematic external peer review became standard only in the mid-twentieth century, driven by growth in submissions, increasing specialisation that outran editors' expertise, and the requirements of research funders.
Almost all the science of the seventeenth to early twentieth centuries, including most of what is regarded as foundational, was published without it.
It filters out work that is clearly flawed in method or reasoning, and this is its most reliable function.
It improves papers. Revision in response to review generally produces clearer, better-qualified work, and this is frequently cited by authors as its main value.
It does not verify results. Reviewers read a manuscript; they do not repeat the experiments, inspect the raw data in most cases, or check the analysis independently. A well-written paper describing work that was never done will generally pass.
It does not reliably detect fraud. Several major fabrication cases passed review at prestigious journals repeatedly, and were exposed afterwards by readers, replication attempts or whistleblowers.
It does not guarantee importance or correctness. Published papers are retracted regularly, and the replication problems described in the replication crisis capsule concern work that passed review.
Reliability between reviewers is low. Studies of agreement find that two reviewers assessing the same manuscript agree only slightly more than chance would produce, which is a serious finding for a filtering mechanism.
Bias has been demonstrated experimentally. Studies resubmitting published papers with altered author names and institutions have found acceptance influenced by prestige, and analyses have found effects associated with author gender and nationality, though the size varies between fields and studies.
Conservatism is a recurring criticism. Reviewers are established researchers, and work challenging existing consensus faces a higher bar, which several accounts of delayed acceptance illustrate.
It is slow. Months between submission and publication are normal and a year is not unusual, which delays dissemination.
It is invisible labour. Reviewing is unpaid work performed for journals that are frequently commercially profitable, which is a persistent grievance and became the basis of organised boycotts.
Predatory journals exploit the appearance of the system, charging fees and performing no meaningful review, and a number of investigations have had deliberately nonsensical papers accepted by them.
Preprint servers allow work to be posted before review, which is now standard in physics, mathematics and increasingly in biology and medicine. This restores something like the older model of publish first and evaluate afterwards, with the review then occurring in public.
Open peer review, publishing reports alongside papers, is used by a growing number of journals and makes the basis of a decision inspectable. The Homo naledi burial dispute treated in its own capsule was conducted almost entirely in public through this mechanism.
Registered reports invert the sequence: the question and method are reviewed and accepted before results exist, which removes the incentive to produce publishable outcomes and directly addresses publication bias.
Post-publication commentary, on platforms dedicated to it, has exposed problems in published work that review did not catch.
Peer review is the mechanism the public and the press treat as the boundary between reliable and unreliable claims, and the phrase peer-reviewed carries substantial authority in argument.
That authority exceeds what the process delivers. It is a screen applied by two or three busy people reading a description of work they cannot verify, and it is better understood as the first filter in a longer process of scrutiny than as a certification of correctness.
