Peer review research assessment: are the reviewers really experts?

This article examines the expertise of peer reviewers involved in Italy’s national research assessment exercise, the Valutazione della Qualità della Ricerca (VQR) 2020–24, focusing on the alignment between reviewers’ disciplinary specialization and the research outputs they evaluate, as well as their scientific performance measured by a normalized citation impact indicator. Findings reveal significant variability in disciplinary coverage, with some Scientific Disciplinary Sectors (SDSs) underrepresented or lacking panel members, leading to reliance on external reviewers or mismatched expertise. While nearly half of the reviewers rank in the top 30% nationally by research impact, a notable minority fall below the median, and reviewers appointed directly by the National Agency for the Evaluation of Universities and Research (ANVUR) generally outperform those selected by lottery, though both groups include less qualified individuals. The study highlights concerns about excessive reviewer workloads, opaque selection processes, and the tension between fairness and expertise, suggesting that without reforms to ensure transparent, evidence-based reviewer selection and manageable evaluation demands, the credibility and effectiveness of large-scale peer review in national research assessments may be compromised.