Key result
Peer review of scientific abstracts for anesthesiology subspecialty meetings demonstrated poor to moderate interrater agreement, suggesting a need for clearer evaluation criteria and interval scales.
Observational
There is poor to moderate agreement among reviewers for anesthesiology subspecialty meeting abstracts, suggesting a need for clearer evaluation criteria and scoring systems.
In Brief Abstracts presented at anesthesiology subspeciality and component society meetings are chosen by peer review. We assessed this process by examining selection criteria and determining interrater concordance. For the societies studied, the level of reviewer agreement ranged from poor to moderate, i.e., slightly better than by chance alone. We hypothesize that having clearer evaluation criteria, scoring systems with interval scales, and assessment based on quality can strengthen the peer review process. IMPLICATIONS: We evaluated the selection process for subspecialty meeting abstracts. There was poor to moderate agreement among reviewers. We suggest that the review process can be improved by using clear evaluation criteria and scoring abstracts using interval scales.
No takes yet. Share an insight, caveat, or question.
Cohen et al. (2006) conducted an observational in Peer review of scientific abstracts. Peer review process was evaluated on Interrater concordance. Peer review of scientific abstracts for anesthesiology subspecialty meetings demonstrated poor to moderate interrater agreement, suggesting a need for clearer evaluation criteria and interval scales.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: