You are seeing this message because your Web browser does not support basic Web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.


ABOUT ARCHIVES
Advanced Search

Welcome   | My Account | E-mail Alerts | Access Rights | Sign In


  Vol. 150 No. 4, April 1996 TABLE OF CONTENTS
  Archives
  •  Online Features
  ARTICLE
 This Article
 •Send to a friend
 • Save in My Folder
 •Save to citation manager
 •Permissions
 Citing Articles
 •Citing articles on HighWire
 •Contact me when this article is cited
 Related Content
 •Similar articles in this journal

Improving participation and interrater agreement in scoring Ambulatory Pediatric Association abstracts. How well have we succeeded?

K. J. Kemper, P. L. McCarthy and D. V. Cicchetti
University of Washington and Swedish Medical Center, Seattle, USA.

OBJECTIVE: To determine whether increasing the number and types of raters affected interrater agreement in scoring abstracts submitted to the Ambulatory Pediatric Association. METHODS: In 1990, all abstracts were rated by each of the 11 members of the board of directors of the Ambulatory Pediatric Association. In 1995, abstracts were reviewed by four to five raters, including eight members of the board of directors, two chairpersons of special interest groups, and 10 regional chairpersons, for a total of 20 potential reviewers. Submissions were divided into the following three categories for review: emergency medicine, behavioral pediatrics, and general pediatrics. Weighted percentage agreement and weighted kappa scores were computed for 1990 and 1995 abstract scores. RESULTS: Between 1990 and 1995, the number of abstracts submitted to the Ambulatory Pediatric Association increased from 246 to 407, the number of reviewers increased from 11 to 20, the weighted percentage agreement between raters remained approximately 79%, and weighted kappa scores remained less than 0.25. Agreement was not significantly better for the emergency medicine and behavioral pediatrics abstracts than for general pediatrics, nor was it better for the raters who reviewed fewer abstracts than those who reviewed many. CONCLUSIONS: The number and expertise of those rating abstracts increased from 1990 to 1995. However, interrater agreement did not change and remained low. Further efforts are needed to improve the interrater agreement.

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Better reporting of randomized trials in biomedical journal and conference abstracts
Hopewell et al.
Journal of Information Science 2008;34:162-173.
ABSTRACT  

Peer Review Interrater Concordance of Scientific Abstracts: A Study of Anesthesiology Subspecialty and Component Societies.
Cohen and Patel
Anesth. Analg. 2006;102:1501-1503.
ABSTRACT | FULL TEXT  





HOME | CURRENT ISSUE | PAST ISSUES | TOPIC COLLECTIONS | SUBMIT | SUBSCRIBE | HELP
CONDITIONS OF USE | PRIVACY POLICY | CONTACT US | SITE MAP
 
© 1996 American Medical Association. All Rights Reserved.