You are seeing this message because your Web browser does not support basic Web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.


Advertisement

ABOUT ARCHIVES
Advanced Search

Welcome   | My Account | E-mail Alerts | RSS | Access Rights | Sign In


  Vol. 150 No. 4, April 1996 TABLE OF CONTENTS
  Online Only
 •  Online First Table of
Contents
  ARTICLES
 •Online Features
 This Article
 •References
 •Full text PDF
 • Reply to article
 •Send to a friend
 • Save in My Folder
 •Save to citation manager
 •Permissions
 Citing Articles
 •Citation map
 •Citing articles on HighWire
 •Citing articles on Web of Science (8)
 •Contact me when this article is cited
 Related Content
 •Similar articles in this journal
 Social Bookmarking
  Add to CiteULike Add to Connotea Add to Delicious Add to Digg Add to Facebook Add to Reddit Add to Technorati Add to Twitter What's this?

Improving Participation and Interrater Agreement in Scoring Ambulatory Pediatric Association Abstracts

How Well Have We Succeeded?

Kathi J. Kemper, MD, MPH; Paul L. McCarthy, MD; Domenic V. Cicchetti, PhD

Arch Pediatr Adolesc Med. 1996;150(4):380-383.


Abstract



Objective
To determine whether increasing the number and types of raters affected interrater agreement in scoring abstracts submitted to the Ambulatory Pediatric Association.

Methods
In 1990, all abstracts were rated by each of the 11 members of the board of directors of the Ambulatory Pediatric Association. In 1995, abstracts were reviewed by four to five raters, including eight members of the board of directors, two chairpersons of special interest groups, and 10 regional chairpersons, for a total of 20 potential reviewers. Submissions were divided into the following three categories for review: emergency medicine, behavioral pediatrics, and general pediatrics. Weighted percentage agreement and weighted {kappa} scores were computed for 1990 and 1995 abstract scores.

Results
Between 1990 and 1995, the number of abstracts submitted to the Ambulatory Pediatric Association increased from 246 to 407, the number of reviewers increased from 11 to 20, the weighted percentage agreement between raters remained approximately 79%, and weighted {kappa} scores remained less than 0.25. Agreement was not significantly better for the emergency medicine and behavioral pediatrics abstracts than for general pediatrics, nor was it better for the raters who reviewed fewer abstracts than those who reviewed many.

Conclusions
The number and expertise of those rating abstracts increased from 1990 to 1995. However, interrater agreement did not change and remained low. Further efforts are needed to improve the interrater agreement.

(Arch Pediatr Adolesc Med. 1996;150:380-383)



Author Affiliations



From the University of Washington and Swedish Medical Center, Seattle (Dr Kemper); and the Yale University School of Medicine (Dr McCarthy) and Veterans Affairs Medical Center (Dr Cicchetti), West Haven, Conn.



Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Delicious Delicious   Add to Digg Digg   Add to Facebook Facebook   Add to Reddit Reddit   Add to Technorati Technorati   Add to Twitter Twitter     What's this?

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Better reporting of randomized trials in biomedical journal and conference abstracts
Hopewell et al.
Journal of Information Science 2008;34:162-173.
ABSTRACT  

Peer Review Interrater Concordance of Scientific Abstracts: A Study of Anesthesiology Subspecialty and Component Societies.
Cohen and Patel
Anesth. Analg. 2006;102:1501-1503.
ABSTRACT | FULL TEXT  





HOME | CURRENT ISSUE | PAST ISSUES | TOPIC COLLECTIONS | CME | PHYSICIAN JOBS | SUBMIT | SUBSCRIBE | HELP
CONDITIONS OF USE | PRIVACY POLICY | CONTACT US | SITE MAP
 
© 1996 American Medical Association. All Rights Reserved.