This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on Web of Science (36)  • Contact me when this article is cited  Related Content  • Similar articles in this journal  Topic Collections  • Medical Practice  • Medical Education  • Alert me on articles by topic  Social Bookmarking   What's this?

A Step in the Direction of Competency-Based Evaluation

Carol Carraccio, MD; Robert Englander, MD, MPH

Arch Pediatr Adolesc Med. 2000;154:736-741.

Background  The Accreditation Council for Graduate Medical Education is embarking on the major task of a paradigm shift in graduate education in the direction of competency-based medical education and evaluation of outcomes. The Objective Structured Clinical Examination (OSCE), a measure of clinical competence that focuses on outcomes via observable behaviors, is gaining national recognition.

Objective  To review the pediatric literature relevant to the OSCE.

Method  A MEDLINE search from the date of the original report of the OSCE (1975) to the present was performed. All English-language studies regarding the use of the OSCE in pediatric education published in the United States and Great Britain were reviewed.

Main Outcome Measures  Reliability and validity of the OSCE were examined. Use of standardized pediatric patients was discussed.

Results  A greater number of stations and similarity between tasks at different stations increased the reliability of the OSCE. A greater number of stations increased sampling of material and content validity. Correlation between the OSCE and precertification examinations ranged between 0.59 and 0.71, with P.01. Correlation between the OSCE and monthly clinical evaluations was much lower (0.39-0.57), but still statistically significant at P.05. Gaps between expected and actual performance were documented. Overall, the experience of being a standardized patient was viewed as positive by children and their parents.

Conclusions  With appropriate attention to design, acceptable reliability and validity can be achieved for the OSCE. Significant correlations between the OSCE and precertification examinations as well as monthly clinical evaluations were found, the former being stronger than the latter. We conclude that the combination of the OSCE, standardized board examinations, and direct observation in the clinical setting has the potential to become the "gold standard" for measuring physician competence.

From the University of Maryland School of Medicine, Baltimore.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Assessing Dental Students' Competence: Best Practice Recommendations in the Performance Assessment Literature and Investigation of Current Practices in Predoctoral Dental Education
Albino et al.
J Dent Educ 2008;72:1405-1435.
ABSTRACT | FULL TEXT  

Continuity-Structured Clinical Observations: Assessing the Multiple-Observer Evaluation in a Pediatric Resident Continuity Clinic
Zimmer et al.
Pediatrics 2008;121:e1633-e1645.
ABSTRACT | FULL TEXT  

Using children as standardised patients for assessing clinical competence in paediatrics
Tsai
Arch. Dis. Child. 2004;89:1117-1120.
ABSTRACT | FULL TEXT  

The State of Pediatrics Residency Training: A Period of Transformation of Graduate Medical Education
Sectish et al.
Pediatrics 2004;114:832-841.
ABSTRACT | FULL TEXT  

Reference Standards, Judges, and Comparison Subjects: Roles for Experts in Evaluating System Performance
Hripcsak and Wilcox
J. Am. Med. Inform. Assoc. 2002;9:1-15.
ABSTRACT | FULL TEXT