 |
|
Does Problem-Based Learning Improve Residents' Self-directed Learning?
Arch Pediatr Adolesc Med. 2001;155:673-675.
THIS STUDY1 investigated the effect of
a problem-based learning (PBL) curriculum on self-directed learning behaviors
among a group of pediatric residents. Following an initial period of daily
lectures (lecture-based learning [LBL]), 80 residents were then divided into
2 groups: 1 group of residents (n = 39) underwent a period of twice-weekly
PBL sessions, while the other group (n = 41) continued receiving LBL. After
this intervention, all residents once again received a period of LBL. Throughout
each 3-month period (termed the pre-exposure, exposure, and follow-up periods,
respectively), residents were given self-administered questionnaires to evaluate
self-directed learning behaviors, including the amount of time they spent
on independent study, medical discussions, and computer literature searches.
The investigators found that while the PBL group reported significantly more
time spent in these self-directed endeavors during the exposure period, both
groups reported equal baseline levels durign the follow-up period. The investigators
concluded that residents exposed to PBL engaged in significantly higher levels
of self-directed learning than their counterparts.
We evaluated this study with respect to design, results, and analysis
according to the Journal of the American Medical Association users' guide format for a therapeutic trial.2, 3
This evaluation was done to assess the quality of the evidence, the strength
of the subsequent conclusions, the generalizability of the outcomes, and the
consideration of benefits vs risk and cost.
RANDOMIZATION OF SUBJECTS
This was a quasi-experimental design using residents' rotations to determine
group assignment. The PBL group consisted of residents on inpatient rotations
at the central residency site, while the LBL group consisted of residents
on other rotations. This raises questions regarding potential confounding
factors that were not accounted for in the analysis. For example, we do not
know any characteristics of the subjects in each group (ie, postgraduate training
level, differences in call schedules, other obligations or time commitments
associated with each rotation). It is generally acknowledged that interns
have less time for independent learning than do upper-level residents, and
residents with more frequent call schedules will have potentially less time
than others. The investigators did not control for or address any of these
issues in their article. In their defense, they did find that both groups
reported similar self-directed learning behaviors in the pre-exposure period,
despite being on inpatient vs outpatient rotations. However, simply because
both groups used similar durations for independent learning does not automatically
reflect whether one group may have had more opportunity or interest in pursuing
independent learning than the other.
ACCOUNTING FOR SUBJECTS
We asked the following questions: (1) Were all subjects who entered
the trial properly accounted for and attributed at its conclusion? (2) Was
follow-up complete? and (3) Were subjects analyzed in the groups to which
they were randomized? All subjects were accounted for and attributed at the
conclusion of the study as indicated by the statement that "complete data
were available on all 80 subjects." We are not told explicitly that subjects
were analyzed in the groups to which they were assigned initially, although
we assume this to be the case. Unfortunately, we were not able to deduce this
with certainty because results are reported only in the form of percentages,
without including the actual number of subjects for the intervention and control
groups.
BLINDED ASSESSMENT
The article states that the subjects were blinded to the hypothesis
of the study. It obviously would not be possible to blind them to the intervention
itself. Yet it is unclear whether the faculty members leading the sessions
were aware of the hypothesis, and whether those analyzing the data (subject
questionnaires) were blinded as well. The specific format of the questionnaire
used to evaluate self-directed learning behaviors was not included in this
article. Nevertheless, it is possible that residents receiving questions about
learning habits in the setting of a "new" format of lectures would be more
conscientious of and potentially biased in their responses. One possible solution
would have been to add sham questions to the questionnaire in an attempt to
avert any transparency bias.
In addition, if the lecturers in the 2 groups were aware of the goals
of the study, it may have influenced their effort and performance in the teaching
sessions, via the Hawthorne effect. Lecturers may have been motivated to increase
self-directed learning in the intervention group, and to de-emphasize it in
the control group.
Finally, it is important that the evaluators of the questionnaires be
blinded to the group assignment. The more objective the outcome data, the
less critical this issue becomes. As stated earlier, the specific questions
in the questionnaire were not included in the article; thus, it is impossible
to assess how the evaluators interpreted the data. For example, if residents
were told to write in the number of hours they spent on independent study,
any ambiguity in the penmanship could potentially lead to bias in interpretation.
However, if the questionnaires included numbers that subjects circled, there
would be a lower potential for bias of interpretation.
SIMILARITY OF GROUPS
There were no statistically significant differences between the 2 groups
at the start of the trial with respect to the outcomes of interest. However,
the groups are not compared with respect to stage of training or mean hours
of work per week. As we mentioned before, these could be potential confounders
or effect modifiers.
WERE THE GROUPS TREATED EQUALLY?
There was no additional intervention offered to either group, although,
because this was not a randomized controlled trial, there were likely to be
significant differences in other variables, such as patient contact hours,
sleep deprivation, and the complexity of patients encountered. Some of these
potential confounders might serve to bias toward the null hypothesis (eg,
working longer hours, as might be expected on the inpatient wards, might decrease
the incentive to conduct literature searches). However, other factors, such
as the added complexity of inpatients, might serve to increase the incentive
to do so.
TREATMENT EFFECT SIZE
The study found a statistically significant increase in several measures
of self-directed learning among the PBL group vs the LBL group during the
intervention period. These included more residents reporting 5 or more hours
of study, 5 or more hours of medical discussions, and 2 or more computer literature
searches, with P<.01. The number needed to treat
(NNT) for 5 or more hours of study is approximately 5, indicating that 5 residents
would need to undergo PBL for each additional resident to demonstrate this
behavior. The NNT for 5 or more hours of medical discussion and 2 or more
computer literature searches are 4 and 5, respectively.
It is unclear whether the increased amount of self-directed learning
was part of the work assigned in the PBL sessions or in addition to these
assignments. As described in the "Materials and Methods" section, residents
in the PBL group attended weekly meetings to work through various medical
cases. The first meeting of each case introduced the clinical scenario, ending
in "self-assignment of the learning issues" to investigate further information
necessary to develop an appropriate differential diagnosis and plan of care,
which was discussed at the next meeting. If the results indicating increased
self-directed learning among these residents with respect to reading, research,
and discussion were because of these assignments, then it is no surprise that
this behavior disappeared after the intervention was completed. That is to
say, once the assignments were discontinued, the impetus for pursuing more
self-directed study returned to baseline.
TREATMENT EFFECT PRECISION
The precision of the treatment effect is determined by the confidence
interval (CI) reported in the analysis. While CIs were not provided in this
study, P values were included, and results were judged
to be statistically significant with P<.01. This
implies a 99% confidence that the results were not due to chance alone.
GENERALIZABILITY
It seems that these results can be generalized to other residency programs,
although the overall benefit would be somewhat questionable in some very small
residency programs given the calculated NNT (roughly 5).
CLINICALLY IMPORTANT OUTCOMES
This study did not demonstrate any long-term benefit to PBL, as evidenced
by the return to baseline of self-directed learning in the follow-up period.
Results from the study showed that self-directed learning behaviors in the
follow-up period were comparable to both the pre-exposure period and between
the PBL and LBL groups. This indicates that while exposure to ongoing PBL
demonstrated increased independent learning, once this format was discontinued
(as would be the case when a resident completes the residency program) the
subjects did not maintain this increased level of independent learning. The
authors acknowledged the apparent short-lived gains in self-directed learning
behavior, as well as the question as to whether residents would benefit from
more prolonged or ongoing exposure to PBL to sustain these improved behaviors.
BENEFITS VS COSTS
The likely benefits of PBL seem to be worth any potential harms or costs.
One would definitely need to take into account the cost of training instructors
in PBL, and the potential changes in resident occupational and recreational
schedules (and quality of life). In addition, further study into maintaining
long-term increases in self-directed learning (implicit in a change in attitude
toward independent learning that would continue throughout one's career) would
need to be carried out to fully assess the benefit of this intervention vs
the associated risk or cost.
CONCLUSION
Despite various limitations in the design of the investigation, the
results of this study have several important implications for the future training
of physicians nationwide. Despite the questionable benefit of PBL in terms
of persistent change in residents' self-directed learning behaviors, the potential
benefits of PBL in any residency program would seem to outweigh the minimal
risks and costs. Further investigation is warranted to study the long-term
effects of PBL— in particular, focusing on whether additional or more
intensive exposure could sustain the increase in self-directed learning beyond
the intervention period alone.
Naya Juul-Dam, MD;
Stephanie Brunner, MD;
Rachel Katzenellenbogen, MD;
Michael Silverstein, MD
Seattle
Dimitri A. Christakis, MD, MPH
Department of Pediatrics
Child Health Institute
University of Washington
146 N Canal St, Suite 300
Seattle, WA, 98103
REFERENCES
1. Ozuah PO, Curtis J, Stein REK. Impact of problem-based learning on residents' self-directed learning. Arch Pediatr Adolesc Med. 2001;155:669-672.
FREE FULL TEXT
2. Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature, II: how to use an article
about therapy or prevention, B: what were the results and will they help me
in caring for my patients? Evidence-Based Medicine Working Group. JAMA. 1994;271:59-63.
FREE FULL TEXT
3. Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature, II: how to use an article
about therapy or prevention, A: are the results of the study valid? Evidence-Based
Medicine Working Group. JAMA. 1993;270:2598-2601.
FREE FULL TEXT
RELATED ARTICLES
Can We Train a Lifelong Learner?
Thomas DeWitt
Arch Pediatr Adolesc Med. 2001;155(6):637-638.
EXTRACT
| FULL TEXT
Evidence-Based Medicine Journal Club: What's It All About and What's in It for Me?
Harold P. Lehmann
Arch Pediatr Adolesc Med. 2001;155(6):639-640.
EXTRACT
| FULL TEXT
Impact of Problem-Based Learning on Residents' Self-directed Learning
Philip O. Ozuah, Jane Curtis, and Ruth E. K. Stein
Arch Pediatr Adolesc Med. 2001;155(6):669-672.
ABSTRACT
| FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
More on Problem-Based Learning and Self-directed Learning
Ozuah and Stein
Arch Pediatr Adolesc Med 2001;155:1278-1278.
FULL TEXT
Evidence-Based Medicine Journal Club: What's It All About and What's in It for Me?
Lehmann
Arch Pediatr Adolesc Med 2001;155:639-640.
FULL TEXT
|
