How much 'better' is good enough? The magnitude of treatment effect in clinical trials
T. N. Raju, P. Langenberg, A. Sen and O. Aldana
Department of Pediatrics, University of Illinois, Chicago 60612.
OBJECTIVES--Among the various factors required to calculate sample size for
clinical trials, the magnitude of treatment effect anticipated is an
important component. The objective of this report is to present some of the
complexities involved in selection of treatment effect size in clinical
trials. As a framework for discussion, an analysis of published reports
related to surfactant therapy was carried out. DESIGN--Twenty-one
consecutive exogenous surfactant trials for neonatal respiratory distress
syndrome were analyzed. The "Methods" sections were reviewed for evaluating
various components of sample size calculation, including the anticipated
treatment effect size. RESULTS--Sixteen (76%) of the 21 reports provided a
description of sample size calculations, and 12 of these gave some reasons
for the choice of the anticipated treatment effect size. Expressed as
percent change, the median treatment effect from intervention anticipated
by the investigators was 50% (range, 15% to 90%), with a positively skewed
distribution. The actual median percent reduction in adverse events from
treatment (as compared with baseline) was 36% (range, 75% reduction to 5%
excess). When the treatment effect was expressed as difference in adverse
event rate, in the 14 (of 16) trials that could be analyzed, the median
observed reduction in adverse events (death, bronchopulmonary dysplasia, or
occurrence of respiratory distress syndrome) was 14.5% (range, 52%
reduction to 2% excess). All trials except one concluded, however, that the
intervention was effective, mostly based on additional subgroup
calculations. CONCLUSIONS--Researchers often select sample sizes capable of
detecting only large treatment effects, thus risking type II error,
although sometimes a much smaller effect could be clinically important.
While pragmatic considerations must be considered during the design of
randomized clinical trials, researchers ought to present a rationale for
anticipating a given magnitude of treatment effect in their sample size
calculations. It may be possible to consider innovative trial designs that
help determine the most appropriate treatment choice with the least
possible sample size.
