 |
|
Are Overreferrals on Developmental Screening Tests Really a Problem?
Frances Page Glascoe, PhD
Arch Pediatr Adolesc Med. 2001;155:54-59.
ABSTRACT
| |
Background Developmental screening tests, even those meeting standards for screening
test accuracy, produce numerous false-positive results for 15% to 30% of children.
This is thought to produce unnecessary referrals for diagnostic testing or
special services and increase the cost of screening programs.
Objectives To explore whether children who pass screening tests differ in important
ways from those who do not and to determine whether children overreferred
for testing benefit from the scrutiny of diagnostic testing and treatment
planning.
Methods Subjects were a national sample of 512 parents and their children (age
range of the children, 7 months to 8 years) who participated in validation
studies of various screening tests. Psychological examiners adhering to standardized
directions obtained informed consent and administered at least 2 developmental
screening measures (the Brigance Screens, the Battelle Developmental Inventory
Screening Test, the Denver-II, and the Parents' Evaluations of Developmental
Status) and a concurrent battery of diagnostic measures, including tests of
intelligence, language, and academic achievement (for children aged 2
years and older). The performance on diagnostic measures of children who failed
screening but were not found to have a disability (false positives) was compared
with that of children who passed screening and did not have a disability on
diagnostic testing (true negatives).
Results Children with false-positive scores performed significantly (P<.001) lower on diagnostic measures than did children with true-negative
scores. The false-positive group had scores in adaptive behavior, language,
intelligence, and academic achievement that were 9 to 14 points lower than
the scores of those in the true-negative group. When viewing the likelihood
of scoring below the 25th percentile on diagnostic measures, children with
false-positive scores had a relative risk of 2.6 in adaptive behavior (95%
confidence interval [CI], 1.67-4.21), 3.1 in language skills (95% CI, 1.90-5.20),
6.7 on intelligence tests (95% CI, 3.28-13.50), and 4.9 on academic measures
(95% CI, 2.61-9.28). Overall, 151 (70%) of the children with false-positive
results scored below the 25th percentile on 1 or more diagnostic measures
(the point at which most children have difficulty benefiting from typical
classroom instruction) in contrast with 64 (29%) of the children with true-negative
scores (odds ratio, 5.6; 95% CI, 3.73-8.49). Children with false-positive
scores were also more likely to be nonwhite and to have parents who had not
graduated from high school. Performance differences between children with
true-negative scores and children with false-positive scores continued to
be significant (P<.001) even after adjusting for
sociodemographic differences between groups.
Conclusions Children overreferred for diagnostic testing by developmental screens
perform substantially lower than children with true-negative scores on measures
of intelligence, language, and academic achievement—the 3 best predictors
of school success. These children also carry more psychosocial risk factors,
such as limited parental education and minority status. Thus, children with
false-positive screening results are an at-risk group for whom diagnostic
testing may not be an unnecessary expense but rather a beneficial and needed
service that can help focus intervention efforts. Although such testing will
not indicate a need for special education placement, it can be useful in identifying
children's needs for other programs known to improve language, cognitive,
and academic skills, such as Head Start, Title I services, tutoring, private
speech-language therapy, and quality day care.
INTRODUCTION
SCREENING TESTS, even those that meet standards for developmental screening
test accuracy, produce failing scores for 15% to 30% of children who, on diagnostic
testing, are not found to have disabilities.1, 2
Such false-positive results are thought to substantially increase the cost
of screening.3 Indeed, some researchers4 suggest that screening programs should be discontinued
when false-positive rates are high. Because false-positive medical screens
have been associated with lingering parental anxiety,5, 6
troubling doubts are cast on the viability of universal developmental screening
efforts. Concerns about the costs, mistakes, and effects of screening are
implicated in the limited use of developmental measures among physicians,
in opposition to recommendations for routine use of standardized tools by
the American Academy of Pediatrics' Committee on Children with Disabilities.7, 8, 9
Such gloomy conclusions about the value of screening illuminate critical
questions for research. Are children overreferred by developmental screening
tests actually normal or do they differ in important ways from children who
pass screening tests? If so, are diagnostic workups on children with false-positive
scores truly unnecessary or can they contribute meaningfully to patient care?
These questions are addressed in the present study by reanalyzing existing
data from screening test validation studies.
SUBJECTS AND METHODS
Subjects were a national sample of 512 parents and their children (age
range of the children, 7 months to 8 years; mean, 52.7 months; SD, 19.78 months)
participating in validation studies of various screening tests.2, 10, 11, 12, 13, 14, 15
Subjects were 61% white, 23% African American, and 16% Hispanic or other ethnicity;
53% were male. Parents averaged 13 completed grades of school, and 14% had
not graduated from high school. In comparing these characteristics with US
population variables, the sample approximated national representativeness
for minorities, levels of parental education, and family socioeconomic status.16
Sites included 4 day care centers (104 children) and 4 public school
systems, including school-based Head Start and Even Start programs (408 children).
Sites were selected to represent the main geographic regions of the United
States: north (Plymouth, Mass), central (Denver, Colo, which is within 400
km of the geographic epicenter of the United States), south (Tampa, Fla, and
Nashville, Tenn), and west (Carson City, Neb). Within each site, schools and
programs were selected if they had a mix of children from various socioeconomic
backgrounds, as determined by proportions participating in the federal free
and reduced cost lunch program or by the presence or absence of federal day
care subsidies.
At each site, psychological examiners or teachers recruited families
largely by sending children home with consent forms and study materials. Nine
families failed to return consent forms and were excluded from the study.
Examiners were graduate-level school psychologists or educational diagnosticians
skilled in test administration. Adhering to standardized directions, examiners
administered developmental screening measures, ie, the Brigance Screens (n
= 408),17 the Battelle Developmental Inventory
Screening Test (n = 103),18 the Denver-II (n
= 103),19 and/or the Parents' Evaluations of
Developmental Status (n = 511).20 Table 1 provides a brief description of
each tool. All children except 1 were administered at least 2 developmental
screening tests. Examiners were blinded to the goals of the study and to the
results of 2 of the 4 screens (Parents' Evaluations of Developmental Status
and Brigance Screens) because both were undergoing standardization and produced
no normative scores.
|
|
|
|
Table 1. List of Measures
|
|
|
Examiners also administered and scored in a standardized manner a battery
of diagnostic measures. Screening tests and diagnostic measures were administered
within 1 week and were given in alternating order. Diagnostic measures included
tests of intelligence, language, and academic achievement (for children aged
2 years and older). Test selection varied across and within studies
based on children's ages, and instruments are listed in Table 1.
Criteria drawn from the Individuals With Disabilities Education Act
for placement in early childhood and public school special education programs
were applied to performance on diagnostic measures. These criteria were used
because they are functional, reflect observable difficulties benefiting from
age-appropriate instruction, and are linked to receipt of actual services. Table 2 shows the criteria used to determine
the presence of disabilities.
|
|
|
|
Table 2. Criteria for Determining the Presence of Disabilities: Criteria
for Special Education Placement*
|
|
|
The performance on diagnostic measures of children who failed screening
but were not found to have a disability (false positives) was compared with
that of children who passed screenings and did not have a disability on diagnostic
testing (true negatives). t Tests,  2
tests, analysis of covariance, and relative risk estimates were used to compare
differences between groups.
RESULTS
Of the 511 children administered at least 2 screening tests, 216 (42%)
had false-positive results (they failed 1 or more of the screens but were
not found to have disabilities) and 219 (43%) had true-negative results (they
passed all screens and were not found to have disabilities). Of the remaining
76, 44 (9%) had true-positive results (they failed 1 or more screens and had
disabilities) and 32 (6%) had false-negative results (they passed all screens
but had disabilities). The false-positive and false-negative rates are higher
than those found in most screening test studies because they reflect the combined
errors across all screens. Table 3
shows the rates for each screen. In comparing only those with 1 or more false-positive
scores with those with true-negative scores, children with false-positive
scores performed significantly lower on diagnostic measures of adaptive behavior
(t428 = 5.56; P<.001),
language (t429 = 6.96; P<.001), intelligence (t434
= 9.47; P<.001), and academic achievement (t348 = 7.57; P<.001).
The quotients produced for each measure, ie, standard scores with means of
100 and SDs of 15, averaged 9 to 14 points lower for children with false-positive
scores than for children with true-negative scores (>7 points difference represents
half of an SD and an ecologically significant difference in classroom performance).
When viewing the likelihood of scoring below the 25th percentile on diagnostic
measures (the cutoff used for placement in remedial reading and math programs
under Title I), children with false-positive scores had a relative risk of
2.6 in adaptive behavior (95% confidence interval [CI], 1.67-4.21) (42% vs
19% in the true-negative group), 3.1 in language skills (95% CI, 1.90-5.20)
(32% vs 11% in the true-negative group), 6.7 on intelligence tests (95% CI,
3.28-13.50) (26% vs 9% in the true-negative group), and 4.9 on academic measures
(95% CI, 2.61-9.28) (35% vs 9% in the true-negative group). Overall, 70% (n
= 151) of the children with false-positive results scored below the 25th percentile
on 1 or more diagnostic measures, in contrast with 29% (n = 64) of the children
with true-negative scores (odds ratio, 5.6; 95% CI, 3.73-8.49). Table 4 shows performance on diagnostic measures across all 4 groups.
|
|
|
|
Table 3. Negative and Positive Results Across Screening Tests*
|
|
|
|
|
|
|
Table 4. Comparison of Children's Performance on Diagnostic Measures
Across Screening Outcomes
|
|
|
Do children with false-positive scores differ in other ways from children
with true-negative scores? To test this, children with false-positive scores
were compared on sociodemographic variables with children with true-negative
scores. Children with false-positive scores were an average of 4 months older
than those with true-negative scores (t434
= 2.57; P>.01), were more likely to be nonwhite (2 = 22.35; P<.001) (47% vs 25% in the true-negative
group), and were more likely to have parents who had not graduated from high
school (2 = 5.09; P<.02) (16%
vs 9% in the true-negative group). Even after adjusting (via analysis of covariance)
for sociodemographic differences between groups, performance on adaptive behavior,
language, intelligence, and academic achievement of children with false-positive
scores continued to be significantly lower than that of children with true-negative
scores (F5,334 = 29.41, 22.43, 21.47, and 12.56, respectively; P<.001).
COMMENT
Children overreferred by developmental screens perform substantially
lower than children with true-negative scores on measures of intelligence,
language, and academic achievement—the 3 best predictors of school success.
Overreferred children also carry more psychosocial risk factors, such as limited
parental education and minority status.21 Thus,
children with false-positive screening results are clearly an at-risk and
underperforming group for whom diagnostic testing appears less an unnecessary
expense and more a potentially beneficial service to the extent that testing
is linked to needed intervention. Although such testing will not indicate
a need for special education placement, it can be useful in determining educational
objectives, individualizing instruction, and identifying programs known to
improve language, cognitive, and academic skills, such as Head Start, Title
I services, tutoring, private speech-language therapy, and quality day care. Table 5 provides a case study of the value
of diagnostic testing with a child who had false-positive results on screening.
|
|
|
|
Table 5. Case Study on the Use of Diagnostic Testing in Response to
False-Positive Screening Results*
|
|
|
Because of the scarcity of diagnostic resources and the economic constraints
of health care and education, it is worth noting that when physicians make
referrals for diagnostic testing (eg, to developmental evaluation centers
or to public schools), there are usually mechanisms in place that cull children
who are likely to have false-positive results. For example, the public schools,
under section 504 of the Rehabilitation Act of 1973, must devise needed modifications
in the classroom and then view the viability of these on student performance
before considering full developmental assessments. Similarly, under the Individuals
With Disabilities Education Act part C (services for children from birth to
the age of 3 years) and in most developmental evaluation centers, intake workers
usually identify and refer likely at-risk but not disabled children to programs
that do not require diagnostic eligibility (parenting classes, behavior modification
training, quality preschool programs, and Head Start). Part C funding also
provides for monitoring before providing diagnostic testing. Such approaches
help ensure that precious diagnostic resources are allocated parsimoniously
while still enabling at-risk children with false-positive scores on screening
tests to receive needed, albeit limited, services. Thus, the knowledge that
developmental screens produce many false-positive results (relative to medical
screens) need not deter physicians from administering tests and making prompt
referrals for children who perform poorly. Even so, high false-positive rates
carry costs and in light of economic constraints in health care and education,
it behooves professionals to use measures with as few false-positive results
(and false-negative results) as possible.3
The results of this study suggest that the lingering anxiety reportedly
experienced by many parents of children who receive false-positive scores
on screening may be more real than apparent. Given that their children are
likely to have lower scores on diagnostic tests and to carry more psychosocial
risk factors, parental concern appears to be appropriate rather than problematic.
Limitations in the study were the use of a cross-sectional design. Future
research should focus longitudinally on the outcomes of children with false-positive
screening results to determine whether their lower scores in adaptive behavior,
intelligence, academics, and language predict emerging disabilities or rather
continued below average performance. Studies of differences in the intensity
of intervention on children with false-positive scores might help elucidate
the levels and types of service that ensure optimal outcomes. It would be
helpful if screening tests themselves more readily identified potential false-positive
results. The Brigance Screens and the Parents' Evaluations of Developmental
Status, for example, have optional scoring criteria that can identify children
with likely false-positive results and target them not for evaluations but
rather to receive information handouts, advice, referrals to noncriterion-based
services, and watchful waiting. Research is needed on the ability of other
screening tests to perform this helpful function. Finally, replication studies
should view the contribution of age differences. Because increased age is
associated with increased rates of disabilities, age differences in the false-positive
and true-negative groups may have contributed to the findings. On the other
hand, a difference of 4 months is small and may have little practical significance.
Given their lower performance in critical developmental areas and the
presence of psychosocial risk factors, children with false-positive scores
clearly need unique attention from their primary health care giver. When these
children are identified, clinicians should make use of diagnostic test results
to actively promote optimal development,22
monitor progress, and make needed referrals. To do this, physicians need to
become familiar with community resources, such as parenting classes, Head
Start, Early Head Start, quality preschool and day care programs, services
under the Individuals With Disabilities Education Act, private therapies,
tutoring, summer school, literacy interventions, and other helpful services.
AUTHOR INFORMATION
Accepted for publication August 25, 2000.
From the Department of Pediatrics, College of Medicine, The Pennsylvania
State University, East Berlin; and the Department of Pediatrics, Vanderbilt
University, Nashville, Tenn.
Corresponding author: Frances Page Glascoe, PhD, Department of Pediatrics,
College of Medicine, The Pennsylvania State University, 25 Bragg Dr, East
Berlin, PA 17316 (e-mail: Frances.P.Glascoe{at}Vanderbilt.edu).
REFERENCES
| |
1. Squires J, Nickel RE, Eisert D. Early detection of developmental problems: strategies for monitoring
young children in the practice setting. J Dev Behav Pediatr. 1996;17:420-427.
FULL TEXT
|
ISI
| PUBMED
2. Glascoe FP, Byrne KE. The accuracy of three developmental screening tests. J Early Intervention. 1993;17:368-379.
3. Glascoe FP, Foster FM, Wolraich ML. An economic evaluation of four methods for detecting developmental
problems. Pediatrics. 1997;99:830-837.
FREE FULL TEXT
4. Friedland DR, Fahs MC, Catalano PJ. A cost-effectiveness analysis of the high risk register and auditory
brainstem response. Int J Pediatr Otorhinolaryngol. 1996;38:115-130.
FULL TEXT
|
ISI
| PUBMED
5. Tluczek A, Mischler E, Farrell PM, et al. Parents' knowledge of neonatal screening and response to false-positive
cystic fibrosis testing. J Dev Behav Pediatr. 1992;13:181-186.
ISI
| PUBMED
6. Fyro K, Bodegard G. Four-year follow-up of psychological reactions to false positive screening
tests for congenital hypothyroidism. Acta Paediatr Scand. 1987;76:107-114.
ISI
| PUBMED
7. Scott FG, Lingaraju S, Kilgo J, Kregel J, Lazzari A. A survey of pediatricians on early identification and early intervention
services. J Early Intervention. 1993;17:129-138.
8. Dobos AE, Dworkin PH, Bernstein B. Pediatricians' approaches to developmental problems: has the gap been
narrowed? J Dev Behav Pediatr. 1994;15:34-39.
FULL TEXT
|
ISI
| PUBMED
9. Committee on Children With Disabilities, American Academy of Pediatrics. Screening for developmental disabilities. Pediatrics. 1986;78:526-528.
FREE FULL TEXT
10. Glascoe FP, Byrne KE, Ashford LG, Johnson KL, Chang B, Strickland B. Accuracy of the Denver-II in developmental screening. Pediatrics. 1992;89:1221-1225.
FREE FULL TEXT
11. Glascoe FP. Parents' concerns about children's development: prescreening technique
or screening test? Pediatrics. 1997;99:522-528.
FREE FULL TEXT
12. Glascoe FP. Do parents discuss concerns about children's development with health
care providers? Ambulatory Child Health. 1997;2:349-356.
13. Glascoe FP. Toward a model for an evidenced-based approach to developmental/behavioral
surveillance, promotion and patient education. Ambulatory Child Health. 1999;5:197-208.
14. Glascoe FP. The validation and standardization of Parents' Evaluations of Developmental
Status. Diagnostique. 1999;23:185-203.
15. Glascoe FP. The accuracy of the Brigance Screening Tests in identifying children
with disabilities. Diagnostique. 1997;21:87-99.
16. US Census Bureau. National Population Estimates. Washington, DC: US Government Printing Office; 2000.
17. Brigance A. Brigance Screens. Billerica, Mass: Curriculum Associates Inc; 1990.
18. Newborg J, Stock JR, Wnek L, Guidabaldi J, Svinicki J. Battelle Developmental Inventory Screening Test. Chicago, Ill: Riverside Publishing; 1989.
19. Frankenburg N, Dodds J, Archer P. Denver-II: Technical Manual. Denver, Colo: Denver Developmental Materials Inc; 1990.
20. Glascoe FP. Parents' Evaluations of Developmental Status: A Method
for Detecting and Addressing Developmental and Behavioral Problems in Children. Nashville, Tenn: Ellsworth & Vandermeer Press Ltd; 1997. Available
at: http://www.pedstest.com. Accessed October 2000.
21. Sameroff AJ, Seifer R, Barocas R, Zax M, Greenspan S. Intelligence quotient scores of 4-year-old children: social-environmental
risk factors. Pediatrics. 1987;79:343-350.
FREE FULL TEXT
22. Glascoe FP, Oberklaid F, Dworkin PH, Trimm F. Brief approaches to educating patients and parents in primary care. Pediatrics [serial online]. 1998;101:E10. Available at: http://www.pediatrics.org/cgi/content/full/101/6/e10. Accessed October 2000.
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter
What's this?
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
|
Implementing Electronic Health Record-Based Quality Measures for Developmental Screening
Jensen et al.
Pediatrics 2009;124:e648-e654.
ABSTRACT
| FULL TEXT
Identifying Infants and Young Children With Developmental Disorders in the Medical Home: An Algorithm for Developmental Surveillance and Screening
Council on Children With Disabilities et al.
Pediatrics 2006;118:405-420.
ABSTRACT
| FULL TEXT
Factors Associated With Age of Diagnosis Among Children With Autism Spectrum Disorders
Mandell et al.
Pediatrics 2005;116:1480-1486.
ABSTRACT
| FULL TEXT
Developmental Stages of Developmental Screening: Steps to Implementation of a Successful Program
Pinto-Martin et al.
AJPH 2005;95:1928-1932.
ABSTRACT
| FULL TEXT
Parent's Reports Predict their Child's Developmental Problems
Tervo
CLIN PEDIATR 2005;44:601-611.
ABSTRACT
The Essential Role of Research in Community Pediatrics
Duggan et al.
Pediatrics 2005;115:1195-1201.
ABSTRACT
| FULL TEXT
Topical Review: Developmental Screening
Rydz et al.
J Child Neurol 2005;20:4-21.
ABSTRACT
Predicting Motor Outcome at Preschool Age for Infants Tested at 7, 30, 60, and 90 Days After Term Age Using the Test of Infant Motor Performance
Kolobe et al.
ptjournal 2004;84:1144-1156.
ABSTRACT
| FULL TEXT
Improving the Early Detection of Children with Subtle Developmental Problems
Williams and Holmes
J Child Health Care 2004;8:34-46.
ABSTRACT
Falik
AAP News 2001;18:144-144.
FULL TEXT
Overreferrals for Developmental Screening Are Not a Problem
Accardo
AAP Grand Rounds 2001;5:39-40.
FULL TEXT
|
