You are seeing this message because your Web browser does not support basic Web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.


ABOUT ARCHIVES
Advanced Search

Welcome   | My Account | E-mail Alerts | Access Rights | Sign In


  Vol. 156 No. 8, August 2002 TABLE OF CONTENTS
  Archives
  •  Online Features
  Article
 This Article
 •Abstract
 •PDF
 •Send to a friend
 • Save in My Folder
 •Save to citation manager
 •Permissions
 Citing Articles
 •Citation map
 •Citing articles on HighWire
 •Citing articles on ISI (22)
 •Contact me when this article is cited
 Related Content
 •Related article
 •Similar articles in this journal
 Topic Collections
 •Evidence-Based Medicine
 •Violence and Human Rights
 •Adolescent Medicine
 •Alert me on articles by topic

School-Based Violence Prevention Programs

Systematic Review of Secondary Prevention Trials

Julie A. Mytton, MBBS, MRCGP, MSc; Carolyn DiGuiseppi, MD, MPH; David A. Gough, PhD; Rod S. Taylor, MSc, PhD; Stuart Logan, MSc, MRCP

Arch Pediatr Adolesc Med. 2002;156:752-762.

ABSTRACT

Objective  To quantify the effectiveness of school-based violence prevention programs for children identified as at risk for aggressive behavior.

Design  Systematic review and meta-analysis of randomized controlled trials. Electronic databases and bibliographies were systematically searched and authors and organizations were contacted to identify randomized controlled trials. Standardized, weighted mean effect sizes were assessed by meta-analysis.

Setting  Elementary, middle, and high schools.

Participants  Children at risk for aggressive behavior.

Main Outcome Measures  Violent injuries, observed or reported aggressive or violent behaviors, and school or agency responses to aggressive behaviors.

Results  Of the 44 trials identified, none reported data on violent injuries. For the 28 trials that assessed aggressive behaviors, the pooled difference between study groups was -0.36 (95% confidence interval, -0.54 to -0.19) in favor of a reduction in aggression with intervention. For the 9 trials that reported data on school or agency responses to aggression, the pooled difference was -0.59 (95% confidence interval, -1.18 to 0.01). Subgroup analyses suggested greater effectiveness in older students and when administered to mixed-sex groups rather than to boys alone.

Conclusions  School-based violence prevention programs may produce reductions in aggressive and violent behaviors in children who already exhibit such behavior. These results, however, need to be confirmed in large, high-quality trials.



INTRODUCTION
 Jump to Section
 •Top
 •Introduction
 •Methods
 •Results
 •Comment
 •Conclusions
 •Author information
 •References

IN 1998 in the United States, 43 of every 1000 children were victims of nonfatal violent crime while at school or on their way to and from school.1 More than 250 000 serious violent crimes (1% of all schoolchildren), including rape, sexual assault, and aggravated assault, were committed against students at school or while going to or from school. Teachers are also victims of school violence, with 31 per 1000 teachers reporting violent crime victimization in 1998. Youth violence in the school has become an increasing concern in the United States and other nations.2-3

Many schools have implemented prevention programs that attempt to address this problem. According to a US Surgeon General's report, "Hundreds of youth violence prevention programs are being used in schools and communities throughout the country, yet little is known about the actual effects of many of them."2, 4 Although previous reviews have attempted to identify model programs and best practices,2, 4 informed decision making by policymakers and school professionals about violence prevention programs requires ready access to information about the effectiveness of such programs based on systematic and comprehensive review and synthesis of all available literature. The most reliable evidence for effectiveness comes from randomized controlled trials.5 We therefore conducted a systematic review and meta-analysis of such trials to explore and quantify the effect of school-based violence prevention programs on aggressive and violent behaviors in children at high risk for violent behavior.


METHODS
 Jump to Section
 •Top
 •Introduction
 •Methods
 •Results
 •Comment
 •Conclusions
 •Author information
 •References

INCLUSION CRITERIA

Studies were included if (1) participants were randomly assigned to intervention and control groups; (2) outcome data were collected concurrently in the 2 groups; (3) the study population was composed of children in grades kindergarten (K) through 12 (or their international equivalent) identified by author-defined criteria as exhibiting or at risk for aggressive behavior; (4) the experimental intervention was designed, either wholly or largely, to reduce aggression and violence; (5) the intervention was primarily school based, although it could contain additional components; and (6) outcome measures included aggressive behavior, school and agency responses to acts of aggression, or violent injuries. We defined outcome measures as follows. Aggressive behavior was defined as scores on standardized tests that assess aggressive behavior (eg, Achenbach Child Behavior Checklist, Miller School Behavior Checklist) or actual counts of aggressive behaviors, such as fights or bullying (eg, via classroom observation and videotapes). School or agency actions were defined as any school or agency actions, such as detention, suspension, or court contact, recorded in official records that were taken in response to aggressive behaviors (eg, fighting and bullying). When school or agency records did not differentiate between responses to aggressive behaviors and responses to nonaggressive misbehaviors, such as truancy, all types of misbehaviors were included. The last outcome measure was violent injuries (eg, emergency department attendances).

For studies with multiple outcome measures, 1 aggressive behavior and 1 school or agency action outcome were chosen on the basis of a predefined hierarchy of factors (in order, data availability, measure specificity, quality assurance of measure, outcome assessor, data completeness and validation of the measure) and random choice if none of these applied (details available from the authors). We did not assess outcomes indirectly related to violence, such as school achievement, knowledge about or attitudes toward violence, mental health outcomes, and measures of aggressive responses to artificial stimuli or experimental tasks. We excluded cluster randomized trials with only 2 randomized schools or classes in which confounding factors cannot be effectively eliminated by randomization.

SEARCH STRATEGY

To identify relevant trials, electronic databases were searched using content terms such as aggress*, violen*, and fight*, with terms such as school*, educat*, and student*. The results were combined with the Cochrane Collaboration's optimally sensitive search strategy to identify controlled trials, adapted as required for each database (full search strategy available from the authors). We searched the Cochrane Controlled Trials Register (1998, issue 1), MEDLINE (1994–June 1998), EMBASE (1980–January 1998), PsycLIT (1887–March 1998), ERIC (Educational Resource Information Centre) (1970–September 1997), CINAHL (Cumulative Index to Nursing and Allied Health Literature) (1982–April 1998), Dissertation Abstracts (1861–March 1998), IBSS (International Bibliography of Social Sciences) (1952-1998), and NCJRS (National Criminal Justice Reference Service) (1970–May 1999) and the bibliographies of published reviews6-8 and relevant trials. Aggression and Violent Behavior (Issue 1, 1996–Issue 3, 1998) were hand searched. We contacted relevant international organizations and experts and attempted to contact the authors of relevant studies to identify unpublished and internal reports. No language or date restrictions were applied.

STUDY SELECTION

Titles, abstracts, and keywords of identified records were screened to exclude ineligible trials (if specified in sufficient detail). Full texts of remaining reports were reviewed and additional ineligible trials excluded. Authors were contacted for clarification where necessary.

DATA EXTRACTION

From eligible reports, 2 of us (J.A.M. and C.D.) independently extracted detailed data on study participants, interventions, outcomes, follow-up, results, methods of group assignment and allocation concealment, blinding of outcomes assessment, and loss to follow-up. A third author (D.A.G.) also independently extracted data on participants, interventions, and outcomes. Differences were resolved by discussion. We attempted to contact all authors of eligible trials to confirm study details, obtain missing data, and identify relevant unpublished outcomes.

ANALYSIS

We compared results of any intervention to no intervention (ie, control or placebo group) immediately after intervention and at the 12-month follow-up in the subsample for which these data were collected. We also assessed the effect of different types of interventions, grouping them according to the predominant training focus: (1) skills of nonresponse, either managed (eg, conflict resolution) or not (eg, anger control); or (2) relationship skills and other interventions of social context (eg, family or social relationships, peer mediation).

Study-specific differences between intervention and control groups for each of these comparisons were pooled using meta-analysis9 (RevMan 4.1; The Nordic Cochrane Centre, Copenhagen, Denmark) to produce an overall estimate of effect. Pooled results are expressed as standardized mean differences (with 95% confidence intervals [CIs]). In trials with multiple intervention or control groups, weighted, pooled means and SDs were used in the meta-analysis to avoid statistical problems with nonindependence of data that would result from including multiple intervention groups as separate trials. Studies comparing different intervention groups or different intensities of the same intervention, with no placebo or control group, were excluded from the meta-analysis but are described in Table 1.10-13


View this table:
[in this window]
[in a new window]
Summary of Secondary Prevention Trials*


Trial heterogeneity was explored with a {chi}2 test using a significance level of .05.9 If there was statistical evidence of heterogeneity, a random-effects model was used. We used formal statistical testing (using Stata statistical software, version 6; Stata Corp, College Station, Tex) and funnel plot analysis (in which study size is plotted against intervention effect)14 to examine effects of study size and bias. In funnel plots, results from small studies tend to scatter widely at the bottom, with the spread narrowing among larger studies, so that the plot should represent a symmetrical inverted funnel. Asymmetry or gaps in the funnel plot may indicate that some studies or study data exist that may not have been published or located.14 It may also indicate poor methodologic quality of smaller studies (which tend to show larger effect sizes [ESs]) or true heterogeneity in the results.14-15 Exploration of heterogeneity by meta-regression, using covariates indicative of study quality such as allocation concealment, use of blinding, and type of intervention, was planned but could not be conducted because of inadequate reporting of such data.

When SDs were not published and could not be obtained, they were imputed by standard statistical methods16 or derived from trials reporting the outcome in a similar population, where possible. Sensitivity analyses were performed on the effect of imputing SDs.

Six trials11, 17-21 used cluster randomization at the level of the class or school. Of these, 3 trials11, 19-20 did not report data suitable for inclusion. One trial18 analyzed results by cluster. We analyzed the results of the other 2 trials17, 21 using a published intraclass correlation coefficient of 0.02 to take into account the cluster randomization.22-23 As sensitivity analyses, we used more extreme values for the intraclass correlation coefficient (ie, 0.0001 and 0.1); there was no effect on results to one decimal place (data not shown).

Subgroup analyses specified a priori included assessing differences in intervention effects by whether the program was administered to primary or secondary school students and boys-only intervention groups vs mixed (or girls-only) groups. "Primary schools" included elementary schools (grades K through 5 or K through 6) or students of equivalent ages if grade was unspecified or study was international. "Secondary schools" included middle, junior high, and high schools (grades 6 through 12 or 7 through 12) or students of equivalent ages.


RESULTS
 Jump to Section
 •Top
 •Introduction
 •Methods
 •Results
 •Comment
 •Conclusions
 •Author information
 •References

We identified 9286 unduplicated electronic records, from which 274 potentially relevant reports were identified and the full texts examined. Thirty-five eligible trials12-13,18-21,24-52 were found. An additional 3 trials10-11,17 were identified from authors, 233, 53 from bibliographies, and 454-57 from expert contacts, giving 44 randomized controlled trials of secondary violence prevention programs. Five authors provided additional data.26, 37, 42, 52-53 We received responses from 66% of all trial authors contacted. No trials reported data on violent injuries. Trial details are shown in Table 1.

MAIN ANALYSES

Thirty-eight trials compared the intervention to a no intervention or placebo (ie, alternative classroom activity) control group and measured aggressive behavior. Twenty-three had complete data, 5 reported group means alone but the SD could be imputed, and 10 reported only partial or no data, giving 28 trials for analysis. Of these, 5 adjusted posttest results for pretest scores. Combining these 28 trials (Figure 1), aggressive behavior was reduced in intervention compared with control groups after intervention (ES, -0.36; 95% CI, -0.54 to -0.19). A test for heterogeneity was significant (P<.001), indicating variation in results across trials. Results were similar for 6 trials that collected data to 12 months after intervention (ES, -0.35; 95% CI, -0.79 to 0.09, with significant heterogeneity; P<.001), suggesting that effects were maintained.



View larger version (73K):
[in this window]
[in a new window]
Figure 1. Comparison of any violence prevention intervention vs no intervention by type of school. The outcome was the difference in aggression scale score or observed aggression by type of school (after intervention). SMD indicates standardized mean difference; CI, confidence interval; and CPPRG, Conduct Problems Prevention Research Group.


Results were similar for different types of training. Training in skills of nonresponse produced beneficial effects on aggressive behavior immediately after intervention (ES, -0.38; 95% CI, -0.65 to -0.11, with significant heterogeneity; P = .001) and at 12 months (ES, -0.56; 95% CI, -1.08 to -0.05, without heterogeneity; P = .38). Interventions to improve relationship skills or social context also reduced aggressive behavior after intervention (ES, -0.65; 95% CI, -1.05 to -0.25, without heterogeneity; P = .24). One trial measured this outcome at 12 months (ES, -0.50; 95% CI, -0.97 to -0.04).

Among 15 trials that measured effects on school or agency actions, 6 published no data and the authors either could not provide data or could not be contacted. The pooled ES among 9 included trials was -0.59 (95% CI, -1.18 to 0.01), indicating a reduction in school or agency actions with intervention (Figure 2). There was significant heterogeneity (P<.001). In 2 trials, there was no apparent effect at 12 months (ES, 0.05; 95% CI, -0.45 to 0.55).



View larger version (23K):
[in this window]
[in a new window]
Figure 2. Comparison of any violence prevention intervention vs no intervention by type of school. The outcome was the difference in school or agency response to acts of aggression and other acts by type of school (after intervention). SMD indicates standardized mean difference; CI, confidence interval; and CPPRG, Conduct Problems Prevention Research Group.


Training in skills of nonresponse produced less conclusive beneficial effects on school or agency actions (ES, -0.32; 95% CI, -0.90 to 0.26, with heterogeneity; P = .003). Training to improve relationship skills or social context produced more favorable effects (ES, -0.69; 95% CI, -1.26 to -0.13), although only 2 trials were included.

SUBGROUP ANALYSES

Primary vs Secondary School

The immediate postintervention effect on aggressive behavior was similar for trials in primary schools (ES, -0.33; 95% CI, -0.54 to -0.12) and secondary schools (ES, -0.43; 95% CI, -0.75 to -0.11). Significant heterogeneity persisted within each subgroup (Figure 1).

There was no evidence of benefit from interventions in primary schools on school or agency actions (ES, 0.11; 95% CI, -0.48 to 0.70, without heterogeneity; P = .05). The remaining 7 studies, in secondary schools, showed a stronger positive effect of the intervention (ES, -0.82; 95% CI, -1.56 to -0.09, with significant heterogeneity; P<.001).

Sex

Most students identified as aggressive or violent were boys. Twelve trials10, 25-26,29, 35-39,43-44,52 studied boys alone, 1 trial13 studied girls alone, and the remaining 31 studied mixed groups in which most students were boys. Violence prevention programs delivered to boys alone had a modest effect on aggressive behavior (ES, -0.18; 95% CI, -0.47 to 0.1, without heterogeneity; P = .18), which may have been due to chance. The ES was greater for interventions delivered to mixed groups or girls alone (ES, -0.44; 95% CI, -0.66 to -0.23), although results were heterogeneous (P<.001). Two trials evaluating the effect of interventions on school or agency actions among male students alone reported no evidence of a benefit (ES, 0.22; 95% CI, -0.20 to 0.63, without heterogeneity; P = .25). Among 7 studies17-18,24, 28, 32, 45, 51 of mixed-sex groups, there was a substantial reduction in school or agency actions after intervention (ES, -0.85; 95% CI, -1.59 to -0.10, with significant heterogeneity; P<.001).

Effect of Imputing Data

The SD was imputed for 5 trials with incomplete reporting.27, 30, 39, 41, 46 Omitting these data, the ES is -0.24 (95% CI, -0.40 to –0.08), a more modest effect than that found in the main analysis. The 5 studies with imputed data showed a strong effect (ES, -1.19; 95% CI, -1.62 to –0.76).

Evaluating Data for Heterogeneity and Publication Bias

The Begg test (P = .07) and Eggar test (bias coefficient P<.001) suggested a relationship between study size and reported effects on aggressive behavior (ie, larger studies reported smaller effects). This is illustrated in the funnel plot (Figure 3). The plot also demonstrates asymmetry, with fewer trials showing a positive standardized mean difference, suggesting that small studies showing harm or no benefit were not included.



View larger version (12K):
[in this window]
[in a new window]
Figure 3. Funnel plot of measures of aggressive behavior for any school-based violence prevention program vs no program. The standardized mean difference demonstrates the treatment effect from each trial; the precision of the treatment effect is the reciprocal of the standard error.



COMMENT
 Jump to Section
 •Top
 •Introduction
 •Methods
 •Results
 •Comment
 •Conclusions
 •Author information
 •References

School-based violence prevention programs for high-risk children modestly reduced both aggressive behaviors and school or agency actions. Training in nonresponse skills and in relationship skills both showed beneficial effects. Although the ESs appear small, the overall benefits may be substantial when implemented on a schoolwide or districtwide basis.

Effects on aggressive behavior were similar in primary and secondary schools, whereas effects on school or agency actions were greater in secondary schools, although this difference may have been due to chance or to the way such actions are implemented at different ages. Although most programs focused solely or largely on boys, program effects appeared to be greater among mixed groups. Whether programs are more effective when delivered to mixed-sex groups or schools or whether such programs have greater effects on girls than on boys cannot be determined from these data.

Both subgroup analyses were prespecified, but, given the relatively small number of studies in each subcategory and the possibility of confounding by other trial characteristics, they should be interpreted with caution.

Tests for heterogeneity highlighted wide variation in the magnitude of results across trials. Funnel plot asymmetry is often used as an indication of publication bias, but it can also be explained by true heterogeneity among studies, resulting from differences in type or intensity of intervention, underlying risk, and study quality.58 We found that differences in age group, sex, and training focus contributed to, but did not fully explain, the substantial heterogeneity. Trials in this review varied substantially in their quality, execution, and reporting. Most sample sizes were small, and many studies provided limited methodologic information (Table 1). Data on indicators of methods, such as allocation concealment, use of blinding, and type of intervention, were so poorly reported that exploration of heterogeneity by meta-regression was not possible. Therefore, we cannot make firm conclusions about the impact of these factors on the apparent heterogeneity.

All the trials selected students at high risk for violent or aggressive behavior. However, selection processes varied considerably. Because of inadequate information on these processes for most trials and difficulty in establishing population rates for the service actions on which selection was based, we could not explore whether program effectiveness differed according to how the study determined "high-risk" status.

Meta-analyses were performed on posttest results. Although randomized, most trials were relatively small. Therefore, differences in the pretest scores and other baseline characteristics (eg, age) may have occurred by chance. Unfortunately, the difference between the posttest and pretest means was not available for most studies. Posttest scores adjusted for pretest differences were used where available, otherwise unadjusted posttest scores were used.

The 5 studies27, 30, 39, 41, 46 in which the SD was imputed showed a greater beneficial effect from intervention than did those reporting full data. This may have been due to quality differences. Methodologically weaker studies, which may be less likely to report complete data, tend to show larger ESs.5 Exclusion of trials with imputed values reduced the ES but did not change its direction. Excluding such trials could introduce bias, however. Of 3 trials that reported group means but were excluded from the meta-analysis because SDs could not be imputed, 234-35 reported that the intervention had beneficial effects in reducing the outcomes measured, whereas 143 reported no effect. Their exclusion may have biased our results toward the null.

Publication bias is an important threat to validity in systematic reviews. Authors may choose not to submit results that are negative or not significant, and journals may not publish such studies.58-59 Publication bias may also operate within individual studies when investigators selectively report outcomes with "significant" results.59 We attempted to contact each author to establish whether any unpublished data were available and whether they knew of any further studies, published or unpublished. In addition, we contacted experts and organizations for unpublished studies. Despite such efforts, many trials did not provide data for relevant outcomes known to have been measured, which may have affected these results. The funnel plot suggests that negative studies may exist that were not retrieved and included.


CONCLUSIONS
 Jump to Section
 •Top
 •Introduction
 •Methods
 •Results
 •Comment
 •Conclusions
 •Author information
 •References

Violence within our society has been a public concern in recent years. Substantial resources have been allocated to preventing violent injuries and crime. Schoolchildren have been intensively targeted. We identified 44 randomized controlled trials that evaluated school-based programs, of which 28 (64%) provided outcome data. Many trials were small and reported insufficient information to assess quality. Nevertheless, pooled results suggest that these interventions may reduce aggressive and violent behavior and school or agency actions in response to such behavior. However, analysis also suggests the possibility of bias toward studies with positive results and genuine differences among trials, which may mean that the true effect is smaller than that indicated. Larger, better controlled trials, with improved reporting of methods, use of adequate methods for allocation, blinding of outcome assessment, and more complete reporting of results, appear warranted to determine whether the apparent benefit is real. In addition, unexpected results emerged regarding differential effects by sex, which warrant further research.


What This Study Adds

Each year, 1 in 25 US schoolchildren are victims of violent crime while at school or on the way to and from school. The aggregate effects of the hundreds of school-based youth violence prevention programs currently being implemented have not been adequately evaluated.

School-based violence prevention programs for high-risk children modestly reduced both aggressive behaviors and school or agency actions in response to aggressive behavior. Effects on aggressive behavior were similar regardless of whether the programs focused on training in skills of nonresponse (eg, conflict resolution or anger control) or on training in social skills or social context changes. The benefits of violence prevention programs were similar in programs introduced in both primary and secondary schools, but appeared to be greater among mixed-sex groups. Additional large, well-controlled trials are needed to determine whether the apparent benefits are real.



AUTHOR INFORMATION
 Jump to Section
 •Top
 •Introduction
 •Methods
 •Results
 •Comment
 •Conclusions
 •Author information
 •References

Accepted for publication February 27, 2002.

This study was funded by the Systematic Reviews Training Unit, Institute of Child Health (Dr Mytton), and Camden and Islington Health Authority, London, England (partial funding to Dr DiGuiseppi).

We thank Ian Roberts, MBBCh, MRCP, PhD, Public Health Intervention Research Unit, London School of Hygiene and Tropical Medicine, London, for his advice throughout this project, and Frances Bunn, BSc, MSc, coordinator of the Cochrane Injuries Group, London.

Corresponding author: Julie A. Mytton, MBBS, MRCGP, MSc (e-mail: jmytton{at}doctors.org.uk).

From the Department of Paediatric Epidemiology and Biostatistics (Drs Mytton and DiGuiseppi) and Systematic Reviews Training Unit (Dr Logan), Institute of Child Health, Social Science Research Unit (Dr Gough), and National Institute for Clinical Excellence (Dr Taylor), London, England. Dr Mytton is now with the Department of Public Health, Oldham Primary Care Trust, Oldham, England. Dr DiGuiseppi is now with the Department of Preventive Medicine and Biometrics, University of Colorado Health Sciences Center, Denver. Dr Taylor is now with the Department of Public Health and Epidemiology, University of Birmingham, Edgbaston, England.


REFERENCES
 Jump to Section
 •Top
 •Introduction
 •Methods
 •Results
 •Comment
 •Conclusions
 •Author information
 •References

1. Kaufman P, Chen X, Choy SP, et al. Indicators of School, Crime and Safety: 2001. Washington, DC: US Depts of Education and Justice; 2001. NCES 2002113/NCJ-190075.
2. US Surgeon General. Youth Violence: A Report of the Surgeon General. Washington, DC: US Surgeon General Office; 2001.
3. Kaufman P, Chen X, Choy SP, et al. Indicators of School Crime and Safety. Washington, DC: US Depts of Education and Justice; 1998.
4. Thornton TN, ed, Craft CA, ed, Dahlberg LL, ed, et al. Best Practices of Youth Violence Prevention: A Sourcebook for Community Action. Atlanta, Ga: Centers for Disease Control and Prevention; 2000.
5. Schultz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408-412. ABSTRACT
6. Rivara FP, Farrington DP. Prevention of violence: role of the pediatrician. Arch Pediatr Adolesc Med. 1995;149:421-429. ABSTRACT
7. Derzon JH, Wilson SJ. An empirical review of school-based programs to reduce violence. Paper presented at: Annual meeting of the American Society of Criminology; November 1999; Toronto, Ontario.
8. Kellermann AL, Fuqua-Whitley DS, Rivara FP, Mercy J. Preventing youth violence: what works? Annu Rev Public Health. 1998;19:271-292. FULL TEXT | ISI | PUBMED
9. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177-188. FULL TEXT | ISI | PUBMED
10. Camp BW. Two psychoeducational treatment programs for young aggressive boys. In: Whalen CK, Henker B, eds. Hyperactive Children: The Social Ecology of Identification and Treatment. New York, NY: Academic Press; 1980:191-219.
11. Cavell TA, Hughes JN. Secondary prevention as context for studying change processes in aggressive children. J Sch Psychol. 2000;38:199-235. FULL TEXT
12. Hughes JN, Grossman PG, Hart MT. Effectiveness of skill training and consultation with aggressive children. Paper presented at: 101st Annual Meeting of the American Psychological Association; August 20, 1993; Toronto, Ontario.
13. Hughes LS. Acquisition of Conflict Management Skills With High School Adolescent Females [dissertation]. Kalamazoo: Western Michigan University; 1992.
14. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315:629-634. FREE FULL TEXT
15. Stuck AE, Rubenstein LZ, Wieland D. Bias in meta-analysis detected by a simple, graphical test: asymmetry detected in funnel plot was probably due to true heterogeneity [letter]. BMJ. 1998;316:469. FREE FULL TEXT
16. Follmann D, Elliot P, Suh I, Cutler J. Variance imputation for overviews of clinical trials with continuous response. J Clin Epidemiol. 1992;45:769-773. FULL TEXT | ISI | PUBMED
17. Conduct Problems Prevention Research Group. Initial impact of the Fast Track prevention trial for conduct problems, I: the high-risk sample. J Consult Clin Psychol. 1999;67:631-647. FULL TEXT | ISI | PUBMED
18. Etscheidt SL. A Comparison of Cognitive, Cognitive-Behavioral and Behavioral Interventions in Reducing Classroom Aggressive Behavior [dissertation]. Minneapolis: University of Minnesota; 1984.
19. Guerra NG, Tolan PH, Henry D, Van Acker R, Huesmann LR, Eron L. Developmental and contextual influences on the prevention of aggression in urban settings: preliminary outcomes from the Metropolitan Area Child Study. Paper presented at: 106th Annual Meeting of the American Psychological Association; August 14-18, 1998; San Francisco, Calif.
20. Mayer GR, Butterworth T, Nafpaktitis M, Sulzer-Azaroff B. Preventing school vandalism and improving discipline: a three year study. J Appl Behav Anal. 1983;16:355-369. FULL TEXT | ISI | PUBMED
21. Prinz RJ, Blechman EA, Dumas JE. An evaluation of peer coping-skills training for childhood aggression. J Clin Child Psychol. 1994;23:193-203. FULL TEXT
22. Donner A, Klar N. Cluster randomization trials in epidemiology. J Stat Planning Inference. 1994;42:37-56. FULL TEXT
23. Rooney B, Murray DM. A meta-analysis of smoking prevention programs after adjustment for errors in the unit of analysis. Health Educ Q. 1996;23:48-64. ISI | PUBMED
24. Booth BA. A Cognitively Based Anger Control Training Program With Aggressive Adolescents in the School Setting [dissertation]. New York: State University of New York; 1995.
25. Boswell JW. Effects of a Multimodal Counseling Program and of a Cognitive-Behavioral Counseling Program on the Anger Management Skills of Pre-adolescent Boys Within an Elementary School Setting [dissertation]. University Park: Pennsylvania State University; 1983.
26. Camp BW, Blom GE, Hebert F, Van Doorninck WJ. "Think aloud": a program for developing self-control in young aggressive boys. J Abnorm Child Psychol. 1977;5:157-169. FULL TEXT | ISI | PUBMED
27. Contreras M. A Study of the Effectiveness of a Structured Life Skills Program in Facilitating Appropriate Classroom Behavior [dissertation]. New Brunswick, NJ: Rutgers University; 1981.
28. D'Elio AR. An Investigation of the Effectiveness of Intervention Strategies on Juvenile Anti-social Behaviors [dissertation]. New York, NY: Fordham University; 1982.
29. Dauer DM. Group Counseling for Anger Control: The Effects of an Intervention Program With Middle School Students [dissertation]. Blacksburg: Virginia Polytechnic Institute and State University; 1994.
30. Day DM, Hartley L. Evaluating a school-based program for aggressive children: comparing outcomes. Paper presented at: 101st Annual Meeting of the American Psychological Association; August 22, 1993; Toronto, Ontario.
31. Deffenbacher JL, Lynch RS, Oetting ER, Kemper CC. Anger reduction in early adolescents. J Counseling Psychol. 1996;43:149-157.
32. Feindler EL, Marriott SA, Iwata M. Group anger control training for junior high school delinquents. Cognit Ther Res. 1984;8:299-311. FULL TEXT
33. Feshbach ND. Empathy training: a field study in affective education. In: Feshbach S, Franzek A, eds. Aggression and Behavior Change: Biological and Social Processes. New York, NY: Praeger; 1979:234-249.
34. Forman SG. A comparison of cognitive training and response cost procedures in modifying aggressive behavior of elementary school children. Behav Ther. 1980;11:594-600.
35. Garrison SR, Stolberg AL. Modification of anger in children by affective imagery training. J Abnorm Child Psychol. 1983;11:115-129. PUBMED
36. Gilberg JA. The Effect of Cognitive Role-Taking on the Classroom Behavior of Aggressive Boys [dissertation]. College Station: Texas A&M University; 1982.
37. Hudley C, Graham S. An attributional intervention to reduce peer-directed aggression among African-American boys. Child Dev. 1993;64:124-138. FULL TEXT | ISI | PUBMED
38. Hudley C, Britsch B, Wakefield WD, Smith T, Demorat M, Cho S. An attribution retraining program to reduce aggression in elementary school students. Psychol Sch. 1998;35:271-282. FULL TEXT | ISI
39. Huey WC, Rank RC. Effects of counselor and peer-led group assertive training on black adolescent aggression. J Counseling Psychol. 1984;31:95-98. FULL TEXT
40. Lee DY, Hallberg ET, Hassard JH. Effects of assertion training on aggressive behavior of adolescents. J Counseling Psychol. 1979;26:459-461. FULL TEXT
41. Lochman JE, Coie JD, Underwood MK, Terry R. Effectiveness of a social relations intervention program for aggressive and nonaggressive, rejected children. J Consult Clin Psychol. 1993;61:1053-1058. <