You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Program Description

Overview The goal of the Maryland After School Community Grant Program (MASCGP), which served Maryland youth in grades 4 through 8, was to strengthen youth resiliency and prevent substance abuse, violence, and delinquency among youth by increasing the availability of high quality, structured after school programs. The program's objectives were to increase participants' supervised after school time, academic performance, social skills, attachments to prosocial adults, aversion to substance use and illegal behavior, and involvement and investment in constructive activities.
Start Date 1997 (completed in June 2002)
Scope state
Type after school
Location urban, rural, and suburban
Setting public school, community-based organization, faith-based organization, private facility, recreation center
Participants elementary and middle school students (fourth through eighth graders, some third graders)
Number of Sites/Grantees approximately 40
Number Served 469 in 2001–2002
Components MASCGP activities varied by sponsoring agency (county or state government, public schools, youth agencies, church, and private organizations), program site (community center or public school), number of youth served, participation “dosage” (number of hours participants experience the program), and fee schedule. All MASCGP centers were required to include three basic components: academic achievement; social skills; and “bonding” activities, such as sports, arts, crafts, and other recreational activities aimed at retaining participants.
Funding Level $1.1 million for the 2001–2002 school year
Funding Sources Safe and Drug Free Schools Program of the U.S. Department of Justice, Maryland Governor's Office of Crime Control & Prevention, parent fees, other sources
Other MASCGP was incorporated into a larger “Youth Strategies” Activity Initiative that combines eight federal and state funding streams and funds a wide variety of prevention activities to reduce or prevent youth substance abuse and crime. The 5-year consolidated grant initiative offered $15 million over 18 months (January 2002–June 2003) to 24 Maryland communities for prevention, intervention, and aftercare services for youth.


Evaluation

Overview The goals of the evaluation were to examine program implementation and impact on participants' behavior and levels of risk and resiliency. Further studies focused on predictors of dropping out of programs, mechanisms linking participation to reduced delinquency, and relationships between program implementation and youth outcomes.
Evaluator Denise C. Gottfredson, Stephanie A. Weisman, Shannon C. Womer, Melissa Kellstron, Sean Bryner, Amy Kahler, Lee Ann Slocum, Shaoli Lu, and David A. Soulé, University of Maryland, College Park
Evaluations Profiled Maryland After School Community Grant Program: Report on the 1999–2000 School Year Evaluation of the Phase 1 After School Programs

Maryland After School Community Grant Program Part 1: Report on the 2000–2001 School Year Evaluation of the Phase 2 After School Programs

Maryland After School Community Grant Program Part 1: Report on the 2001–2002 School Year Evaluation of the Phase 3 After School Programs

Attrition From After School Programs: Characteristics of Students Who Drop Out

Do After School Programs Reduce Delinquency?

After-School Programs, Antisocial Behavior, and Positive Youth Development: An Exploration of the Relationship Between Program Implementation and Changes in Youth Behavior
Evaluations Planned none (The “Youth Strategies” Activity Initiative will be evaluated by Dr. Denise C. Gottfredson of the University of Maryland.)
Report Availability Weisman, S. A., Soulé, D. A., & Womer, S. C., under the direction of Gottfredson, D. C. (2001). Maryland After School Community Grant Program: Report on the 1999–2000 school year evaluation of the Phase 1 after school programs. College Park: University of Maryland.

Weisman, S. A., Womer, S. C., Lu, S., Soule, D. A., Bryner, S. L., Kahler, A., et al., under the direction of Gottfredson, D. C. (2002). Maryland After School Community Grant Program part 1: Report on the 2000–2001 school year evaluation of the Phase 2 after school programs. College Park: University of Maryland.

Weisman, S. A., Womer, S. C., Kellstrom, M. A. Bryner, S., Kahler, A., & Slocum, L. A., under the direction of Gottfredson, D. C. (2003). Maryland After School Community Grant Program part 1: Report on the 2001–2002 school year evaluation of the Phase 3 after school programs. College Park: University of Maryland.

Weisman, S. A., & Gottfredson, D. C. (2001). Attrition from after school programs: Characteristics of students who drop out. Prevention Science, 2, 201–205.

Gottfredson, D. C. Weisman, S. A., Soulé, D. A., Womer, S. C., & Lu, S. (2004). Do after school programs reduce delinquency? Prevention Science, 5, 253–266.

Weisman, S. A., Soule, D. A., Gottfredson, D. C., Lu, S., Kellstrom, M. A., Womer, S. C., & Bryner, S. L. (2005). After-school programs, anti-social behavior, and positive youth development: An exploration of the relationship between program implementation and changes in youth behavior. In J. L. Mahoney, J. S., Eccles & R. W. Larson (Eds.), Organized activities as contexts of development: Extracurricular activities, after-school, and community programs. Hillsdale, NJ: Lawrence Erlbaum Associates.


Contacts

Evaluation Denise C. Gottfredson, Ph.D.
Project Director
Department of Criminology and Criminal Justice
2220D LeFrak Hall
University of Maryland
Tel: 301-405-4717
Fax: 301-405-4733
Email: dgottfredson@crim.umd.edu
Program Andrea Alexander
Youth Services Division Chief
Governor's Office of Crime Control & Prevention
300 E. Joppa Rd, Suite 1105
Baltimore, MD 21286-3016
Phone: 410-321-3521 ext. 356
Fax: 410-321-3116
Email: andrea@goccp-state-md.org
Profile Updated May 19, 2006

Evaluation 3: Maryland After School Community Grant Program Part 1: Report on the 2001–2002 School Year Evaluation of the Phase 3 After School Programs


Evaluation Description

Evaluation Purpose To examine MASCGP's effectiveness at influencing how youth spend their time and how much time youth spend in supervised settings, and in improving youth outcomes. Also, to examine what program characteristics are related to program effectiveness.
Evaluation Design Quasi-Experimental and Non-Experimental: Implementation data were collected from all sites. Evaluators conducted 5 site visits at each program.

In addition, pretest and posttest data were collected for MASCGP participants and a comparison group of nonparticipants. The comparison group was comprised of youth recruited by programs who were demographically similar to program participants. Data were collected from all MASCGP participants who received parental consent for involvement in the evaluation. In total, 594 treatment and 476 comparison group youth were included in the pretest sample, with 78% of these youth completing a posttest survey at the end of the year. Attrition rates were similar across both groups. Pretest/posttest school attendance data were received for 82% of pretested youth, while pretest/posttest grade point average (GPA) data were received for 88% of pretested youth.

Of 13 measures on the pretests, two showed differential attrition between the treatment and comparison group. First, dropouts in the comparison group were found to have lower levels of commitment to education than dropouts in the treatment group, who were found to be no different in their level of commitment than treatment stayers (which would lead to a bias in favor of the comparison group). Second, dropouts in the treatment group scored lower on belief in conventional rules than drop outs in the comparison group (which would lead to a bias in favor of the treatment group). Because these two sources of bias were in opposite directions and because no differential attrition was found for the other 11 measures, the evaluators concluded that differential attrition was not a major biasing factor.

Analysis revealed a number of significant differences between the treatment and control groups. Treatment youth were more likely to be non-White (p < .01) and to spend time in self-care after school (p < .10); they were less likely to be involved in constructive activities (p < .05), engage in rebellious behavior (p < .05), and to have been arrested (p < .05); treatment youth also had lower GPAs (p < .05) than comparison youth. For middle school youth only, treatment group youth were less likely to have peer drug models (p < .05) and have engaged in delinquent behavior (p < .01) than comparison youth. Overall, the evaluators concluded that the comparison youth were more at risk than treatment youth at pretest. Pretest differences were controlled for in all analyses. The evaluators also cautioned that 50% of the comparison group reported participating in an after school program during the school year, indicating that nonprogram participants were substantially involved in alternative after school activities. The treatment group reported spending an average of 3.5 days per week in after school programs, as compared to 1 day per week for comparison youth (p < .01).
Data Collection Methods Document Review: Logs and forms completed by program staff provided data on daily facets of program administration and components (academic, recreation, and social skills).

Observation: Site visits measured program characteristics, including the structure of program components, levels of youth engagement, social climate, and behavior management.

Secondary Sources/Data Review: Youth's school records (grade point average [GPA] and attendance data) were collected from the prior (2000–2001) and present (2001–2002) school years.

Surveys/Questionnaires: Youth completed the What About You? survey (Gottfredson, 1991) at the beginning and end of the school year. The survey measures: youth's attachment to school, rebellious and delinquent behavior, drug use, attitudes about drug use, peer relationships, parental supervision, commitment to education, belief in rules, and unsupervised after school time and involvement in constructive activities.

Tests/Assessments: Additional items were added to the What About You? survey from the Social Skills Rating System (SSRS) Elementary Level Student form (Gresham & Elliott, 1990), which measured youth's social skills, and from the Communities That Care Youth Survey (Arthur, Pollard, Hawkins & Catalano, 1995), which measured youth's attachment to prosocial adults.

References
Arthur, M., Pollard, J., Hawkins, J. D., & Catalano, R. (1995). Communities that care youth survey. Seattle, WA: Developmental Research and Programs.

Gottfredson, G. (1991). What about you? Ellicott City, MD: Gottfredson Associates.

Gresham, F., & Elliott, S. (1990). Social skills rating system. Circle Pines, MN: American Guidance Service.
Data Collection Timeframe Data were collected during the 2001–2002 school year.


Findings:
Formative/Process Findings

Activity Implementation On average, youth received 3.1 hours of academic assistance activities per week, such as tutoring, homework assistance, and computer skills training.

On average, youth received 2.1 hours of structured social skills per week. The percentage of programs failing to provide the minimum social skills instruction hours of 1.5 per week fell from 58% in 2000–2001 to 27% in 2001–2002.

Observations revealed that the delivery of social skills components was fairly well structured across the programs (as measured by the presence of a lesson plan, specific order of content delivery, and advance planning).
Program Context/Infrastructure On average, the programs met 56% of overall implementation standards (an improvement over the 47% that were met the previous year). This ranged from one program site that met only 29% of the standards to another that met 87%.

In small programs (under 30 youth), there were no significant differences between treatment and comparison group youth in regards to whether youth spent time unsupervised after school. In large programs, however, treatment group youth were significantly more likely than comparison group youth to spend time unsupervised after school (p < .01).

Observations indicated that programs used 88% of positive youth discipline management practices specified by evaluators prior to the evaluation, such as clear and communicated expectations for behavior, consistency in managing behavior, effective monitoring of behavior, and immediate responses to misbehavior.

Overall, 18 of 22 programs operated the required 90 days during the school year. The programs averaged 110 days of operation over the school year.
Recruitment/Participation Every program recruited at least 20 participants, but four programs failed to serve 20 youth each month.

Only two programs met the attendance standard of having all youth attend 80% of all program days. The percentages of youth meeting the 80% standard ranged from 38% to 100% across sites.

Overall, 17% of youth dropped out of the program between November and March. Only two programs failed to meet the standard that no more than 33% of youth would withdraw during this period.

Approximately 25% of youth said they usually went to the program fewer days than they are expected to go.
Staffing/Training Only one program met the standard of having all direct-service staff attend youth development training. However, many sites had staff who worked in the educational field and may have had previous training in youth development.

Twenty of the 22 programs met the staff-to-youth ratio standard of 1:10. The average ratio was 1:5.


Summative/Outcome Findings

Academic While the comparison group began the school year with a significantly higher GPA than the treatment group (p < .05) and both groups declined slightly in GPA from the 2000–2001 school year to the 2001–2002 school year, the treatment group experienced a significantly greater decline in GPA than the comparison group (p < .01).

At the posttest, more comparison than treatment youth reported often paying attention in class (71% vs. 57%, p < .01). This gap was smaller in programs that served up to 30 youth as compared to programs that served more than 30 youth (p < .01).

Despite similar levels of commitment to education at the pretest, the comparison group increased in commitment while the treatment group decreased in commitment over the course of the year, resulting in significant differences at the posttest (p < .01).

At the posttest, more comparison than treatment youth reported turning homework in on time (61% vs. 53%, p < .05).
Prevention While the comparison group reported significantly more rebellious behavior at pretest (p < .05), the treatment group increased in their rebellious behavior at a rate that significantly surpassed the comparison group as measured at posttest (p < .01).

While the treatment group reported higher (although not significantly) intentions to use drugs at pretest than the comparison group, the treatment group had significantly lower intentions to use drugs than the comparison group at posttest (p < .10).

At the posttest, more comparison than treatment youth reported that they would try marijuana or other drugs (18% vs.12%, p < .05).

In programs with more structured social skills activities, significantly more comparison than treatment youth reported intentions to smoke cigarettes at the posttest (19% vs. 12%, p < .05). There were no significant differences between these groups on this measure in programs with less structured social skills activities.

Treatment youth in large programs (over 30 youth) reported that they had significantly more friends who sold drugs in the last year than comparison youth (p < .01), while there were no significant differences for the two groups in small programs (up to 30 youth).

There were no significant differences between treatment and comparison groups for actual drug use overall and frequency of drug use in the last month, nor differences in the last year for variety of drug use, use of cigarettes, alcohol, or marijuana, or of selling marijuana.
Youth Development While both treatment and comparison group youth reported similar levels of social skills on the pretest, significantly more comparison than treatment youth reported that they often followed their teacher's directions at the time of the posttest (70% vs. 55%, p < .01).

There were no significant differences between the treatment and comparison groups for: attachment to prosocial adults, involvement in constructive activities, belief in rules, or positive peer associations. This lack of significant youth outcomes held regardless of whether programs were categorized as using efficient procedures, high level of behavior management, high levels of overall structure, more time in academics, or higher levels of youth attendance.

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project