You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Program Description

Overview The Project for Neighborhood Aftercare (PNA) is a school-based after school program in Nashville, Tennessee serving the children in the neighborhood of each sponsoring school for one hour before school and two hours after dismissal during school days. Services are provided to families at no charge, with the exception of a nominal registration fee on enrollment. PNA offers academic, recreational, and cultural enrichment activities while providing students with a safe, supervised environment during the after school hours. These services are intended to impact academic performance, social skills, citizenship, and delinquency.
Start Date 1997
Scope local
Type after school, before school
Location urban
Setting public school
Participants kindergarten through eighth grade students
Number of Sites/Grantees four schools in 1998–1999; 14 schools in 2003–2004
Number Served approximately 200 in 1998–1999; 1,000 in 2003–2004
Components The objectives of the PNA program are to: (1) provide a safe haven for children in their neighborhood by providing caring adult supervision during critical hours following school dismissal; (2) enhance academic performance by providing homework assistance and drilling on core curriculum skills; (3) decrease delinquency through caring adult supervision, decreased contact with delinquent peer groups, homework support, promotion of educational values, pro-social activities, social skills training and promotion of self-esteem; and (4) provide culturally enriching experiences that participants would otherwise not have (e.g., performing in a holiday show, Kwanzaa celebration, creative dance, poetry, crafts, and cooking).

Each program site is chartered to enroll any mainstream education student, kindergarten through eighth grade, who either attends the school or resides within a 1.25-mile radius of the PNA program site. Three of the four original sites were located in low-income, high-crime urban neighborhoods, and the fourth school was located in a rural, low-to-moderate-income area, with a relatively low crime rate. The program has since expanded to 14 sites, and seeks to serve a diverse population across Nashville and Davidson County, including urban and rural areas, low- and low-to-moderate-income areas, and high- and low-crime areas. Most students who attend the program live within walking distance of the PNA school and 80–95% of the program participants attend the school that houses the program.
Funding Level $226,600 in 1997; $772,268 in 2003–2004
Funding Sources Metropolitan Government of Nashville & Davidson County; 21st Century Community Learning Centers grant; parent fees (originally free during the time of the evaluation; currently charges a nominal fee [$30 for after care, $10 for before care]) for registration in the program.
Other Originally, the Child First Team, a nonprofit organization launched by Tying Nashville Together (TNT), provided oversight for the PNA program. The fiscal agent for the PNA program was the Metropolitan Department of Social Services. The Child First Team Liaisons for each PNA site met periodically with a PNA site director and parents in PNA site committee meetings that provided the PNA program with a structure for feedback, goal identification, and program development. In 2003–2004, a Board of Directors provides oversight for PNA, consisting of one staff person from the Metro School Department of Community Education, two school principals from PNA sites, one staff person from the Mayor's Office for Children and Youth, two members of TNT congregations who have helped develop PNA, and three parents of children enrolled in the program.

The fiscal agent for the program is now the Mayor's Office for Children and Youth. Originally run during the time of the evaluation as purely an after school program (3–6pm), the program switched to a mixed before and after school program in 2000 after the school superintendent changed the middle school start time to 8:45am, allowing for a before school program to accommodate parents' child care needs. The program, as a school and community-based public service, has many stakeholders, including parents, students, teachers, school principals, PNA staff, the Child First Team, and the local government.


Evaluation

Overview A formative assessment was conducted of PNA's first year of operation, when the program was run solely as an after school program at four school sites.
Evaluator Vanderbilt Institute for Public Policy Study's Center for Evaluation Research and Methodology and the Center for State and Local Policy
Evaluations Profiled Final Report on a Formative Evaluation of the First Year of the Project for Neighborhood Aftercare (PNA): A School-Based After-School Program
Evaluations Planned None
Report Availability King, R. D., Lipsey, M. W., Shayne, M. W., & Hoskins, A. (1998). Final report on a formative evaluation of the first year of the Project for Neighborhood Aftercare (PNA): A school-based after-school program. Nashville, TN: Vanderbilt Institute for Public Policy Studies.


Contacts

Evaluation Mark Lipsey, Ph.D.
Vanderbilt Institute for Public Policy Studies
1207 18th Ave. South
Nashville, TN 37212
Tel: 615-322-8505
Program Project for Neighborhood Aftercare
1001 18th Ave. South
Nashville, TN 37212
Tel: 615-327-1670
Profile Updated October 28, 2003

Evaluation: Final Report on a Formative Evaluation of the First Year of the Project for Neighborhood Aftercare (PNA): A School-Based After-School Program


Evaluation Description

Evaluation Purpose To examine several aspects of PNA program operation: participation characteristics and patterns, program activities, satisfaction, improvements in school performance, study habits, attendance or attitudes toward school, and suggestions for improving the program. The evaluation's primary focus was to provide feedback to the program management and funders in order to assist program development and successful implementation. At the beginning of the first program year, PNA used a set of flowcharts to document the intended outcomes and concepts for achieving those outcomes. Future assessment of program impact would involve comparing PNA student measures over time with those of similar students not enrolled in PNA. The evaluation team determined that an assessment of the program's impact on prevention of delinquency would not be fruitful, but that determining the program's impact on academic performance would be feasible in future years.
Evaluation Design Non-Experimental: Data were collected from several sources during numerous visits to each site.
Data Collection Methods Document Review: Data were collected from PNA quarterly reports and related program documents and a literature review.

Interviews/Focus Groups: The evaluation began with interviews of the program staff to learn what the goals of the program were, how the program intended to achieve these specific goals, and what actions of the program were considered key to achieving the goals of the program.

The resulting program logic models or flowcharts were then presented individually to the staff for confirmation that the logic models accurately reflected how the program was intended to make a difference in the lives of students. The program's goals and logic models are expected to evolve with the program.

Interviews were also conducted with school principals and members of the program oversight committee to learn more about program operations and context.

Phone surveys were conducted with parents/caregivers in PNA participant households. These surveys were used to gather demographic information and information about level of participation in PNA, reasons for participation, other after school opportunities, employment status, satisfaction with PNA, and suggestions for program improvement.

Phone surveys were also conducted with parents/caregivers in eligible neighborhood households. These surveys were used to gather demographic information and information about child's participation in other after school programs, satisfaction with these programs, awareness of PNA and reasons for not participating, and types of program services desired in an after school program.

Observation: Observation of programs, staff, and oversight meetings were conducted during site visits.

Secondary Source/Data Review: Student records data, crime mapping data, and neighborhood income data were collected.

Surveys/Questionnaires: A survey administered to parents who were picking up children from the program was used to gather information about parent satisfaction with the PNA program and perceptions about the influence of the PNA program on the child's motivation at school, school performance, self-esteem, attendance, homework habits, and relationships with teachers and classmates. A survey administered to teachers was used to gather information about the influence of the PNA program on students' homework habits, motivation to succeed in school, behavior, self-esteem, expectations, and grades and on parent involvement in the school.
Data Collection Timeframe Data were collected during the 1997–1998 school year.


Findings:
Formative/Process Findings

Activity Implementation Site visits by the evaluation team found the PNA program to be providing the following services as planned: caring adult supervision, a safe environment after school, regularly scheduled homework time, homework tutoring and academically enriching activities, staff demonstrating and rewarding social skills, and culturally enriching activities.

Program sites vary in the amount of enrichment provided through affiliation with other groups. These opportunities included: behavioral counseling, presentations by Department of Health staff, grade-level tutoring, individual tutoring and reading help, and performance and literature activities.

When asked about the kinds of services they are looking for in an after school program, parents identified: academic enhancement, structured environment, physical activity, convenience, social skills training, cultural enrichment, adult supervision, and fun.

Participants' parents provided two categories of suggestions for improving the program: PNA should provide more of the services they already provide and PNA should add new features to the program.
Program Context/Infrastructure After school care alternatives for PNA families include fee-based programs, church-oriented programs, parks centers, and other school-based after school programs.

If the PNA program were not available, all but one of the parents surveyed on-site said that their children would be at home under adult supervision or playing in the neighborhood while an adult was home. Interviews of teachers presented a different story. Teachers reported that over 30% of students returned to homes where adults were not present after school.
Recruitment/Participation Program-eligible students live within a 1.25-mile radius of their PNA school and are enrolled in mainstream education classes from kindergarten to eighth grade.

Comparisons between participants and nonparticipants were made using Metro Department of Education demographic data on race, grade level, and a proxy for low income, i.e., enrollment in a reduced-cost-for-lunch or free-lunch program. No large differences were found between PNA participants and eligible nonparticipating students. Metro Department of Education records show that the characteristics of PNA students are similar to those of all students in their neighborhoods in terms of race (70% African American, 24% white) and participation in reduced/free-lunch programs (69% non-PNA, 85% PNA).

Three of the four program sites are located in low-income, high-crime areas of Nashville while the income and crime rates for the fourth site are, respectively, moderate and low.

Some of the program sites faced challenges in terms of enrollment (as low as 25% of capacity during a two-week period at one site). Phone surveys of 50 neighborhood households whose children were not enrolled in PNA indicated that 62% were unaware of after school programming in their neighborhood, 32% said that they had their children enrolled in some form of after school program, and 10% said that they preferred having the child at home with them after school. After canvassing with flyers and networking for teacher referrals of students, enrollments at sites returned to above 75% of full enrollment within a month.

The PNA program serves approximately 7% of the eligible population and had many fifth and sixth graders (41% of enrollment), but fewer seventh and eighth graders (11% of enrollment.)

Phone surveys of 64 PNA households found that 22% were single parents; 78% of PNA participant care providers reported working outside the home.

Barriers to participation included: transportation, parental choice, student choice, or prior enrollment in another program or athletic team.

The responses to the question of why participants chose to enroll in the program were sorted into seven categories of reasons for enrollment: work schedule/transportation from school/convenience, adult supervision, student or teacher interest, affordable/free, academic/cultural enrichment, prevention, and miscellaneous.

In response to a question about why some participants stopped using the program, several reasons were cited: transportation problems, parental preference for having the child at home, the program was perceived as remedial, and children not getting along with others in the program.
Satisfaction The on-site parent survey produced an overwhelmingly positive assessment of the program. Program satisfaction was high for both parents and students, with 72% of students answering that they enjoy the program and parents surveyed on-site and in phone interviews expressing high levels of satisfaction with the program, with an average 4.47 score on a five point scale.

Although the PNA program is free to parents, the survey indicated that all parents would be willing to pay a fee for the PNA program the next year. This was qualified by how much the fee would be.


Summative/Outcome Findings

Academic In on-site interviews, parents endorsed the program for improving their children's academic motivation, homework performance, attendance, and grades. Teachers were also generally positive about the program's capacity to reduce problem behavior, build student expectations for success in school, and enhance students' learning skills.

 

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project