You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Program Description

Overview The North Carolina Support Our Students (SOS) initiative is an effort by the state of North Carolina to encourage quality after school programs for students in both urban and rural communities. Administered by the North Carolina Department of Juvenile Justice and Delinquency Prevention, SOS awards grants to nonprofit, 501(c)(3) organizations to run quality after school programs for students. The specific goals of the program are to: reduce juvenile crime and the number of young people who are unsupervised after school; improve youth participants' academic performance, attitudes, and behavior; meet the physical, intellectual, emotional, and social needs of young people; involve community volunteers; improve the coordination of existing resources; and enhance collaboration between agencies.
Start Date March 1994
Scope state
Type after school, before school, summer/vacation, mentoring
Location urban, rural
Setting public school, community-based organization, recreation center, religious institution, private facility
Participants elementary and middle school students (72% middle school)
Number of Sites/Grantees 244 sites (190 school-based and 54 community-based) in 98 counties in 2001–2002 and 100 counties in 2002–2003
Number Served Twenty three thousand in 2001–2002—16,833 were served during the regular school year, while another 10,000, some of whom were in the regular school year programs, were provided services either during the summer months or when students in year-round schools were tracked out. Tracked out refers to the periods during the calendar years for year-round schools in which students are out of school—typically three-week periods.
Components Although overall goals were constant across all sites, each SOS site was given the autonomy to create programs based on what they determined would benefit the students in their community. All of the county programs offered homework assistance, nearly all offer tutoring in math and English or reading, and many have a mentoring program. Most SOS programs provide one-on-one tutoring and teach students good study habits. Other widespread program components include field trips, sports, arts and crafts, games, community service opportunities, computers, and various life skills, conflict resolution, anger management, and/or violence prevention activities.
Funding Level Grants ranged from $60,000 to $250,000 per year for each of the 98 counties. In addition, many sites supplemented these funds with in-kind contributions and outside funding sources. Total funding from the state is $12.5 million.
Funding Sources North Carolina Department of Juvenile Justice and Delinquency Prevention (DJJDP), other outside sources (as secured by individual SOS counties or sites)


Evaluation

Overview Each year, EDSTAR evaluates program activities and participant outcomes by collecting data on program participants and activities. Each evaluation looks at who participated and how they benefited from various services, documents which practices contribute to reaching desired outcomes for children, and provides information that can be used to improve programs.
Evaluator EDSTAR
Evaluations Profiled Evaluation Report North Carolina Support Our Students: 2001–2002
Evaluations Planned Evaluations are scheduled to continue on an annual basis.
Report Availability Johnson, J. (1999). North Carolina Support Our Students: 1998–1999 evaluation report. Raleigh-Durham, NC: Author.

Johnson, J. L., & Jenkins, D. R. (2000). North Carolina's Support Our Students: Evaluation report for school year 1999–2000. Raleigh-Durham, NC: EDSTAR Educational Evaluation Consultants.

EDSTAR. (2001). North Carolina's Support Our Students: Evaluation report for school year 2000–2001. Raleigh-Durham, NC: Author.

EDSTAR. (2002). Evaluation report North Carolina Support Our Students: 2001–2002. Raleigh-Durham, NC: Author.

Available at www.edstar.org/reports.html.


Contacts

Evaluation Janet Johnson
President
EDSTAR, Inc.
6308 Lakeland Dr.
Raleigh, NC 27612
Tel: 919-844-0529
Fax: 919-676-7458
Email: evaluations@edstar.org
Program Leigh Hines
Communications Director
Department of Juvenile Justice and Delinquency Prevention
1801 Mail Service Center
Raleigh, NC 27699-1801
Tel: 919-733-3388 ext. 281
Fax: 919-733-6809
Email: leigh.hines@ncmail.net
Profile Updated April 25, 2003

Evaluation: Evaluation Report North Carolina Support Our Students: 2001–2002


Evaluation Description

Evaluation Purpose To accomplish three primary goals: (1) describe who was served by SOS programs and how they benefited from services, (2) identify practices that resulted in desired outcomes for youth participants, and (3) provide information that can be used to improve programs.
Evaluation Design Quasi-Experimental and Non-Experimental: Data were collected from surveys and interviews with 98 county SOS program directors, 4,825 SOS students in first through eighth grade, students' classroom teachers, and records of school absences, academic outcomes, and standardized achievement test scores. Test score data was collected for 2001–2002 participants from multiple years, allowing the researchers to compare SOS students to themselves over time.

Students surveyed were in grades 1–8 and were from the regular after school portion of the program. The evaluators surveyed all students at all sites. However, at some sites, not all students took the survey and at other sites 100% took it. So for the state report, the evaluation used a stratified random sample of the surveys submitted so that each county was represented proportionately in the sample. The sample size was determined by working backwards from the smallest county return rate, which was 29%, to the largest county return rates of 100%.

For every SOS participant who attended the SOS program for 30 or more days site coordinators were required to collect information from that student's math and language arts teachers. While there were no formal response rates for these teacher surveys, sites are required to collect this information and the evaluators report that they have strong compliance from sites. Many teachers had multiple SOS students in their classroom. EDSTAR aided the programs in collecting this information by sending letters to the schools explaining the evaluation process and by making forms for each teacher to facilitate data collection.
Data Collection Methods Interviews/Focus Groups: Telephone interviews were conducted with county program directors to identify the specific activities offered by each county program, as well as the hours of operation and average daily attendance at all sites. Information was also collected regarding volunteers and the degree to which programs succeeded in securing other grants and assistance from collaborating agencies.

Surveys/Questionnaires: Web-based surveys of program directors collected information regarding program activities, attendance, and other program-related data. Student participant surveys collected information on demographic indicators, perceptions of the SOS programming, and various student outcomes of interest. Classroom teacher surveys collected information on SOS participants and potential benefits of SOS programming.

Tests/Assessments: North Carolina End-of-Grade (EOG) achievement test scores were collected for SOS participants from the 2001 EOG (pretest) and 2002 EOG (posttest) reading and math tests. Students' scale score gains on these tests were compared to state standards for improvement as well as grade-level standards of proficiency. Site coordinators collected roster data on forms created by EDSTAR. The data collected included the students' names, school, ID used by the NC School System for data tracking, and program attendance. From these, EDSTAR created one large file of names and student IDs and the North Carolina Department of Public Instruction provided the following data for the students: race, gender, three years of standardized test scores, attendance records, parent education level, grade level, lunch status, and special education status. The Department of Public Instruction then gave EDSTAR the new data, keeping the school name, site name, and county variables, but stripping off individual student identifiers. This way no confidentiality was breached, yet the data set could be analyzed overall and for subgroups within counties.
Data Collection Timeframe Data were collected during the 2001–2002 school year (although this data includes North Carolina Department of Public Instruction Data from the 1999–2000, 2000–2001, and 2001–2002 school years for 2001–2002 SOS students).


Findings:
Formative/Process Findings

Activity Implementation At many sites, points or tokens, often called “SOS bucks” or “club bucks,” were issued to students for positive attitude and behavior. Once a week or month, these bucks could be redeemed for items at an SOS “store” or an SOS auction.

Over 80% of programs used some sort of behavior modification system in their curriculum. Examples include behavioral contracts, courses in martial arts to instill self-confidence and responsibility, and rewards such as pizza parties and a special end-of-the year trip to an amusement part.

Field trips included visits to museums, plantations, hospitals, prisons, factories, bowling alleys, skating rinks, and shopping malls.

Nearly two-thirds of SOS participants reported using computers at least “once in a while” in the SOS program.

Community service projects were considered by project directors to be an important aspect of SOS. Examples included reading to elderly residents at nursing homes, picking up trash, planting flowers, working at a food bank, and collecting donations for a rescue mission. One site worked with organizations aiding women and children displaced due to domestic violence, prison, job loss, etcetera, and chronicled their experiences and interviews with clients and staff through a video documentary.
Cost/Revenues Nearly all counties used volunteers, which was estimated to have saved SOS $1.1 million for the school year, based on the assumption that programs would have had to hire tutors at $15 per hour if they had not used volunteers.

The value of in-kind contributions (e.g., use of school facilities, transportation, etc.) was estimated to be over $2.5 million.

Statewide, the average program cost in SOS dollars (dollars paid by the state of North Carolina through DJJDP) to provide after school services for one student was about $3 per hour. This cost has decreased by $0.50 per hour each year for the last two years. This decrease in cost stemmed primarily from EDSTAR's efforts in training sites in grant writing, which allowed individual sites to generate more money from outside sources. Many sites also increased their volunteer recruiting efforts.

Roughly 42% of county SOS programs charged a fee for services. These fees ranged from a one-time $5 registration fee to $6 per day. Across the state, over half a million dollars in revenue was raised through these fees. Ninety percent of SOS programs that charged a fee for services offered full scholarships to students who qualified.
Recruitment/Participation The average daily attendance for all 98 counties combined was 8,718 students.

SOS programs served approximately the same number of males and females.

About 54% of SOS participants were black, 39% were white, 4% were Hispanic, and the remainder were Asian, Native American, and other.

Of all SOS participants, 57% were eligible for free or reduced-price lunch.

On days when students were not attending an SOS program, 25% of the elementary school children and 44% of the middle school children were usually not supervised by an adult.

At the beginning of the year, about one-third of SOS participants were not proficient in reading at their grade level as measured by the 2001 North Carolina End-of-Grade (EOG) reading tests.

Overall, 39% of SOS participants came from single-parent households, although this figure combines some counties where over 80% of children came from single-parent homes and some where less than 20% came from single-parent homes.

Fifty-one percent of SOS students' parents had a high school diploma, 37% had formal education beyond high school, and 11% did not finish high school.

About 18% of SOS students had a learning disability or an emotional or physical handicap and another 8% were classified as gifted.

Most counties had specific attendance requirements and served only regular attendees (students attending the program at least 30 days). Programs used incentives such as SOS bucks and field trips to encourage students to attend as often as possible. Most of the students were enrolled four or five days per week.
Satisfaction More than 70% of participants said they would recommend the program to a friend.
Staffing/Training Across the state, 1,258 individuals worked in SOS county programs in 2001–2002, but only about two-thirds of these staff members were paid with SOS funds.
Systemic Infrastructure Approximately 69% of program directors reported that they used the evaluation reports provided by DJJDP for accountability to other funding sources, obtaining a total of more than $4.6 million from these funding sources.

The number of sites has increased steadily over the previous years, growing at about 10% per year.


Summative/Outcome Findings

Academic On the student survey, 41% reported liking school more than they did before starting the SOS program, more than six times more than those who said they liked school less than they did before joining SOS.

The percentage of middle school SOS participants receiving out-of-school suspensions decreased as compared to their previous year in school (from 13% to 8%), as did the percentage of in-school suspensions.

Classroom teachers reported that 41% of participants had improved behavior in math class throughout this school year, and 42% had improved behavior in English/language arts classes.

At every grade (of grades five through eight) except sixth, SOS participants who attended SOS at least 30 days demonstrated mean EOG reading scale score improvements that exceeded the state's improvement goals.

The percentage of students who scored at grade-level proficiency increased in both reading and math from 2000–2001 to 2001–2002 (from 67% to 71% at grade level in reading and 77% to 81% at grade level in math).

African-American SOS participants made greater gains in EOG reading scores compared with every other ethnic group (p<.0001). This was true after also controlling for baseline achievement levels and risk factors.

Minority SOS participants made nearly twice the gains of white SOS students in EOG reading scale score gains.

Hispanic SOS participants made the greatest gains in EOG math scores, followed by African Americans (p<.01), although these gains were not as great as the differences observed in reading scores.

Classroom teachers reported that over 40% of regularly attending SOS participants improved their grades in English and/or math.

More than two-thirds of participants who have been involved in the program for three years improved two proficiency levels in their EOG scores. The average yearly improvement in EOG scores for SOS participants was almost half a proficiency level.

In EDSTAR's student survey, when program participants were asked to identify characteristics of an ideal after school program, more students chose homework assistance than any other category. Almost half (47%) of students surveyed said they completed their homework during SOS, but didn't believe they would have completed it if not there.

Students most frequently identified the math homework assistance as offering the most help, with 49% of students reporting that the math assistance helped “a lot,” and another 26% reporting that it helped “some.” For reading/language arts, no students reported the assistance helping “a lot,” although 85% reported that it helped “some.”

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project