Search Publications | The Evaluation Exchange | FINE Network | OST Database

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

A Profile of the Evaluation of Summer Career Exploration Program

Program Description

Overview	The Summer Career Exploration Program (SCEP) is a summer jobs program for low-income teens in Philadelphia and Delaware counties in Pennsylvania and in Camden, New Jersey. One of the largest private-sector youth-jobs programs in the country, SCEP is designed to enhance teens understanding of the connection between academic achievement and career success, provide youth with adult support, and offer meaningful career-related jobs in the private sector.
Start Date	1983
Scope	local
Type	summer/vacation
Location	urban
Setting	community-based organizations

Participants	high school students (who have completed Grades 10, 11, or 12)
Number of Sites/Grantees	28
Number Served	1,633 in 1999
Components	At the time of the evaluation, SCEP was administered through the Philadelphia Foundation, which gives local sponsoring agencies responsibility for recruiting youth, developing summer work placements, managing mentors, and providing other services. The program is open to teens who come from families with incomes less than 150% of the federal poverty level and who are enrolled in school and have completed the 10th or 11th grade, or have just graduated in the previous school year. Students may participate in the program for up to three summers. Participating agencies recruit employers and young people, and are expected to ensure that at least 25% of the teens continue working, at the employers expense, beyond the 6-week program. SCEP has four major programmatic elements: (a) summer work in the private sector, (b) pre-employment training for workplace readiness, (c) emphasis on the value of education, and (d) personal support from college student monitors and work site supervisors. Students work for 25 hours a week for 6 weeks during the summer, receiving minimum wage for their work. A program stipend pays for the first 20 hours, and employers pay for the additional 5 hours. SCEP attempts to match jobs with teens career interests. Students are employed at a wide variety of jobs depending on the location of the site. In 1999, participants worked in retail businesses, insurance agencies, real estate agencies, child care centers, health care organizations, law offices, computer service companies, small manufacturers, travel and hotel service providers, restaurants, landscaping companies, the automotive service industry, and sales agencies. All participants undergo pre-employment training. The bulk of this training focuses on “soft skills” and how they apply to interviewing for and maintaining a job. Depending on the site, participants also learned about career choices, job readiness (including proper verbal communication and body language, hygiene and dress, proper attitude, timeliness, and how to ask questions), working with supervisors, time management, accepting criticism and feedback, and thinking critically about the summer employment experience and how it related to their career goals. SCEP stresses the value of education through the use of college monitors as role models for appropriate workplace behavior, demeanor, and dress. SCEP also has formal ties to College Access Centers and encourages participants to learn about the college admission process. Monitors also provided personal support for SCEP participants, and were expected to serve as entry-level counselors, providing broad-based academic, career, and personal counseling. Monitors “shadowed” participants to advocate on their behalf, coach them to succeed, and suggest ways to address problems encountered in the workplace. Monitors were expected to visit each student twice a week at work to ensure that employers provided students with a well supervised and safe work experience, and that students were meeting employers expectations.
Funding Level	$1,624,936 in 1998 and $1,712,422 in 1999
Funding Sources	Primary funding for SCEP comes from the William Penn Foundation, and the program is also supported by a collaborative of 36 foundations, corporations, and trusts.

Other

SCEP is currently administered through the Philadelphia Youth Network. The Philadelphia Youth Network (PYN) is a nonprofit intermediary organization dedicated to youth workforce development and designed to align programs, systems, and resources that promote academic achievement, career success, and productive citizenship for all 14- to 21-year-old youth residing in the city of Philadelphia.

Evaluation

Overview	The evaluation documents the programs implementation and examines its efficacy in terms of job placement, youth participation, and short-term and intermediate impacts on participants. One goal of the evaluation was to see what could be learned from SCEP to help improve other summer youth employment efforts.
Evaluators	Wendy S. McClanahan, Cynthia L. Sipe, and Thomas J. Smith, Public/Private Ventures
Evaluations Profiled	Enriching Summer Work: An Evaluation of the Summer Career Exploration Program
Evaluations Planned	none
Report Availability	McClanahan, W. S., Sipe, C. L., & Smith, T. J. (2004). Enriching summer work: An evaluation of the Summer Career Exploration Program. Philadelphia: Public/Private Ventures. www.ppv.org/ppv/youth/youth_publications.asp?section_id=9

Contacts

Evaluation	Wendy S. McClanahan Senior Policy Researcher Philadelphia Office 2000 Market Street, Suite 600 Philadelphia, PA 19103 Tel: 215-557-4400 Fax: 215-557 4469 Email: wmcclanahan@ppv.org
Program	David Suter John F. Kennedy Center, Room 580 734 Schuylkill Avenue Philadelphia, PA 19146 Tel: 215-875-3180 Fax: 215-875-5740 Email: pyninfo@pyninc.org
Profile Updated	January 31, 2005

Evaluation: Enriching Summer Work: An Evaluation of the Summer Career Exploration Program

Evaluation Description

Evaluation Purpose	To answer the following questions: (a) How is a privately funded, multi-agency summer jobs program that targets low-income youth designed and implemented? Is it successful in placing young people in jobs? What support services does it provide? (b) What kinds of young people participate in SCEP? (c) Does SCEP have short-term and intermediate impacts on participants?
Evaluation Design	Experimental and Non-Experimental: This study consisted of two components: an initial implementation study of 15 agencies implementing SCEP in 1998 and a random assignment impact study conducted in 1999. Youth who were applying for a 2nd or 3rd year of participation were excluded from the random assignment process and subsequent data collection. The implementation study consisted of case studies of 15 high-functioning sites chosen randomly. The sample reflected the diversity of the various agencies operating SCEP as well as a range in program size, location, and affiliation with an umbrella organization. From March 1999 through July 1999, 1,708 first-year SCEP applicants were randomly assigned to either the treatment (n = 1,157) or control (n = 551) groups. Members of the control group were allowed to enter the treatment group after 1 year. Pretest data were collected from the entire sample prior to the program, and posttest data were collected from 93% of the sample 3 months later and from 89% of the sample 1 year after their application to the program. Sample sizes varied according to how many youth had complete information for each assessed outcome area. No systematic differences were found between the treatment and control groups at the baseline pretest. While all treatment group members were included in the analysis, 22% were unable or unwilling to take a SCEP job (14% found employment through their own resources, while 8% were did not find employment through any means). In addition, 17 youth assigned to the control group were omitted from the analysis because their control status was compromised by being placed in a SCEP summer job. The evaluation sample was more female than male (62% vs. 38%). Participants were predominantly minority, as 72% were African American, 17% were Hispanic, 5% were Asian, and 6% were White or other race/ethnicity. Students had just completed either 10th (56%), 11th (31%), or 12th (14%) grade prior to entrance in the program.
Data Collection Methods	Interviews/Focus Groups: At the 15 implementation study sites, evaluators conducted face-to-face interviews with program coordinators, college monitors, and worksite supervisors, a series of three in-depth interviews with SCEP participants, and focus groups with participants and monitors. Observation: At the 15 implementation study sites, evaluators conducted observations of participants at their worksites, and shadowing of the monitors and program coordinators for a half day. Secondary Source/Data Review: Unemployment insurance and criminal records were collected for both treatment and comparison groups for the 18 months after the summer of 1999 ended. Surveys/Questionnaires: All of the participants in the program, including both treatment and control groups, completed a baseline questionnaire when they applied. This questionnaire included questions about demographics (e.g., gender, race, grade level) as well as career interests, academic characteristics, work orientation, self-efficacy, and work readiness. A 3-month follow-up survey consisted of questions about participants summer employment history and experiences with SCEP. A 1-year follow-up survey was completed prior to the start of the 2000 program, since both control and treatment youth were permitted to participate in SCEP during the summer of 2000. This interview consisted of questions about the academic impacts of SCEP, orientation toward work, and employment.
Data Collection Timeframe	Data were collected between 1998 and 2000.

Findings:
Formative/Process Findings

Activity Implementation	The implementation study revealed that most of SCEPs activities for ensuring that participants were well prepared for their work experiences focused on how to apply for jobs, improving interview skills, and how to maintain a job. Some agencies provided instruction only prior to the actual job placement, while others conducted mandatory training sessions throughout the summer. Some participants received limited counseling about career choices and how to think about the tasks they enjoyed and the work environment they preferred. Participants also learned about job readiness, focusing on issues such as proper dress and hygiene, communication, and time management. Implementation data revealed that SCEP coordinators used a variety of techniques to develop jobs, including writing letters to their agencies board members, colleagues, individuals on agency mailing lists, and newly opened businesses in the immediate area. Once a roster of work sites was compiled, SCEP coordinators interviewed representatives at each work site to gauge the safety of the environment and the employers commitment to the programs goals. Implementation data revealed that many coordinators attempted to develop sites near their agencies, where most participants lived, in order to help keep students transportation costs down. However, some agencies branched out to other areas, particularly Center City Philadelphia, to accommodate students career interests.
Recruitment/Participation	When the baseline survey was administered, 86% of participants believed they could get enough information about college, 72% planned to take or had already taken the SAT or ACT, 69% had received mostly Bs and Cs or higher on their last report card, 64% planned to attend a 4-year college after high school, and 59% had visited a college campus. Of participants, 33% had never before worked for pay, 51% had worked for pay before but were not working at the time of application, and 16% were already working at the time when they applied to SCEP. At the time of the baseline survey, participants indicated that they were interested in careers with the following characteristics: those in which they could help others (58%), in sales or public relations (35%), in computer programming (34%), in clerical/secretarial positions (33%), involving reading/writing (30%), in professional positions (31%), involving math skills (20%), in scientific positions (13%), and in mechanical positions (13%). At baseline on the work readiness scale (where 4 is the highest score), 86% of respondents had an average item score of 3 or higher and 16% had an average score of 3.75 or higher. For the scale measuring motivation to do good work when they enrolled in the program (where 4 is also the highest score), 92% of respondents had an average item score of 3 or higher and 28% had an average score of 3.75 or higher.
Staffing/Training	The program hires college student monitors to support teens throughout their summer job experience. Since college attendance is an aspiration that SCEP hopes to instill in participants, agencies hire college students with backgrounds similar to the teens in order to serve as role models. In summer 1998, SCEP program coordinators hired 165 college monitors similar in gender and race to the participants. One third of them had worked as SCEP monitors previously, and one fifth had participated in the program in high school. The majority of college monitors attended local colleges and universities. Although they pursued a wide variety of majors, including finance, communications, business, biology, chemistry, and fine arts, majors in education and the social science dominated. Most program coordinators had little trouble recruiting interested college students, and many sponsoring agencies received two or three times as many applications as needed in 1998. Recruitment methods that year included referrals from other monitors, notices in local newspapers, college placement offices, work-study programs, and agency newsletters. Word of mouth was a significant factor in recruitment: Participating organizations got referrals from current and former monitors, as well as from other SCEP sites. SCEP required all monitors to receive at least 8 hours of training but strongly encouraged agencies to provide 16 hours. The training regimen included information about the structure, background, and goals of the SCEP program, as well as preparation for their roles as mentors and counselors. Newly hired monitors also attended Monitor Plus, a supplemental training session conducted by some of SCEPs more experienced program coordinators. Monitor Plus underscored the topics covered at the agency-level sessions. Of first-time SCEP participants in 1999, about 62% saw their college monitors at least twice a week, 37% saw their monitors about once a week, and less than 1% of participants said their college monitors did not visit them at all. More than 85% of participants said they talked with monitors about their relationships with coworkers and their supervisors. Almost as many youth said they discussed post-high school plans with their college monitors, and about two thirds discussed strategies to develop their work skills and explore their career interests. Youth were least likely to report discussing personal issues with their college monitors (22%).

Summative/Outcome Findings

Academic	No significant program effects were found at the 1-year follow-up for youths grades, class effort, courses elected to take in high school, or likelihood of graduating high school. No significant program effects were found at the 1-year follow-up for likelihood of taking college entrance exams, visiting a college campus, consulting adults about college applications and financial aid, or applying for financial aid for college in the year following their application to the SCEP program. Significantly more SCEP participants reported that they were enrolled in a college preparatory or specialized academic program at the 1-year follow-up (12% of SCEP participants vs. 8% of control group, p = .01). Significantly more SCEP participants than control group youth (44% vs. 30%, p = .001) had visited a College Access Center during the year following the program. Both groups were equally likely to believe they had obtained as much information as they needed about college.
Prevention	Analyses of criminal records 18 months after the program did not show a reduction in criminal or delinquent activity as a result of SCEP participation.
Workforce Development	Ninety-two percent of the treatment group worked during summer 1999, as opposed to 62% of youth in the control group (p = .001). No significant differences were found in the types of jobs that SCEP participants held when compared to those of the control group. SCEP participants were more likely to feel that they had received “personalized” training than the control group members (p = .05). Both groups were equally likely to agree that their supervisors gave clear instructions, were responsive to questions about how to do the work, trusted the teens to do their jobs, were not overly critical, and tended not to expect too much of them. Both SCEP participants (83%) and control group youth (81%) reported satisfaction with their summer work experiences. Working members of the control group received $5.69 an hour on average, significantly more than SCEP members average wage of $5.30 an hour (p = .01). This difference was primarily because SCEP ties its participants’ salaries to minimum wage. At the 1-year follow-up, there were no significant differences in the attitudes of the two groups toward work. No significant effects were found for feeling capable of making decisions about careers and reaching career goals 1 year after the program. Both groups reported receiving virtually the same level of support in career planning. SCEP participants reported significantly more confidence after the program in their ability to teach or hold a job that requires reading and writing than that of the control group 1 year after the program (p < .05). No significant effects were found for continuing working once school resumed, with about 60% of both groups continuing working once school resumed. In contrast with program goals, significantly fewer (32%) SCEP participants who received a job during the program stayed with the same employer in the following year compared with 45% of the control group (p = .001). No significant effects were found for school-year earnings or frequency and duration of employment. Both groups earned about the same hourly wage at their school-year job: $6.12 an hour. The working SCEP participants earned an average of $2,647, while working nonparticipants earned an average of $2,526 during the school year. Both groups worked an average of 18 to 19 weeks from September to June, at an average of about 23 hours per week. At the 1-year follow-up, no significant effects were found for the probability of continuing working (approximately 20% of each group had continued working at this time). No significant effects were found for the probability of receiving unemployment insurance 18 months after the program. No significant effects were found for the ability to see the connection between school and work 1 year after the program.
Youth Development	There were no significant differences between the number of SCEP participants and control group youth who reported at the 3-month follow-up that their jobs gave them a chance to learn new things, helped them learn to work with other people, or improved their ability to think and solve problems.

A Profile of the Evaluation of Summer Career Exploration Program

Profile Information