The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
A Profile of the Evaluation of Ascend Summer Youth Program
Profile Information
- Full Text (HTML)
- Request reprint permission
- How to cite
- About the OST Database
Program Description
| Overview | The Ascend Summer Youth Program (ASYP) used technology-supported, project-based learning experiences to enable youth in Washington, DC, to create meaningful digital products in a supportive environment. ASYP integrated mentoring, workforce readiness, and project-based learning experiences with a paying job. In addition, program staff used information technology to address social, affective, cognitive, and academic outcomes necessary for youth’s success in postsecondary schooling and the workplace. |
| Start Date | 2004 |
| Scope | local |
| Type | summer/vacation, mentoring |
| Location | urban |
| Setting | public school |
| Participants | high school |
| Number of Sites/Grantees | 1 |
| Number Served | 100 youth (2004); 500 youth (2005) |
| Components | ASYP provided activities for 4 hours per day during the work week over a 6-week period, during which youth were paid $5.15 per hour to serve as computer aides. Their duties as computer aides were to learn various computer applications in the context of program activities. Other ASYP activities centered around technology and career/work readiness, including (a) field trips to cultural/historic venues (e.g., museums) and local colleges; (b) Web research of postsecondary schooling options; (c) practical application of word-processing skills by preparing Microsoft Word tables comparing various career/postsecondary options; (d) developing PowerPoint presentations of goals and postsecondary options; and (f) panel discussions by college students on the challenges of getting into and staying in college. Youth were divided into small teams, which were then clustered into larger groups led by a group leader and supported by a youth assistant. Groups were further split into morning and afternoon sessions. A key component of Ascend’s approach is providing vicarious experiences through role models, which is achieved by employing youth leaders/peer mentors to deliver programming. Youth leaders are minority youth from local neighborhoods, many of whom have completed the Ascend school year program and attend college. Participants were referred by the Washington, DC Department of Employment Services and attended a number of different schools in the DC area. |
| Funding Level | $450,000 |
| Funding Sources | The Washington, DC Department of Employment Services and the DC Workforce Investment Council. |
Evaluation
| Overview | The evaluation sought to provide information to ASYP staff and other interested stakeholders about how the program was being implemented and what impact it seemed to have on youth participants. |
| Evaluators | Raymond McGhee, Jr., and Natalie Nielsen, SRI International |
| Evaluations Profiled | Evaluation of the Ascend Summer Youth Program 2005: Summative Report |
| Evaluations Planned | None. |
| Report Availability | McGhee, R., & Nielsen, N. (2005). Evaluation of the Ascend Summer Youth Program 2005: Summative report. Washington, DC: Ascend, Inc. Available at: www.youthengaged.org/summer2005.html |
Contacts
| Evaluation | Raymond McGhee, Jr. Research Social Scientist SRI International 1100 Wilson Blvd., Suite 2800 Arlington, VA 22209-2111 Tel: 703-524-2053 Fax: 703-247-8493 Email: mcghee@wdc.sri.com |
| Program | Mr. Joseph A. Davis II Director Ascend, Inc. 1025 Connecticut Ave, NW, Suite 1012 Washington, DC, 20036 Tel: 202-857-1468 Fax: 202-857-9799 Email: jdavisii@ascend21.org |
| Profile Updated | November 17, 2006 |
Evaluation: Evaluation of the Ascend Summer Youth Program 2005: Summative Report
Evaluation Description
| Evaluation Purpose | To answer the following questions: (a) What strategies did ASYP staff use to engage youth, (b) what contextual and organizational factors influenced the implementation of ASYP; and (c) how did ASYP benefit youth? |
| Evaluation Design | Quasi-Experimental and Non-Experimental: Evaluators collected data through program observations, participant surveys and logs, and informal program staff interviews. Each week, researchers observed one of the 5 ASYP groups, typically over a 3–4-hour period. In week 1, 345 out of 347 youth completed a pretest survey. In week 4, 237 of 343 youth still in the program completed a posttest survey. In week 6, 201 of 236 youth still in the program completed an exit survey. Each week starting in week 2, different groups of youth completed participant logs in which youth answered questions about ASYP (84 youth in week 2, 46 in week 3, 71 in week 4, 29 in week 5, and 48 in week 6). Staff interviews were conducted each week during site visits. From data gathered, characteristics were identified that might influence youth outcomes: (a) youth characteristics of gender, age, computer use at home, postsecondary goals, and participation in extracurricular activities or year-round Ascend programs; and (b) program leaders’ college major and whether they had previously been Ascend team or group leaders. |
| Data Collection Methods | Document Review: In participant logs, youth were asked to answer 2–3 open-ended questions about various aspects of ASYP. Questions were different each week and ranged from “What is your favorite part of the summer program so far?” to “Do you like working on the computers here? Why or why not?” to “Think about your favorite team leader or group leader. List 3 reasons why that person is your favorite.” Interviews: Staff interviews included questions about how the program was functioning, the types of activities offered, how technology was being used, the kinds of feedback given to participants, and the nature of the relationships between program staff and participants. Observation: Observations included a mix of morning and afternoon teams and computer- and noncomputer-related activities, with the goal of understanding program implementation (i.e., how the program looked, what activities were offered, use of technology, etc.) and the relational aspect of the program (i.e., how leaders provided feedback and established rapport, the nature of relationships among staff and participants). Surveys/Questionnaires: The pretest/posttest youth survey included 32 items about youth background characteristics and outcomes data. Outcome items were grouped into scales of postsecondary option awareness (e.g., “I know what it takes to succeed on a job”); applied computer use (e.g., “I like working on a computer”); comfort level with computers (e.g., “Using a computer is easy”); and self-efficacy (e.g., “I am happy with who I am”). The exit survey contained 7 questions about outcomes from the larger pretest/posttest survey in addition to 4 open-ended questions about program implementation. |
| Data Collection Timeframe | Data were collected in July and August 2005. |
Findings:
Formative/Process Findings
| Activity Implementation | Staff were able to manage hundreds of youth daily through a database that tracked individual youth. Participants quickly learned these systems, and programming began without delays or major complications each day. By dividing youth into morning and afternoon groups, staff created small heterogeneous teams that allowed staff to (a) create a schedule that integrated program components within a limited summer schedule, (b) establish mentoring relationships, and (c) set expectations early among teams and groups. The configuration of teams and groups also allowed staff to rotate youth through a range of activities (i.e., computer use, noncomputer activities, and field trips) and to maximize the use of a limited number of computers and facilities. Due to the limited number of rooms available in the host school, teams were clustered into larger groups, requiring leaders to monitor many youth at any given time. ASYP staff felt a tension in trying to accomplish the ambitious goal of preparing youth for a successful future via a 6-week, 4-hour-per-day program. Faced with a choice between activity depth and breadth, a range of activities were developed that allowed for exposure to a variety of experiences, while allowing for mastery in some of these experiences. Log data, especially in the context of identifying concerns about ASYP, indicated that youth viewed ASYP as a summer job and considered themselves computer aides. Some youth expressed confusion at how field trips related to being a computer aide, and some expected more computer time as a computer aide. Some youth mentioned that the program felt too much like summer school, and at least one noted that ASYP should feel more like work than summer school. As a result, at times, youth did not appear to understand ASYP’s broader goals, which may have been compounded by the fact that staff varied in how well they integrated program components; some staff linked activities together (e.g., referring to college trips while youth did Web research on colleges), while others did not seem to do so. A few youth reported a desire for greater program structure and organization. Staff altered schedules and activities as necessary when groups needed more time to complete activities or when weather precluded a field trip. Group leaders were required to coordinate and communicate frequently with their team leaders to ensure that different work activities were timed so that each group adhered to the published weekly schedule. Technology was the program’s main theme; each activity had a technological component. For example, youth used digital cameras to take pictures on field trips and Photoshop to create collages when they returned. In addition, youth used the Internet to research colleges and careers and Microsoft Office tools to synthesize and present their findings. For many participants, ASYP was their first paying job, and according to the Ascend director, most of these youth entered the program unfamiliar with the world beyond their neighborhoods. To prepare youth for future success, they were required to show up on time, conduct themselves in a professional manner, account for their time, and learn to solve problems and work with others. To get paid, youth had to learn to use Electronic Bank Transfer accounts, and some activities required youth to learn new skills, such as purchasing Metro cards and navigating the Metro system. From the director’s perspective, introducing youth to different college settings was a way to provide them with the knowledge and skills deemed necessary for future success, with the intention of allowing youth to participate in conversations about college so that they would not be left behind. In week 2 logs, 41 youth identified field trips as their favorite part of ASYP. The morning group preferred college trips while the afternoon group preferred museums. Some youth liked field trips because they offered opportunities to get out of the building. Others said that they planned to go to college and consequently appreciated being able to experience college life. In week 3 logs, 19 youth said that they liked working on computers at ASYP, and 8 of these mentioned learning as a reason for liking the computers and identified jobs and college as particular topics of interest. Youth frequently expressed a desire for faster/more computers and a more diverse array of activities. Although the schedule was arranged so that no more than two groups used computers at a time, participants felt that there were not enough computers for the number of youth (approximately 70 to 80 computers were available to accommodate up to 250 youth per session), and several said that the high ratio of youth to computers resulted in idle time while waiting for a turn. This waiting caused delays in completing assignments and activities, which appeared to affect the program schedule. In addition, among 18 youth who reported in week 3 logs that they did not like using the computers, 12 cited the age of the computers. They noted that the machines were slow and, in some cases, broken. In observations, youth appeared to be engaged in computer activities (even when waiting for a turn), excited about field trips, and eager to demonstrate their technology skills. |
| Program Context/Infrastructure | Youth and leaders frequently mentioned the lack of air conditioning as a major problem. Except for the classrooms, the building was without air conditioning during the program’s first few weeks. Several youth said that they enjoyed field trips because they provided an opportunity to leave the heat of the school building. |
| Recruitment/Participation | Of youth participants, 60% were female, 96% were African American, and more than 90% were 14 or 15 years old. Many of these youth lived in high-poverty areas. From previous experience, staff assumed that many of these youth had low school achievement levels. In terms of future education and career plans, on the pretest survey, 95% of participants reported that they planned to finish high school, 1% reported that they did not plan to finish high school, and 11% reported that they planned go to work straight out of high school (these items were not mutually exclusive). In addition, 39% each reported that they planned to complete a job training program, graduate from a 2-year community college, or graduate from a 4-year college. On the pretest survey, 58% of youth reported participating in sports, 15% in band/chorus, 11% in arts, and 9% each in clubs or drama. Nearly a quarter (24%) reported that they did not participate in any of the activities listed. On the pretest survey, the majority of youth reported using a computer at home (70%), while some also had used computers at school (22%), at a friend’s house (17%), at the library (10%), and at other locations (5%). Only 4% reported never using computers. |
| Staffing/Training | The leaders’ style appeared to influence youth’s perceptions. For example, youth reported in logs that they appreciated and learned the most from leaders who were fun to be with, funny, or nice, and that having a leader with a good personality made their work easier. In addition, youth seemed to appreciate leaders who helped them with job-related skills. For instance, when asked how their favorite leader helped them, several youth said that leaders reminded them to check in and kept them out of trouble. In week 5, youth said that their favorite leaders took them somewhere or showed them how to get there by themselves. |
Summative/Outcome Findings
| Academic | On surveys, youth had no significant gains in awareness of postsecondary options overall, although they showed significant gains on the item “I know what my strengths are” (p < .001); younger participants had significantly greater gains on this item than older participants (p < .05). Although 35 youth in week 6 logs mentioned going to college in their goals for the future, none mentioned a specific college that they wanted to attend. In week 4 logs, 12 youth identified information about college as the most important skill learned in the program. |
| Workforce Development | In discussing their goals for the future in week 6 logs, 23 youth noted specific careers or majors that they wanted to pursue; others identified vague goals, such as “being successful” or “graduating from college and getting a job”; and a few mentioned computer or technology careers. When asked in week 4 logs about the most important skills learned at ASYP, 47% of youth identified digital photography because that knowledge would be useful in the future, while smaller percentages cited creating resumes (16%), computer skills (15%), and work skills (7%). In week 6, the most common responses as to what youth learned were digital photography and Web design; they also appreciated using technology more generally. |
| Youth Development | On surveys, youth showed no significant gains on computer comfort levels overall. However, compared to other groups, youth in groups with a 2nd-year Ascend leader had significantly lower comfort levels (p < .001) than youth without such a leader and youth in groups with a leader who was an education major had significantly higher comfort levels (p < .01) than youth with a leader who did not. In addition, youth showed significant gains in how easy they found it to use computers (p < .01). These gains were significantly larger for youth who were involved in extracurricular clubs than those who were not ((p < .05). Prior computer proficiency did not predict these attitudes. In addition, significantly fewer youth felt nervous about creating and designing Web pages at the end of the program than at the beginning (p < . 001). On surveys, youth showed no significant gains in overall attitudes about applied computer use, although significant gains were found for their reports of feeling able to organize their work with computers (p < . 05). Those who did not plan to pursue postsecondary education made significantly greater gains on this item than those who did plan on it (p < .05). Youth in groups with 2nd-year Ascend leaders were less likely to find computers useful for organizing work than were youth in other groups. Gains in attitudes about applied computer use were significantly greater for younger participants than older ones (p < .01) and for youth who were active in clubs than those who were not involved (p < . 05). On surveys, youth showed significant overall gains in self-efficacy (p < .05). Significantly greater gains were found for younger participants than older ones (p < .05), youth who had previously participated than those who had not (p < .05), and youth in groups with leaders who were education majors than youth in groups without such leaders (p < .01). When asked what they learned in ASYP, several youth mentioned that they learned to “get around on the Metro” or “use an EBT account.” |