You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼


Program Description

Overview In March 2000 Public/Private Ventures (P/PV) developed a literacy model, known as Youth Education for Tomorrow (YET) Centers, to complement in-school reading instruction. The centers operate in churches and other faith-based institutions throughout Philadelphia, and receive management assistance from P/PV. P/PV launched the centers with two main goals in mind: (1) to provide an effective literacy service to children and young adults and (2) to find out whether a diverse group of independent faith-based institutions could collectively deliver an effective service. The YET program encourages children to participate in class and to understand the connections between reading, speaking, writing, and the events of the world around them. This secular program works with faith-based organizations to achieve outcome-based measurable results for youth.
Start Date March 2000
Scope local
Type after school, summer/vacation
Location urban
Setting religious institution
Participants preschool through high school
Number of Sites/Grantees 23 sites (2000–2001)
Number Served almost 1,000 children total in the first 18 months (2000–2001)
Components The YET program is based on a literacy model developed by P/PV. All components of the YET program are grounded in research: the strong relationship between oral language/vocabulary activities and reading (Hart & Risley, 1995), the value of student writing (National Research Center on English Learning and Achievement, 1998), emphasis on daily reading (Terrance, 1998), the importance of reading to children (National Research Council, 1998), teacher assessment (U.S. Department of Education, 1998), the use of trained volunteers (Wasik, 1998), and the need for ongoing professional development (Morris, Shaw & Perney, 1990). The model functions in the following ways:
  • Classes are held after school for 90 minutes, four days a week throughout the school year (or the summer for summer programs).
  • Students and a teacher engage in four basic activities in each class: (1) an oral language/vocabulary activity (talking about words and ideas in the context of current events, holidays, issues in the news, or other things of interest to students), (2) the teacher reading aloud to students, (3) student reading, and (4) student writing.
  • The target population is children three years or less behind their grade level; the goal is to bring them up to grade level.
  • Teachers are professionally qualified, approved by P/PV, and paid out of the sites' YET Center budgets.
  • Volunteers (recruited by sites and trained by P/PV) assist teachers, listen to children read, read aloud to them, and help in other program areas.
  • Classroom reading materials are supplemented by take-home materials.
  • Children are tested for reading levels at the beginning of the year, midway through the year, and the end of the year, using a standardized instrument called the Informal Reading Inventory (IRI).
  • YET Centers are meant to be as “faith-neutral” as possible. While not asked to eliminate all evidence of their faith, sites are asked, for example, not to pray or worship in class, and to rely on secular reading materials.
P/PV's model also requires a number of “essentials,” including a clearly posted daily schedule, displays of student work, and a “word wall” of high-frequency words. YET sites are asked to make sure students have library cards and access to books in class. Finally, YET sites are required to assist P/PV's efforts to gather data on children and their families, through interviews with students' parents and teachers.

References
Hart, B., & Risley, T. R. (1995). Meaningful differences in everyday experiences of young American children. Baltimore: Paul H. Brookes.

Morris, D., Shaw, B., & Perney, J. (1990). Helping low readers in grades 2 and 3: An after-school volunteer tutoring program. The Elementary School Journal, 91(2).

National Research Center on English Learning and Achievement. (1998, Spring). Effective early literacy instruction: Complex and dynamic. English Update, 1(8).

National Research Council. (1998). Preventing reading difficulties in young children. Washington, DC: National Academy Press.

Terrance, P. (1998). Patterns of reading practice. Madison, WI: Institute for Academic Excellence.

US Department of Education. (1998). Longitudinal evaluation of school change and performance: Some preliminary findings from first two years. Draft. Washington, DC: Author.

Wasik, B. A. (1998). Using volunteers as reading tutors: Guidelines for successful practices. The Reading Teacher, 51(7).
Funding Level $4.5 million over two years (1999)
Funding Sources The Pew Charitable Trusts


Evaluation

Overview The evaluation examined whether faith-based sites were able to successfully implement the YET program, which included recruiting and retaining children for the YET program, putting the YET model into operation, and obtaining and recording reading improvements among students.
Evaluator Bill Hangley, Jr. and Wendy S. McClanahan, Public/Private Ventures
Evaluations Profiled Mustering the Armies of Compassion in Philadelphia (an implementation study)
Evaluations Planned none
Report Availability Hangley, B., Jr., & McClanahan, W. S. (2002). Mustering the armies of compassion in Philadelphia. Philadelphia, PA: Public/Private Ventures. Available at: www.ppv.org/ppv/publications/publications.asp?search_id=19.


Contacts

Evaluation Wendy McClanahan
Public/Private Ventures
2000 Market Street, Suite 600
Philadelphia, PA 19103
Tel: 215-557-4400
Fax: 215-557-4469
Email: wmcclanahan@ppv.org
Program Leigh Bernstein
Director of Literacy Initiatives
Public/Private Ventures
2000 Market Street, Suite 600
Philadelphia, PA 19103
Tel: 215-557-4400
Fax: 215-557-4469
Email: lbernstein@ppv.org
Profile Updated May 13, 2003

Evaluation: Mustering the Armies of Compassion in Philadelphia



Evaluation Description

Evaluation Purpose To measure the effectiveness of YET Centers during the first year of implementation using four basic categories of data: recruitment, retention, requirements, and results.
Evaluation Design Quasi-Experimental and Non-Experimental: The sample included 23 centers that were implemented during the 2000–2001 school year. Of these, 21 centers ran continuously throughout the school year (one program dropped out of the study after the first half of the year and another began in April and ran to the end of the school year). YET staff obtained parent and youth permission to participate in P/PV's study. YET sites had to address two groups of requirements: (1) implementation of the model and (2) recording and reporting data. In both cases, P/PV staff were actively engaged at all times.
Data Collection Methods Interviews/Focus Groups: Dozens of interviews with site staff and organizers explored the sites' response to the YET Center experience. Evaluators talked to staff about program implementation, who is served, what was harder than expected, etc.

Observation: P/PV staff rated the quality of program implementation based on site visits. Each site was assigned a score from 1 (poor) to 5 (outstanding) in each of four categories (oral language, writing, reading, and overall consistency) with a maximum score of 20.

Secondary Source/Data Review: YET Centers are required to complete applicant intake forms for each participating youth. These forms provide data about program participants and contain questions about the youth's schooling, home environment, reading frequency, attitude toward and interest in reading, as well as background characteristics, parent or guardian demographics, and attitudes toward reading. Sites provided enrollment information for only 65% of school-year participants. Throughout the program, centers are asked to record and report daily program attendance. During the course of the study, Centers submitted attendance forms more than 90% of the time. Fourteen sites provided complete attendance information.

Tests/Assessments: Teacher assessment forms were received from 421 of the 786 YET Center participants. Teachers were asked for detailed information on the child's reading progress, behavior, and grades in school. Evaluators discovered that sites that were not located in schools had a really hard time connecting with the teachers long enough to complete the form.

In addition, YET participants were administered the Informal Reading Inventory (IRI), a standardized reading test composed of a series of graded word lists and passages used to determine decoding and comprehension skills. Five types of comprehension questions follow each reading passage: topic, fact, inference, evaluation, and vocabulary. The IRI allows YET staff to establish students' baseline reading levels, start them working at the appropriate level, and monitor their development. IRIs also allow P/PV to monitor its students' progress. Sites gave initial tests to 684 of all 786 recruits, but only 392 children were tested twice (at least 90 days apart)—almost exactly half of the recruits. Among the 19 centers for which complete data were available, six gave two tests to more than two-thirds of their recruits, nine gave two tests to between one-third and two-thirds, and four gave two tests to one-third or less.
Data Collection Timeframe Data were collected during the summer of 2000 and the 2000–2001 school year.


Findings:
Formative/Process Findings

Activity Implementation All sites achieved baseline implementation, although some were more effective than others. Most sites fulfilled the basic requirements: scheduling the time, setting aside space, finding a qualified teacher, and sending staff to the required training sessions.

Sites' ability to implement the program consistently and in detail varied, with sites displaying different strengths and weaknesses in different areas. The more specific P/PV's demands, the more difficult consistent implementation was.

Sites adhered to restrictions of religious expression during YET class time, and usually said they understood P/PV's point of view, although they did not always like it.

The success of the program's implementation seemed to depend on the site's basic ability to organize itself in pursuit of a specific goal, its ability to draw a consistent group of children, and its willingness to accept the YET model's goals as its own.

All but one of the six best implementation performers were schools—four Catholic schools and a Protestant charter school.

Implementation of the program's details (e.g., the “word wall,” posted schedules, proper display of materials) was rather uneven. Out of 20 points, six sites scored 15 points or more, nine scored between 9 and 14, and four scored 8 or lower (out of 19 reviewed).

Neither faith nor size nor location proved the determining factor for whether a site would achieve the attendance and implementation numbers that led to YET Center success. That is, big sites did not perform categorically better than small sites; school-based sites did not perform categorically better than community centers; overtly religious sites did not perform categorically better than nondenominational or less religious sites. However, the data do suggest some advantages. For example, schools seem to have better attendance and thus tended to have better overall results, but not every school performed as well on these counts, and not every one of the best performers was a school.

Almost all site staff spoke of their desire to blend YET's methods with their own. In many cases, staff acknowledged that their usual methods were not suited to getting measurable results in a classroom setting, but no one wanted to give up the old for the new.

Throughout YET sites, staff supported the notion that a proven, structured curriculum could help maximize progress and that testing children is a good way to measure that progress. This was seen as a way to help both children and the institutions themselves.

The ways faith plays out in classrooms and manifests in the working day of YET Centers varies greatly. Some YET classrooms provide no outward indications of religion. Others feature prominent crucifixes, inspirational Biblical messages, and religious pamphlets. Some teachers discuss “the Word” during reading sessions. Some staff pray with students before or after YET class time. Others consciously work to impress YET students with core values and behavioral lessons without explicit reference to religion. As far as P/PV knows, sites have not used YET Center time to actively proselytize.
Parent/Community Involvement Few sites had consistent and sizable volunteer contributions. Several sites had steady and dedicated volunteers; these sites tended to develop a relationship with another institution, like a school or partner church, that provided a consistent, reliable, and committed corps.

The majority of sites reported that committed, consistent volunteers were hard to find, organize, and retain. Most commonly, sites report that adults in their communities are too busy. In addition, volunteers require supervision and organization that can tax site staff. These problems are exacerbated by the relatively high demands the YET model places on volunteers.

Many site staff reported that volunteers are relatively easy to find for one-shot events, like running a bake sale or sorting donated coats. Finding qualified people to work with children once or twice a week for a year is more difficult.
Program Context/Infrastructure The 21 YET Center hosts that continued throughout the year included five Catholic sites, four of which are Catholic schools, and the other is a community-based organization run by a Catholic nun. Seven sites are Protestant. Of these, one is a school, another is a community-service organization, and several are churches. Two sites are run out of public schools in partnership with faith-based organizations, one of which is Jewish, another of which is Episcopalian. One is in a charter school established by a Hispanic clergy association.

While almost all the sites ran the YET Center as an after school program, four school-based sites integrated some or all of their YET classes into their school days.

Almost all site staff interviewed requested more flexibility on elements of the YET model, such as broader eligibility rules for children, more grammar or comprehension in the pedagogy, no testing, a more traditional classroom setting, and the ability to express their religion. Sites accepted structure if they felt it balanced the program's needs with theirs, but resisted when they felt forced to place the model's needs over the immediate needs of their staff or students. When asked how to resolve this problem, sites usually suggested that they be given outcomes to measure, targets to hit, and a basic structure to work within, and then be allowed to tailor the program to address these needs and their own needs.

The YET model challenged sites to juggle schedules, rearrange spaces, find new teachers, or reorganize their after school setups.

The sites that host YET Centers share some basic elements, such as physical space to house the YET model, basic fiscal capacity to operate the model, and a mission of service to the youth/community. Site staff unanimously consider the children they serve to be “at-risk” and want to reduce that risk through literacy improvement and other means. The other major unifying factor of the sites is that they are all located in the hearts of some of Philadelphia's poorest communities.

The sites have different levels of experience with fiscal matters, administration, hiring and firing staff, developing specific job profiles, and evaluating performance based on outcomes.

Each one of the YET sites has its own history and methods for how to help children. Some sites, like Catholic schools, have a long-standing pedagogical tradition. Others have developed their own goals and methodologies. Some sites have well-defined curricula, while others seek mainly to provide safe havens for children.

YET sites range from those that overtly promote the religious beliefs of a particular denomination to those that consider themselves spiritually motivated but not sectarian.

The sites' religious or spiritual missions are often implicitly expressed in the daily life of the sites, including during YET classes. Protestant sites consider their religion to be central to everything they do and expressions of their beliefs are everywhere in the site, posted on the walls or spoken by the staff. Catholic sites tend to expect their students to understand the Catholic point of view and to abide by an established code of behavior, the basis of which is ultimately spiritual. Still other sites work with a religious vision that allows them to focus on the practical. One site seeks to impart a nondenominational kind of spirituality.

The YET program's restrictions on program eligibility clashed with sites' desire to build and maintain relationships with the children they serve, regardless of test scores or age.

YET staff seemed confident that they could help their children, but not that they could change their communities; most sites think their children's main problem is poverty—not just personal poverty, but of the entire community. Sites are anxious to address illiteracy and other personal problems, but in most cases, their primary goal is to provide an alternative to the disorder and danger in their children's lives.
Recruitment/Participation In the summer and school year of 2000–2001, the YET program was able to enroll a total of about 855 youth (786 during the school year, 190 in the summer of 2000, of which 69 continued on into the school year) through 7 summer and 23 school-year programs.

Most sites planned to serve between 15 and 30 children, although a few sites (all schools) aimed for 40 or more, and one aimed to serve 75. All but five centers met their initial enrollment goals, and many exceeded their goals by wide margins. School-based sites enrolled the most youth (38 on average). Community centers and church-based sites set recruited more than 30 children each on average.

Intake data on YET students show that 80% were African American and 7% were Hispanic; the rest were Caucasian or “other.”

Most YET students were living at or below poverty level—75% of those in school received free or reduced-price school lunch.

YET Centers served more females (57%) than males to (43%).

Sixty percent of all YET students were in grades one to five, 10% in grades six to eight, and 21% in high school. Additionally, one program served out-of-school students and another served preschool youth.

Almost all sites planned to teach children who were selected, at least in part, from the larger group that already patronized the institutions (whether as congregants, students, or participants in other programs). About half of the sites drew recruits from students or congregants that were already closely involved with the site, and would have remained so even if the YET Center had not been available (e.g., a school offers the program as an after school option for students).

In most other cases, the YET Center allowed sites that already served some children to increase the “slots” available (e.g., a church increases its after school roster from 30 to 50). In two cases, the program allowed sites to provide services to children for the first time. A third site, based in a school, used the YET program to serve some students from the school along with others from outside the school. Most sites already had programming for children and drew YET students from the same sources that filled other programs.

Sites faced the problem of what to do with children that do not fit in the program. Not all have the option of placing them in other programs in their church, and some sites found it counterintuitive to sort children. There were almost no instances of a site turning a child away. Many sites believe that their children's need for healthy relationships with responsible adults was a priority, stating that, given the choice between addressing the model's needs and what they saw as their children's needs, they would choose the latter.

Sites serving young children found plenty of recruits that fit the eligibility guidelines (children three years or less behind grade level). However, since older children can fall much farther behind, sites serving them found that the three-year window was too small.

Despite any reading deficiencies, most YET students came into the program saying that they liked reading. While 68% said that reading was at least “a little” hard for them, 37% liked reading “a lot,” and another 32% liked it “some.”

Most YET students came from moderately rich reading environments: 64% had a library card when they entered the program, 25% reported being read to a couple of times a week at home, and 34% said that they were read to every day. Only 16% said that they never read for fun, and only 7% did not like reading at all.

Seventy-one percent of the participants' parents were high school graduates, just over half of whom had some post-high-school training or education.

Participants arrived in the programs with average grades of B- to C, and only 2% were in special education.

Some behavioral problems were evident among participants: during the four weeks before starting YET, one-eighth of YET students had been sent to the school principal for disciplinary action, one-tenth had been suspended, almost half had come late to school, and 9% were identified by their teachers as having moderate to severe behavioral problems. Nonetheless, on a scale of 0 (poor behavior) to 4 (good behavior), 78% of YET students were rated by their regular schoolteachers as a 3 or 4.

A number of factors seemed to affect attendance, some of which are internal, like the consistency of the site's scheduling and after school routine, the efficiency of its administration, and its experience with other programming. Others are beyond sites' control, like their physical proximity to the YET students' schools or the safety of their neighborhoods.

Less than half of YET's recruits (47%) attended enough classes (60) for P/PV to expect the program to have a positive impact on their reading ability. Out of 786 participants from school year 2000–2001, 7% never came to class, and 22% came no more than 20 times. The biggest group came between 21 and 100 times (42%), and another 30% came 101 times or more. Broken down, the percentages for attendance look like this: 0 classes attended, 7%; 1 to 20, 22%; 21 to 40, 15%; 41 to 60, 10%; 61 to 80, 9%; 81 to 100, 8%; 101 to 120, 11%; 121 to 140, 13%; 141+, 6%.

Retention numbers differed from one site to another. Six sites had over three-quarters of their students attend more than 60 classes. At another six sites, between 50% and 75% of students came 60 times or more. Four had less than half of their students come 60 times, and another four had less than a quarter come that often.

School-based sites tended to be the most successful at retaining students, which evaluators conjecture is because they ran their YET programs during or immediately following the school day. Of the six sites that got more than 75% of YET students to come to 60 classes, four were Catholic schools, one was a Protestant school, and one was a church-based community center.
Satisfaction Site staff liked many elements of the program; the focus on literacy made sense to them, and the YET Centers enabled many to provide a new level of service in keeping with their basic institutional goals. Although they found testing and measuring for outcomes difficult, they liked having clear evidence of their effectiveness. They recognized that the rigor and structure that YET offered could lead to positive results.

Sites indicated that the program helped the participants. That many YET students' reading ability improved noticeably, sites reported, made the process worthwhile. Sites generally agreed that YET was an improvement on what they previously offered, that it expanded their understanding of literacy and programming in general, and that the YET experience left them better prepared to solicit funding and design programs in the future.

Almost all sites sought continued YET funding, suggesting that they see the program as a gain overall.
Staffing/Training Site staff differed greatly, depending on the institution, with respect to their salary, skills, level of responsibility, experience, and the scope and focus of their work.

P/PV struggled to find training and communication methods that suited every site; they learned that no one-size-fits-all model would work with the entire group.

The most critical factor in the success of the implementation of the classroom model was the teacher. As long as the teacher understood and embraced the YET model, the four-component classroom strategy and testing usually fell into place.

Many sites found their staff stressed by the work of implementing the model, recruiting volunteers, retaining children, and reporting data.
Systemic Infrastructure One factor was consistent for all the sites-implementation of the YET model required a highly directive and interactive relationship between P/PV and site staff, since the goal was to get all sites to deliver the same program.

Many implementation problems arose when P/PV adjusted their expectations without clearly communicating what they were doing. Sites routinely asked for a clear picture of what they had to do. P/PV learned over time how to improve communications with sites and how to set expectations and obligations.

YET collaboration put P/PV and many site staff in unfamiliar territory; both sides had to recognize and work through diplomatic conflicts and confusion as the project progressed.

Each site had to be dealt with slightly differently. Some were organized hierarchically, and P/PV could count on supervisors communicating down the line to other site staff. Sometimes, this was not the case. In addition, some sites were used to dealing with outsiders' guidelines, but others were sensitive to incursions on their turf.

P/PV learned that sites are easier to work with when met on their own turf or when group meetings were arranged in places where things like parking or traffic were not problems.

Site staff had mixed feelings about the movement to fund them. More than one person worried that the new attention on faith-based initiatives is a ploy to fob off urban problems on churches. Sites were quick to acknowledge their limitations, whether internal (staff and resource shortages) or external (students living with unstable families or in dangerous neighborhoods). They generally felt that they were only a piece of the puzzle in their community, and many noted that faith-based institutions cannot be expected to replace public institutions. However, everyone agreed that any support is better than none. They believe that they do good despite their limitations, and they believe that they can do more good with more resources.

Site staff were often confused by P/PV's attempt to fund faith-based sites without funding their faith. They did not understand why P/PV would want to partner with them without taking advantage of their religion. More than one pointed out that if P/PV wanted to treat sites simply as “buildings” that P/PV would miss out on what sites considered a vital strength. In most sites, service and faith tended to be inseparable.

The sites' have varied relationships with funders and external systems. Some get public funding or foundation grants for specific programs. Some collaborate within their denomination or with other partners on projects large and small. Some rely on their immediate communities or on congregational support.


Summative/Outcome Findings

Academic Students who consistently attended YET usually improved as readers. Overall, participants improved an average of 1.4 grade levels between their first and second IRI tests (at least 90 days apart); 44% of those who were tested twice averaged over a grade level of improvement. Students who attended more averaged greater gains—those whose two tests were more than 180 days apart averaged approximately 1.5 grade levels. Controlling for length of participation, about two levels of reading growth can be expected for students who participate for an entire academic year.

The elementary school children (grades 1 to 6) served by 14 programs who were tested twice using the IRI at least 90 days apart, on average, were in grade 2.7. They arrived reading, on average, 1.7 levels below the proper level. Two sites started with students averaging less than a year behind grade level. Four others started with children averaging between one and two years behind, and eight started with children averaging more than two years behind grade level. When tested a second time, they read at an average of 0.5 grade levels below the proper level, for an improvement of 1.2 grade levels; the average deficiency was reduced by 70%. By site, gains ranged from less than one grade level (four sites) to almost two grade levels (two sites). The rest averaged between one and two grade levels.

In the five sites that served middle schoolers, the students (grades 6 to 12) who were tested twice using the IRI at least 90 days apart, on average, were in ninth grade and arrived reading at about a fifth-grade level (an average of 3.8 grade levels below the proper level). By site, students averaged 2.6 to 4.9 years behind grade level. When tested a second time, students tested at an average sixth/seventh grade level (2.2 grade levels below the proper level), an improvement of 1.6 grade levels. Students reduced their deficiency an average of 58%. By site, improvements ranged from less than one (one site) to three grade levels (one site).

The more a student attended the YET program, the more the student improved (b=.005, p=.0435). Centers with average attendance of over 100 days (not including participants that never attended) had participants with larger reading gains (1.9 grade levels on average), than those with average attendance of less than 100 days (1.1 grade levels on average). The effect of attendance remained important even after such individual participant characteristics as race, gender, age and low-income status were considered.

The more effectively the sites implemented the YET program, the better the students did. The eight centers that scored 14 or more on P/PV's implementation rating had reading gains of 1.6 grades on average, whereas the 10 centers that scored 13 or less averaged 1.2 grades of growth.
Systemic Sites gained in various ways from the YET Center experience, learning about service delivery, outcome measurements, collaborations, and work with outside funders.
Youth Development Within a few months of the YET program's launch, virtually every site could point to specific students for whom the focus on literacy resulted not only in improved reading, but also in improved social skills, self-esteem, classroom behavior, focus, and confidence.

Site staff almost unanimously agreed that work toward the specific outcome of literacy helped them advance the general outcome they consider important—an overall improvement in the life, health, and prospects of the children they serve.

 

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project