You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼


Program Description

Overview AfterSchool KidzLit (KidzLit) is a research-based academic enrichment program designed to develop youth’s reading motivation, capacity to read, thinking skills, and prosocial development in out-of-school time settings.
Start Date 2000
Scope national
Type after school
Location urban, suburban, rural
Setting public school, private school, community-based organization, religious institution, private facility, recreation center
Participants kindergarten through middle school students
Number of Sites/Grantees approximately 2,500 sites nationally
Number Served approximately 150,000 children and youth annually
Components KidzLit offers youth opportunities to hear books read aloud by an adult, to read with a peer, and to read on their own. Participants then discuss the stories with one another. Using the books as a starting place, they talk about issues that matter to them, such as why characters behave the way they do, the choices they face, the advice they might need, and how these stories relate to their own lives. To help make these ideas stick, participants explore these issues more deeply through art, drama, discussion, sharing, and journal writing. These shared activities help youth form stronger, closer relationships with peers and site staff. KidzLit materials consist of 120 books with individual guides (50 titles for youth in grades K–3, 50 titles for youth in grades 3–5, and 20 titles for youth in grades 6–8), personal journals for each participant in the program, and a “Quick Tips” guide that offers on-site leaders and facilitators advice about how to lead lively discussions and organize the suggested activities.
Funding Level The cost of the full AfterSchool KidzLit program for grades K–8 is $2,020 plus shipping and handling. The program can also be ordered in subsets for selected grade levels.
Funding Sources Not applicable
Other KidzLit was created by the Developmental Studies Center (DSC) in collaboration with the Boys & Girls Clubs of America, the YMCA of the USA, and youth development organizations in several major cities nationally. DSC provides training and workshops for program staff on how to implement KidzLit.


Evaluation

Overview DSC’s researchers conducted an initial evaluation during the 2001–2002 school year at eight LA’s BEST after school sites in Los Angeles, California. This evaluation is profiled here.
Evaluator Developmental Studies Center
Evaluations Profiled AfterSchool KidzLit Outcome Study

Evaluations Planned None
Report Availability Developmental Studies Center. (2003). AfterSchool KidzLit outcome study. Oakland, CA: Author.


Contacts

Evaluation Jill Casey
Research Associate
Developmental Studies Center
2000 Embarcadero, Suite 305
Oakland, CA 94606-5300
Tel: 510-533-0213
Fax: 510-464-3670
Email: Jill_Casey@devstu.org
Program Megan Weber
Manager of After-School Programs
Developmental Studies Center
2000 Embarcadero, Suite 305
Oakland, CA 94606-5300
Tel: 510-533-0213
Fax: 510-842-0348
Email: info@devstu.org
Profile Updated June 20, 2006

Evaluation: AfterSchool KidzLit Outcome Study



Evaluation Description

Evaluation Purpose To examine youth program exposure and outcomes; staff training, assignments, background, planning, and support from consultants and coworkers; and program implementation quality and site conditions.
Evaluation Design Quasi-Experimental and Non-Experimental: The evaluation examined eight KidzLit sites operated by LA’s BEST (Los Angeles’ Better Educated Students for Tomorrow), an after school program in Los Angeles, California. The study assessed changes over 8 months among second and fourth graders. Reading-related attitudes and behaviors, vocabulary development, and social attitudes and behaviors were measured.

Sites were selected based on the following criteria: (a) staff who received the basic KidzLit training from DSC staff developers or DSC-trained staff developers; (b) staff and site coordinators who were enthusiastic about KidzLit; (c) September to June schedules, maximizing the number of youth exposed to KidzLit for the full period between pretest and posttest; and (d) the ability to implement KidzLit twice per week in 45–60 minute sessions for 15–20 youth. In addition, DSC provided further KidzLit training for staff, site coordinators, and activities consultants at the selected sites and stipends of $500 per site for the purchase of additional youth reading materials. DSC asked that the selected sites schedule KidzLit in sessions of 15–20 youth, offered twice each week, for 45–60 minutes from October to May.

The sample consisted of 83% of second and fourth grade KidzLit participants at the eight sites who completed pretest and posttest surveys (n = 393, ranging from 27 to 79 youth per site). These youth were representative of all second and fourth grade participants at these sites in ethnic distribution. The sample had a slightly higher proportion of girls than did site enrollment overall (55% vs. 51%). Approximately 40% of sample youth were enrolled at LA’s BEST when the site had its initial KidzLit training and may have had 4–12 months of KidzLit exposure before the study, although youth with previous KidzLit exposure did not demonstrate different outcome patterns.

Program implementation data were collected through observations and interviews done in March and May across the eight sites. Interviews were conducted with 15 staff members implementing KidzLit and all 8 site coordinators. Surveys were completed by 13 of the 14 remaining KidzLit staff at the end of the school year and 7 of the 8 activities consultants (program experts who provide consulting to site coordinators and staff on implementation of programs offered by LA’s BEST) at posttest. Overall implementation quality ratings were created by averaging the KidzLit implementation quality ratings across staff members’ observations/interviews.
Data Collection Methods Interviews/Focus Groups: Staff member interview responses were rated for program implementation quality, using a 4-point scale (where 4 = particularly effective with kids, masterful implementation; 3 = effective with kids and 75%–80% implemented; 2 = marginally effective with kids and/or only partially implemented; 1 = not happening, very minimally happening, or missing even though it should have been there). Ratings were made based on answers to questions concerning three components of the KidzLit sessions: introduction, vocabulary, and discussion. Staff were shown a list of strategies for each of the three components and then asked to talk about how they typically used the strategy and their reason for using it. For example, with discussion, staff were shown a list of the following strategies and asked to choose up to three that they used most often: (a) pose a general or specific question as a starting place for a discussion; (b) go back and reread an aspect of the story to focus on a theme or issue; (c) ask the youth to talk in pairs (“partner chat”) before or during group discussion; (d) have kids summarize elements of the story or ask a number of quick recall questions about the story; (e) focus on students’ feelings or connection to the story and on how the story might connect to their own lives and experiences; (f) use questioning strategies to promote deeper thinking (“Why do you think that…” “Tell me more about….”); (g) ask students to write down their ideas, feelings, or related experiences before or during group discussion; and (h) pass around a soft toy or some other item to help kids know who talks and who listens.

For each of the three most-often-used strategies, each staff member was asked to describe how they used the strategy (typically or in a specific instance), why they used it, and how students typically responded to the strategy. Analyses showed that interview ratings were inflated compared to observation ratings (i.e. staff members reported using the strategies more often than observers reported seeing them used). As a result, interview ratings were reduced by 25% (1 point).

Site coordinator interviews focused on the site’s KidzLit schedule and attendance; consistency of implementation; literacy resources and programming; communication with parents, school, and teachers; KidzLit staff; integration of KidzLit into the site’s programming; and perceived youth and staff benefits.

Observation: Session observations were rated for KidzLit implementation quality, using a 4-point scale (where 4 = particularly effective with kids, masterful implementation; 3 = effective with kids and 75%–80% implemented; 2 = marginally effective with kids and/or only partially implemented; 1 = not happening, very minimally happening or missing even though it should have been there). Ratings were made on eight dimensions of the KidzLit sessions: introduction, reading style and listener engagement, comprehension, vocabulary, discussion, connection activities, writing, group management, and caring.

Secondary Source/Data Review: LA’s BEST provided youth demographic, program enrollment, and program attendance data; and dates and descriptions of the initial KidzLit training for each site.

Surveys/Questionnaires: A 17-item questionnaire was administered to second graders and a 35-item questionnaire was administered to fourth graders. Both surveys addressed youth’s enjoyment of reading, sense of efficacy as a reader, frequency of reading outside of KidzLit sessions, and enjoyment of read-alouds. Additionally, the fourth graders’ survey asked about concern for others and altruistic behavior.

Staff survey questions focused on their KidzLit training and support; availability of books, guides and materials at their site; successes and challenges they experienced with KidzLit; staff background; and perceived youth benefits.

The activities consultant survey focused on the extent of support they provided to site staff regarding KidzLit and successes and struggles that they observed with KidzLit.

Test/Assessments: Youth’s vocabulary development was assessed using the Receptive One-Word Picture Vocabulary Test (ROWPVT), Spanish-Bilingual Version. This is an individually-administered, norm-referenced measure of oral (hearing) vocabulary in English and Spanish that was standardized on a national sample of bilingual youth aged 4 through 12. The total number of words correctly identified and the proportion of correct words identified in each language are recorded. This assessment allows a comparison of growth in vocabulary relative to growth among similar youth nationally.
Data Collection Timeframe Data were collected during the 2001–2002 school year.


Findings:
Formative/Process Findings

Activity Implementation On average, sites offered KidzLit for 45–60 minute sessions, 6 days per month, during 25 of the 29 possible weeks, for a total of 34 hours per year (71% of the hours specified for the study). Six sites had occasional cancellations of sessions, and three sites had large breaks in sessions due to staff turnover or conflicts with other program offerings.

Youth in KidzLit were exposed, on average, to 19 books, while it was expected that staff would use approximately 29 books over the year.

Staff typically used one book for every 2 session hours, slightly more than the 1.67 hours expected in the curriculum.

On-site storage and accessibility to KidzLit books for staff did not appear to be an issue, though a common challenge identified by staff was the need for more books. Most sites (n = 7) did not have the whole set of KidzLit books, and some had very few books.

About one third of the books to which participants were exposed were from outside the KidzLit collection. Staff who read the most books overall also read the highest proportion of non-KidzLit books.

Staff strongly agreed that KidzLit books were appropriate for youth at their sites and did not see the vocabulary and writing activities as too much like school.

The average implementation quality scores for KidzLit components ranged from 2.0 to 2.6 on a 4-point scale with 4 indicating high-quality implementation. Implementation quality was highest for connection activities (2.6), caring (2.6), and group management (2.5), while discussion (2.0) was the lowest rated.

Site coordinators reported a variety of factors impacting the success and challenges of implementing KidzLit. KidzLit books and materials were frequently mentioned as a reason for success while missing KidzLit books were frequently mentioned as a challenge.

Art and other work youth created during KidzLit were displayed in seven sites, most often in each site’s LA’s BEST offices.

At six sites, staff turned in weekly lesson plans to their site coordinators.

Staff used the KidzLit facilitator’s guides for an average of 57% of the books used.

In general, staff with higher implementation scores spent less time planning KidzLit sessions and used fewer facilitator guides. Staff who spent more time planning tended to be those who had less experience with youth.
Parent/Community Involvement With one exception, site coordinators reported making contact with parents regarding KidzLit. Sites made contact with parents regarding KidzLit most often by talking with them individually. They also held parent meetings which focused on or included discussion of KidzLit, sent home letters about KidzLit, and included information about KidzLit in parent newsletters.
Program Context/Infrastructure Sites had difficulty achieving consistent exposure and desired scheduling for KidzLit, particularly when youth had the choice to participate in “pull-out” activities (e.g., drill team, science club, etc.) and in sites that were expected to offer a wide range of youth activities. “Pull-out” activities occurred during fourth grade KidzLit sessions at six sites and during second grade KidzLit sessions at three sites.

Site coordinators, activities consultants, and staff agreed that KidzLit fit with the other programs and activities offered at LA’s BEST.

All LA’s BEST sites had other literacy activities in addition to KidzLit, most often drama and essay contests.

Most LA’s BEST sites (n = 6) connected KidzLit to other activities occurring on site, most often arts and crafts activities.

KidzLit sessions were fit into LA’s BESTs’ daily schedules, generally after homework and before clubs.

Time to read on their own in the school library was available to youth at four sites, while all sites provided youth with access to classrooms or LA’s BEST libraries. At seven sites, youth had access to KidzLit books outside of KidzLit sessions, and youth at two sites were allowed to check these books out to take home.
Program–School Linkages With one exception, site coordinators reported contact with teachers at the school where their site was located, which ranged from weekly to every day, most often through face-to-face discussions. Site coordinators most often reported that staff and teachers talked about youth behavior and homework but did not mention discussing KidzLit with teachers.

Problems staff reported regarding KidzLit school classroom meeting spaces were, most often, lack of storage space and frequent changes in location. Site coordinators and activities consultants did not report concerns regarding space.

Staff and site coordinators at seven sites reported that KidzLit supplies (which included pens, pencils, crayons, glue, paper, and scissors, and less often, folders, journals, and costumes) were adequate. Activities consultants did not report any concerns about supplies.
Recruitment/Participation Youth attended an average of 21 of the 25 weeks that sites offered KidzLit during the school year (86% of KidzLit sessions offered).

Youth were exposed, on average, to 25–30 hours of KidzLit sessions (a maximum of 57% of the 48 exposure hours specified in the study conditions).

Site coordinators reported an average of 16–21 youth in each KidzLit session, while staff reported an average of 22 youth, and observations revealed group sizes of 15–17 youth. Reasons for these discrepancies included: absences (average 4 per session), early departures from sessions (average 3–4 youth per session), and “pull-out” activities that occurred during sessions involving youth who would otherwise participate.

Approximately 40% of the youth in the outcome study sample were enrolled at their LA’s BEST site when the site had its initial KidzLit training. These youth could have had 4 to 12 months of exposure to KidzLit before the 2001–2002 outcome study.
Satisfaction Across sites, staff reported that most youth responded to KidzLit with enjoyment and excitement, some responded with boredom, and none to some responded with frustration.
Staffing/Training Staff often rated “managing groups of young people” as a strong skill they possessed, as well as a rating it as a common challenging skill. Staff also reported skills related to discussion, caring, comprehension, and building vocabulary as strengths that they possessed. Staff most often rated getting youth interested in a book or story and talking about sensitive topics with youth as their biggest implementation challenges.

Site coordinators reported a number of staff successes and challenges in implementing KidzLit. The most common success they noted was discussion and discussion techniques, while the most common challenge was getting and maintaining youth focus.

At the eight sites, 93% of KidzLit staff attended basic KidzLit training and a reunion session. In addition, they attended an average of two of four “Upper Division” training workshops that covered key KidzLit topics like “introducing stories” and “reading aloud effectively.

Staff feelings of preparation to implement KidzLit averaged 3.3 on a 5-point rating scale. Staff who felt more prepared had somewhat higher KidzLit implementation quality scores.

Staff cited trainings and workshops as a major factor contributing to their KidzLit success.

At four sites, one staff member led both second and fourth grade groups, while at the other four sites, two staff members led the two grade groups, one staff member for each. Four sites had changes in staff assignment to KidzLit during the year, though this did not appear to affect implementation quality.

Staff, on average, had 2–3 years experience working in school settings and/or in after school settings. Approximately 80% of staff attended college during 2001–2002, and 50% worked as teaching assistants in school classrooms. Both in-school and out-of-school experience were positively correlated with quality of KidzLit implementation (p < .05 for in-school experience; positive but not significant for out-of-school experience).

Staff experience with KidzLit at posttest ranged from 3 to 21 months, averaging 12 months. There was a small, positive, though not significant, relationship between staff members’ prior KidzLit experience and implementation quality.

Staff had positive feelings about reading and reported reading for their own enjoyment. The more positive staff felt about reading and the more they reported reading for their own enjoyment, the more positive was implementation quality (p < .05).

Staff spent an average of 30 minutes planning for each KidzLit book. At seven sites, staff had planning time during LA’s BEST hours (30 minutes per week on average), although less than 20% of KidzLit planning was done during these hours; staff did most planning on their own time.

On average, staff reported receiving consultants’ support once or twice during the year overall; for implementation, such as brainstorming session ideas and reviewing lesson plans, once or twice to every 2–3 months; and for logistics such as getting books and supplies, once or twice during the year. Extent of overall activities consultant support had a moderate positive (though not significant) relationship with implementation quality.

Nine of 13 staff reported working with another staff member on site regarding KidzLit, while staff at seven sites reported sharing different ideas with coworkers about KidzLit (on average, every 2–3 months). The frequency with which staff did these things had small, positive (though not significant) relationships with implementation quality.

Staff at five sites reported sharing group management ideas, while staff at four sites reported recommending books to each other, and developing lesson plans together (once or twice to once every 2–3 months). These activities were not related to implementation quality.

Staff at two sites reported coleading KidzLit sessions and doing so 1–3 times a month. This activity was positively though not significantly related to implementation quality.

Staff at five sites reported having had some staff or volunteer assistance during KidzLit sessions. Staff or volunteer assistance was not related to implementation quality.

The strongest benefits staff reported as a result of implementing KidzLit were improved lesson planning, more structured, organized ways of working with youth, better communication or relationships with youth, and more enthusiasm for exposing youth to reading and books. Site coordinators also reported that staff had more creative ways of working with youth as a result of KidzLit. Activities consultants also reported that staff had better communication or relationships with youth and more enthusiasm for exposing youth to reading and books as a result of participating in the KidzLit program.

On average, staff reported site coordinator enthusiasm and support of KidzLit as 4.06 on a 5-point scale. The extent to which staff felt site coordinators were enthusiastic and supportive had a significant, positive relationship with implementation quality (p < .05).


Summative/Outcome Findings

Academic Second graders showed significant increases in their sense of reading efficacy and in the amount of reading they did, though they showed significant decreases in both liking reading and enjoyment of read-alouds (p < .001 for each).

There was a significant positive correlation between second graders feelings of reading efficacy and the number of books read in KidzLit (p < .05). Implementation quality and hours of KidzLit exposure were not strongly correlated with reading-related attitudes.

Fourth graders showed significant increases in their sense of reading efficacy and amount of reading they did (p < .001 for both), but did not show increases in liking reading. They were more likely to show increases in the amount of reading they did in programs with lower quality implementation (p < .10), with exposure to fewer books (p < .05), and with fewer hours of KidzLit exposure (p < .10).

There was a significant, positive correlation between the number of books fourth graders read during KidzLit, and the degree to which they reported enjoying reading (p < .05).
Youth Development Fourth graders showed significant increases in concern for others and altruistic behavior (p < .001 for both). The higher the implementation quality, the more they increased in their concern for others (p < .05).According to KidzLit staff, the biggest benefits youth experienced as a result of KidzLit, on a scale of 1 to 5, were deeper engagement with connection activities (art, drama, music, etc., 4.17); greater ability to think critically and express ideas out loud (3.92); and greater understanding of self and others (3.85).

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project