You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

www.HFRP.org

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Program Description

Overview The Making the Most of Out-of-School Time (MOST) Initiative is a system-building initiative. MOST seeks to contribute to the supply, accessibility, affordability, and quality of out-of-school time programs, especially for low-income children, by strengthening the system of out-of-school time programs. The MOST initiative involves three cities: Boston, Chicago, and Seattle.
Start Date 1994–2001
Scope national
Type after school, summer/vacation, before school, weekend
Location urban
Setting public schools, community-based organizations
Participants kindergarten through middle school students
Number of Sites/Grantees three cities (Boston, Chicago, Seattle)
Number Served not available
Components MOST began in 1994 with one-year planning grants to Boston, Chicago, and Seattle to develop three-year action plans to address the Initiative's goals. These cities then received grants of $1.4 million each for the three-year implementation of their action plans (1995–1998, Phase I). Under the umbrella of one or more lead agencies, each MOST city: brought together the stakeholders in the out-of-school time program system to set priorities, did joint planning, shared information, coordinated activities, and began to develop citywide strategies for addressing after school challenges. Then, the cities provided funds directly to out-of-school time programs through an RFP process.

The National Institute on Out-of-School Time (NIOST) received a grant from the Wallace-Reader's Digest Funds (WRDF) to provide technical assistance and training support to the cities. WRDF also made a grant to the Center for Career Development in Early Care and Education at Wheelock College in Boston, Massachusetts to help MOST communities develop career ladders, increase the availability of affordable and accessible training, and develop college-level courses that lead to a degree or certificate for local providers. WRDF has also supported the National School-Age Care Alliance's (NSACA) work to develop national standards of program quality.

In October 1998, WRDF awarded additional three-year grants totaling $3.3 million to the three cities and NIOST. The purpose of these grants is to: support efforts to strengthen, document, and assess the models in Boston, Chicago, and Seattle; work with national organizations to provide technical assistance to other communities interested in creating similar systems; and inform key decision makers about promising strategies for meeting families' needs for accessible, high-quality school-age care.
Funding Level $9.7 million over seven years
Funding Sources The Wallace-Reader's Digest Funds


Evaluation

Overview The purposes of the evaluation were to describe the MOST strategies in each city and examine their effectiveness. Additionally, the evaluation sought to describe the important dimensions of the after school program system in each city (supply, program quality, and financing) in order to provide context for understand MOST efforts, and to aid future investment in the field.
Evaluator Chapin Hall Center for Children, the University of Chicago
Evaluations Profiled Interim Evaluation Report: 1997
Final Evaluation Report: 2000
Evaluations Planned none
Report Availability Halpern, R., Spielberger, J., & Robb, S. (1998). Making the Most of Out-of-School Time: Executive summary. Interim findings from an evaluation conducted by Chapin Hall Center for Children at the University of Chicago. Chicago: Chapin Hall Center for Children at the University of Chicago.

Halpern, R., Spielberger, J., & Robb, S. (2001). Evaluation of the MOST (Making the Most of Out-of-School Time) Initiative: Final Report and Summary of findings. Chicago: Chapin Hall Center for Children at the University of Chicago.
Available at: www.chapinhall.org/article_abstract_new.asp?
ar=1316&L2=62&L3=105


Contacts

Evaluation Robert Halpern
Chapin Hall Center for Children at the University of Chicago
1313 East 60th Street
Chicago, IL 60637
Tel: 773-753-5900
Program Ian Beckford
The Wallace Funds
Two Park Avenue, 23rd Floor
New York, NY 10016
Tel: 212-251-9700
Profile Updated December 2, 2001

Evaluation 1: Interim Evaluation Report: 1997


Evaluation Description

Evaluation Purpose To develop and test a methodology for ascertaining the supply of school-age care in each city, to examine National Institute for Out of School Time's (NIOST) efforts to pilot an approach called Assessing School-Age Quality (ASQ), to examine the question of what is meant by a school-age care system. ASQ, developed by NIOST, is a structured self-assessment approach to program improvement, linked to an accreditation system sponsored by the National School Age Childcare Alliance. It identifies 21 categories of program quality, under the general headings of human relationships, indoor environment, outdoor environment, activities, safety, health, and nutrition.
Evaluation Design Non-Experimental: The interim evaluation consisted of a series of mini-studies, each examining a different aspect of MOST. For the ASQ study, evaluators examined 10 school-age care programs in each city (for a total of 30). MOST staff in each city selected these programs.
Data Collection Methods Document Review: Reports and other documents were examined from the MOST lead agencies, committees, and working groups, and school-age care programs.

Interviews/Focus Groups: Interviews were conducted with MOST staff, staff of school-age care programs, host organizations, support organizations, funders, regulators, and the staff of NIOST, among others. The interviews were held to learn more about the school-age care supply and system in each city, the quality of school-age care programs, program costs and revenues, and the MOST infrastructure.

Observation: Evaluators observed MOST committees and working groups, MOST-sponsored training events and workshops, and school-age care programs.

Secondary Source/Data Review: Administrative data were collected, particularly data which related to supply and financing of out-of-school time programs.
Data Collection Timeframe Data were collected between the spring of 1997 and the summer of 1998.


Findings:
Formative/Process Findings

Systemic Infrastructure The ASQ Experience: Participants in the ASQ pilot process generally regarded it as a useful program improvement tool and process, although experience varied widely depending on program, organization, funding, contextual, and staff characteristics. Participants also reported being surprised and, in some cases, frustrated at the amount of work involved in the ASQ. Not every program benefited equally from the ASQ process, but most benefited to some degree. Participants found that the process created mechanisms for and sources of feedback that led programs to reflect on their strengths and weaknesses. It also raised consciousness in the host agencies around quality issues. Lessons also emerged about how to implement program improvement processes, including the fact that stronger, healthier, and more stable programs are better able to use and benefit from program improvement supports such as ASQ.

MOST as a System-Building Initiative: All three MOST cities had a patchwork of large and small providers, although there was some variation across the cities. The nature of the intermediary organizations also varied by city. There was no centralized governance mechanism or institutions for school-age care in any of the three MOST cities. Leadership of MOST initiatives was diffuse, informal, based largely on length of involvement in the field, and to some extent self-selected. The functioning of the system is shaped by a variety of working relationships between providers and funders and between providers and intermediary institutions; working relationships are also important among intermediaries, and to a lesser extent, among providers.

Evaluators noted that organizational heterogeneity, diffuse identity, and varied perspectives about the purpose of school-age care programs complicate the system-building task in school-age care. Evaluators noted that MOST reflects a complementary strategy. The strategy has been to identify the various agencies, organizations, and component elements that are part of a school-age care system, broadly defined, and try to both strengthen each in place (to help them do their daily work as it is currently defined), and to create new structures and mechanisms for citywide systemic activities. Lessons and conclusions also emerged about how to measure the supply of school-age care within a city.

Overall MOST Strategy: In each city, MOST had a distinct personality and had fit itself into the larger political, social, and institutional context in a different way. MOST staff and collaborators were responsive and attentive to the mission of strengthening school-age care in underserved neighborhood and among underserved populations. MOST funds were used to strengthen school-age care programs and lead agencies were growing steadily in their ability to manage grants. MOST was helping to clarify the attributes of a strong school-age care system. MOST's reliance on voluntary committees and working groups for oversight, planning, and priority setting has been a mixed experience.

Evaluators pointed out that the largest question facing MOST is whether its voluntary, collaborative structure is intended to serve as the germ for a more permanent governance structure for school-age care. Evaluators also noted that MOST has many objectives and, as a consequence, MOST resources have been spread too thinly in these large cities. They recommended that the second phase of MOST be focused on a well-defined strategy to strengthen program quality, and selectively promote system building, rather than on broad-based system building and increasing the supply of school-age care slots.


Summative/Outcome Findings

Systemic By the end of the first two and a half years, the three MOST cities had made moderate to substantial progress toward their goals of increasing the supply of school-age care slots. New slots were created in a variety of ways: providing MOST funds to existing programs to be used to pay part of the cost of new slots, using MOST funds to start new programs or satellite sites for existing programs, and providing funds to making existing slots more affordable.

Evaluation 2: Final Evaluation Report: 2000


Evaluation Description

Evaluation Purpose To examine sponsoring and supply of out-of-school time programs, program quality, financing, and MOST's effectiveness. Evaluation challenges, such as the lack of a baseline, lack of comparison sites, and identifying the boundaries of MOST, led evaluators to focus on determining whether MOST investments and efforts created things (e.g., new programs or affordable after school slots, new relationships between individuals or organizations, new program activities, or improved facilities) that had not existed before, and probably would not have been created in the natural course of events, had MOST not existed (original emphases). An evolving policy and financing environment locally and nationally complicated the task.
Evaluation Design Non-Experimental: The final report of the evaluation consisted of three descriptive studies (sponsoring and supply, quality, and financing) as well as an examination of MOST's effectiveness. For the quality study, evaluators conducted in-depth observations of 10 programs over the course of a program year (1997–1998); a set of six attributes of good after school programs was used to guide the observations. These attributes were developed by drawing on child development literature, previous work in the field, and existing standards and include staff quality, staff to child ratios, rules and supervision, qualities of program schedule and variety, quality of snacks, and facilities.
Data Collection Methods Document Review: Reports and other documents from the MOST lead agencies, committees and working groups, and school-age care programs. For the cost and revenue study, primary data were budgets from about 40 individual programs in Boston, and about 10 each in Chicago and Seattle, as well as some information from large public and private providers such as schools and park districts.

Interviews/Focus Groups: Interviews were conducted with MOST staff, staff of school-age care programs, host organizations, support organizations, funders, regulators, and the staff of NIOST, among others. The interviews were intended to provide information about the school-age care supply and system in each city, the quality of school-age care programs, program costs and revenues, and the MOST infrastructure.

Observation: MOST committees and working groups, MOST-sponsored training events and workshops, and school-age care programs were all observed.

Secondary Source/Data Review: Existing administrative data (particularly on supply and financing of out-of-school time programs) were reviewed.
Data Collection Timeframe Data were collected between the spring of 1997 and the summer of 1998.


Findings:
Formative/Process Findings

Cost/Revenues Full Program Costs: There is a lack of basic information on various dimensions of financing, therefore a rudimentary study had to be conducted. The full cost of providing a five-days-a-week program appears to range from $65 to $95 per child per week, with an average of about $80 per week or $4,000 per child for a 50-week year. Subtracting in-kind contributions, the full cost appears to be about $50 to $60 per child per week, or $2,500–$3,000 per child for a 50-week year.

Sources of Revenue: Sources of revenues for after school programs in the study included public funding, primarily childcare subsidies (between 40–50% of program revenues), parent fees (between 15–25%), and private contributions (United Way, corporate or community foundations, and donations, between 10–20%). There were also indirect, in-kind contributions such as free space. Although parent fees are the main source of revenue for after school programs serving more advantaged children, they are only a modest revenue source for programs serving low-income children. When parent fees are charged, there is invariably a sliding scale.

The number of revenue sources, the amount of revenue from each source, and year-to-year continuity in amount and sources of revenue varied widely across programs. Although there was significant variation among the programs studied, revenues were typically about $1,500–$2,000 per child, considerably less than program costs. These deficits were often covered by sponsoring agencies, more adequately funded programs, donations, or reduced investments in staff or facilities.
Program Context/Infrastructure The after school field in these cities is made up of a broad and diverse group of providers. The role and importance of particular types of providers varies from city to city, and within cities, from neighborhood to neighborhood. The largest providers of after school programs for low- and moderate-income children are private, nonprofit social service agencies and schools.

Only a modest proportion of low-income children in each city participated in after school programs. Evaluators suggest that reasons for this include inadequate public financing that limits both the number of slots and parental demand, transportation difficulties, lack of parental awareness of available options, and/or lack of parental wherewithal to seek out and enroll children in programs.

Although the 10 programs studied had a mixture of strengths and weaknesses, most fell short on the majority of the six quality indicators. Evaluators suggested that two important factors seem to influence quality: inadequate funding and the composition of the workforce. Inadequate funding leads to inadequately staffed programs, insufficient supplies and materials, lack of resources for management functions, lack of investment in facilities and equipment, and low wages. The characteristics of frontline staff positions (part-time, low-wage, and no benefits) hampers efforts to recruit and retain high-quality staff.


Summative/Outcome Findings

Systemic All three sites achieved their numerical goals for slot creation, but through different routes. They implemented a diverse strategy that focused on making existing slots more accessible and affordable, creating new ones, improving information for consumers, and engaging in public education on the need for after school programs. In each city, MOST directed its supply-building resources equitably, with an emphasis on locating gaps in supply and supporting new slots where the need was the greatest. Two issues that accompanied supply-building efforts were the quality of new supply and sustainability. Each city approached these issues differently.

The bulk of MOST resources and efforts in each city was directed toward strengthening the quality of after school programs in low-income communities. NSACA standards served as the framework guiding program improvement efforts. MOST faced the challenge of finding an appropriate balance between depth and breadth in its program improvement investments.

Program improvement efforts involved three types of investments: in facilities, equipment, and materials; in individual program staff; and in programs as a whole. Investments in facilities, equipment, and materials tended to have the clearest, most immediate benefits. MOST investments in individual staff had quite modest effects on the quality of after school programs, both because of the nature of the workforce and general staff instability.

Strategies that invested in programs as a whole (through promulgation of standards, structured self-assessment, technical assistance, and linking programs to external resources) were somewhat more effective at bringing about program improvement than those directed at individual staff. The MOST Initiative illuminated the challenges of program improvement and the many factors that affect it, including lack of resources, staff turnover, organizational receptivity, different organizational motives, and limited capacity to use assistance. The MOST experience also taught that progress in program improvement is not linear.

MOST worked in a variety of ways to strengthen after school programs as a city-level system. MOST engaged in four distinct system-building activities: governance and leadership development, linkage strategies, leveraging strategies, and advocacy and public awareness.

A unique characteristic of MOST in all three cities was the development of a set of collaborative structures. MOST clearly made a contribution to the coherence and “working effectiveness” of after school programs as a system in each city. MOST looked across the whole city and found opportunities to link stakeholders with complementary strengths and needs. Over time, MOST efforts yielded scores of new working relationships, collaborations, and networks. Planning for MOST brought together local leaders and representatives of different segments of the system together on a sustained basis. MOST's governance groups and working committees created new venues for decision making around after school issues.

MOST leadership efforts tended to be shaped by the context of each city. MOST efforts to leverage new resources for after-school programming were successful on a number of fronts. Evaluators pointed out, however, that the three years allotted to the first phase of MOST was only enough time to begin the long-term task of system building. This is one of the reasons that the three cities and NIOST received funding for another three years.

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project