The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume VI, Number 1, Fall 2000
Issue Topic: Out-of-School Time Issue #1
Promising Practices
Reflections on Access and Equity Implications for After School Program Evaluations
Laurie Olsen, Executive Director California Tomorrow, highlights the importance of addressing issues of access and equity in the evaluation of after school programs.
In response to the day-to-day realities of 28 million school-age children and the seven million latchkey kids who care for themselves and each other every afternoon, there has been an explosion of public and private investment in after school programs. The policy intents behind this encompass a variety of concerns, from juvenile crime, child safety, and support for mothers in the workforce to youth development and assuaging poor academic performance. While funding tends to be universal and non-needs based, the concerns addressed by after school programs disproportionately impact poor children, immigrant children and youth of color. In the past, there have been too many examples in which low-income communities were disproportionately excluded from resource allocations, and low-income, immigrant youth, and children of color were not well served by public institutions. This is often because resources and programs do not explicitly target their needs and address their barriers to involvement. Without deliberate efforts to reach out to the less well served, more sophisticated organizations and communities tend to garner most of the resources, funding and services for after school programming.
With major public investment in after school programs and a dramatic expansion of the field, program evaluations play a critical role in raising and framing questions for programs and communities, collecting data to inform policy direction, and monitoring patterns of inclusion and access. For these reasons, it is crucial to consciously focus on equity and access in after school program evaluations.
21st Century Community Learning Centers:
|
As California Tomorrow, with a focus on both equity-centered school reform and capacity building in diverse communities, explored issues of equity and access in after-school programs, it became clear that few programs, funders, or evaluators addressed these specific dimensions within the program movement. Yet many with whom we spoke voiced an interest in more deeply understanding how an access and equity framework might help shape evaluation research questions, the nature of the measures used to evaluate success and quality, and the methods used to collect data.
As a result, California Tomorrow framed a working set of principles defining equity and access in after school programs, presented here with reflections on their implications for evaluators.
-
After school programs and policies should result in improved outcomes for youth and should play a role in countering current gaps and inequities in social and educational supports and outcomes. In order to measure after school programs’ abilities to close gaps in social and educational supports and outcomes for youth, it is essential that measures of outcome improvements be accompanied by disaggregated comparative data that provide a base line and over-time measure of the size of gaps between racial/ethnic groups, gender, social and economic status levels, and levels of English fluency. Gaps should be measured both within and across programs. Because most programs are neighborhood and residentially based and regions are often divided down racial and ethnic lines, this comparative data should enable the evaluator to examine, for example, the level of support provided and the impact of programs in poor communities with the level of support and impact of programs in wealthier or suburban communities.
-
Resources should be allocated with a focus on the creation of after school support for children living in communities where such programs are scarce. This principle requires close examination of who receives funding to provide services. Location of funded programs can be overlaid on a map of existing community resources and services, allowing for an analysis of locales where services are scarce. Furthermore, a comparison of the location of funded programs with socioeconomic status, health status, measures of safety, and racial/ethnic data about communities will demonstrate whether the services are getting to children and communities most in need. Where possible, it would be helpful to document “best practices” in initiative and policy design that result in resources being accessible to and used by those communities most in need. Within programs, comparative data on those actually served by a program and the youth population in the school attendance area enable evaluators to explore exactly who receives support and whether those served are the children in greatest need.
-
Policies and practices in the field should ensure that no community is excluded from being able to provide or access after school services due to barriers related to language, income, skin color, immigrant status, gender, and physical disability. All of the above characteristics have a history of sorting young people into those with access and better social and educational outcomes and those without. Because of this, it is imperative that evaluations of program effectiveness include an examination of the barriers present that result in limited access and involvement. Some of these may be obvious (i.e., information about funding and grant application meetings are only advertised and conducted in English, despite the existence of large language minorities in a community; program sites without ramps or elevators and so access is precluded for young people with physical disabilities), but often an evaluation of barriers requires an analysis of whether students are representatively enrolling. Programs should deliberately seek the perspectives of potentially excluded groups to determine those barriers that may hinder their participation in the program.
-
Programs should foster a positive sense of identity, build upon the cultures of the families, and offer a curriculum that values and responds to the strengths, challenges, and needs of all of the different kinds of youth in their communities (e.g. youth of different ethnicities, class backgrounds, spiritual beliefs, genders, sexual orientations, and physical or cognitive abilities). An access and equity lens should not only focus on who enrolls in programs, it should also be concerned with the responsiveness and appropriateness of what occurs in programs. Many times it is the practices, attitudes, and hidden or overt curriculum that give some young people the message that they are worth less than others. Basic quality indicators should include aspects related to responsiveness and inclusiveness of all students. An analysis of this principle requires frameworks for assessing cultural and linguistic responsiveness, observations and data collection aimed at documenting attitudes and expectations, examination of the content of curriculum and programs, and approaches to elicit the perspectives and experiences of young people and parents about these issues.
-
After school programs should strengthen the capacity of young people to be active, contributing members of their families, communities, and our increasingly diverse society. Youth development principles have been at the heart of many after school programs in the past. In light of recent program expansion and focus on academic and social outcomes, some programs may work to develop youth leadership, building their ability to be active, contributing members of their families and communities. Evaluations should include multiple kinds of outcome measures, both academic and social. They should seek evidence of ways in which young people are supported and involved in their communities, providing leadership within their programs, and reaching across divides of language, culture, skin color, and class.
- Policies and practices should support communities, parents and youth in shaping programs and determining which organizations should provide after school services. To examine this aspect of programs, an evaluator would document governance practices and policies and the degree of representative involvement that parents and youth have in decision making. It is important to look for which parents and youth are involved to ensure that the voices of those who may find it more difficult to be heard are, in fact, present. Evaluations would be concerned with whether or not the policies and procedures in design and funding allow for community involvement in determining which organizations would provide the program.
Conclusion
Key to all of these reflections is the importance of collecting demographic data and disaggregating outcome data, designing evaluations in ways that promote multiple perspectives, conducting evaluations in ways that support the involvement and engagement of people who are often excluded from governance and decision making (students, parents, language minority communities, etc.), and developing frameworks for observations and data collection to elicit information on barriers to confirm the presence or absence of culturally responsive practices. But, perhaps most importantly, we must use the evaluation process and data to promote dialogue, inspire reflection, and stir engagement within and among communities about their desired outcomes for youth and their visions of equity and inclusion.
We invite and encourage dialogue from readers about our working framework, ideas for equity-centered approaches to after school program evaluation, and about what is needed to support evaluators in deeper investigation of the equity and access dimensions of after school programs.
For more information, email Amy Sharf, Senior Program Associate at California Tomorrow at amys@californiatomorrow.org. California Tomorrow is a nonprofit research, technical assistance, and advocacy organization committed to building a strong, fair, and inclusive multicultural society.
Laurie Olsen
Executive Director
California Tomorrow
436 14th Street, Suite 820
Oakland, CA 94612
Tel: 510-496-0220