Volume XI, Number 3, Fall 2005

Issue Topic: Democratic Evaluation

Promising Practices

Combining Research Rigor and Participatory Evaluation

Anju Malhotra and Sanyukta Mathur from the International Center for Research on Women describe a study in Nepal that compared participatory and more traditional approaches to evaluating adolescent reproductive health interventions.

Working with the Nepal Adolescent Project (NAP), the International Center for Research on Women (ICRW) has developed a model of participatory public health intervention and evaluation intended to define and address adolescent reproductive health concerns. NAP's goals were to (a) improve youth reproductive health outcomes on indicators such as knowledge of pregnancy, STDs, HIV prevention, and other adolescent changes and problems and access to contraceptive, HIV, maternal care, and basic reproductive health services; and (b) empower youth and adults to change social norms and values on early marriage and childbearing, lack of mobility and voice for girls, and lack of opportunity and options for young women and men. The project was conducted between 1998 and 2003 in collaboration with an international service delivery organization, Nepali governmental and nongovernmental organizations, and local communities.

Comparing the Approaches
Despite its hypothesized advantages, the success of a participatory approach in reaching young people in developing countries and improving their reproductive health previously had not been scientifically tested. ICRW rigorously tested this approach through a quasi-experimental study at four sites. Two study sites, one rural and one urban, used a participatory approach for research, intervention, monitoring, and evaluation, while two control sites used more traditional reproductive health research and interventions.

Study and control sites differed in both how the program was implemented and what elements it included. The study site's intervention design and implementation were more comprehensive, inclusive, and interactive, with an emphasis on building community ownership and involvement. This was achieved through several mechanisms and structures, such as advisory groups, coordination teams, and consultative committees that engaged youth and adult community members. ICRW and partners used an action-planning process to design interventions, with communities receiving and analyzing needs assessment data and developing task forces to prioritize and design feasible interventions. Implementation structures were inclusive, facilitating adult and youth authority and decision making. Control sites lacked these mechanisms.

Intervention components at the study sites were substantially more elaborate, locally tailored, and comprehensive than at the control sites. They addressed structural, normative, and systemic barriers to youth reproductive health, while control sites addressed only the most immediate risk factors, such as STDs or unwanted pregnancies. Study site interventions tended to link direct youth reproductive health programs with other programs aimed at influencing the youth's environment, such as those addressing adult education, social norms, and economic livelihoods. In comparison, at control sites, project staff designed and implemented three standard reproductive health interventions focused on basic risk factors for disease and unintended pregnancy.

ICRW compared changes in key indicators at the control and study sites. Triangulated methodologies—quantitative, qualitative, and participatory—were used to collect data from young people, parents, other relevant adults, community leaders, and service providers.

The Results of the Approaches
The participatory approach yielded more positive results on outcomes of interest overall. While study site results were only marginally more positive on basic risk factors such as contraceptive use or reporting of STD symptoms, they were substantially more positive on broader, more fundamental determinants of youth reproductive health, such as delayed marriages, childbearing, girls' education and mobility, young men's understanding of maternal issues, and the ability of young women and men to understand and articulate the familial, power, and gender dynamics of reproductive health issues.

Additionally, the participatory approach in the study sites was substantially more successful in empowering youth and community members, building the capacity of local partner organizations, and laying the seeds of sustainability. Young people learned to mobilize on their own behalves, whether by organizing communication campaigns that extended beyond original program messages (e.g., to address drug and alcohol use) or fundraising for a youth center with a library. Similarly, parents and leaders acquired the information and confidence to hold service providers and politicians accountable and assert their entitlement to benefits. Local organizations acquired conceptual expertise in addition to training and tools and now are contributing to a substantial increase in development practitioners with higher skill levels. Most critically, new structures, mechanisms, and processes, such as networks of health service providers, parent committees, and legitimate social spaces for young people, now provide hope for continued and long-term action and innovation.

Anju Malhotra
Director
Population and Social Transitions
Email: anju@icrw.org

Sanyukta Mathur
Public Health Specialist
Email: smathur@icrw.org

International Center for Research on Women
717 Massachusetts Avenue, NW, Suite 302
Washington, DC 20036