The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.
Volume X, Number 1, Spring 2004
Issue Topic: Evaluating Out-of-School Time Program Quality
Promising Practices
Using Technology to Link Evaluation and Funding: San Francisco’s Contract Management System
Tajel Shah and Nani Coloretti of the San Francisco Department of Children, Youth and Their Families reflect on the value and limitations of a web-based contract management system for youth programs.
Two trends are currently colliding within city agencies, foundations, and other public and nonprofit arenas: the push to strengthen data-driven evaluative processes, and the desire to fund programs based on proven, positive results. San Francisco’s Department of Children, Youth and Their Families (DCYF) is one place where this intersection is showing promising results. DCYF funds, strengthens, and coordinates over 200 child and youth services in the City and County of San Francisco through distribution of the Children’s Fund, a property tax set-aside of over $28 million per year. The DCYF approach is one of results-based accountability, and a key component of the process is an innovative web-based data management system known as the Contract Management System (CMS).
An Accountability Tool for DCYF
DCYF originally developed the CMS as a paperless grant management system of online reports, forms, and invoicing. DCYF then enhanced the system to capitalize on its potential for collecting outcomes-based evaluation data, a critical component of DCYF’s results-based approach. Today the CMS is a sophisticated tool required of all DCYF-funded agencies.
Community-based organizations (CBOs) must use the CMS to record their projected annual work plans, which are tied to citywide goals for youth and to standardized performance measures. CBOs must then record data such as participant enrollment and demographics, and use the CMS to submit monthly reports and invoices online. The monthly timeframe and detailed data allow DCYF program officers to monitor grantees’ progress toward department goals, to analyze citywide trends, and to gauge the impact of Children’s Fund dollars.
Related ResourcesDepartment of Children, Youth and Their Families. (2003). Snapshot: San Francisco’s children and youth today. San Francisco: Author. www.dcyf.org/Pubs/final_CNA.pdf Department of Children, Youth and Their Families. (2002). Children’s services allocation plan: A blueprint for the children’s fund. San Francisco: Author. www.dcyf.org/Pubs/csap/CSAP_final_1125.pdf CitySpan Technologies, Inc., in Berkeley, California, is an e-services consultancy providing web-based information systems to public agencies and nonprofit corporations. www.cityspan.com |
A Self-Assessment and Self-Promotion Tool for Grantees
Using preset report templates, CBOs can generate critical information for managing their programs more effectively, for communicating with their staff and board of directors, and for presenting detailed and accurate data to potential funders.
The CMS has also helped grantees think more critically about their organizational goals. Analyzing outcomes helps agencies define their work and articulate it to others. Some CBOs, for example, have integrated CMS data into grant proposals, supporting DCYF’s hope that the CMS can help leverage additional resources for San Francisco children and youth.
A Tool Enhanced by Other Methods
While the CMS has thus far proven to be a powerful tool for linking on-the-ground programs to results, some limitations should be noted. Although the CMS can help to document whether a program achieved a certain outcome, it cannot analyze why a program did or did not work. Another limitation is that the CMS does not document whether some children use more than one program. As a result, when all local program data are aggregated, the number of children served across the community may be overstated. While enhancements to the CMS may mitigate some of the system’s limitations, the technology cannot supplant the role of DCYF staff and evaluators, who regularly visit funded agencies to assess organizational health and program impact.
Scaling Up: The Full Impact of the CMS
The CMS has already proven its value to DCYF and the children’s services community of San Francisco. However, its full capability will not be realized until DCYF fully implements its results-based approach this year. This change will be completed with the next request for proposals (RFP), due in March 2004, which will make an estimated $25 million available per year for 3 years to child- and youth-serving agencies that have been selected through a competitive process. DCYF staff is currently using CMS data to help design the RFP. The data will also be used to determine, in part, which programs and agencies will receive new grants. Once funding has been awarded, use of the CMS will continue to play a central role in the cycle of program planning and tracking, and in DCYF’s use of the Children’s Fund as an instrument of social change for the city’s young people.
Tajel Shah
Assistant Director of Technology and Communications
Email: tajel@dcyf.org
Nani Coloretti
Director of Policy, Planning, and Budget
Email: nani@dcyf.org
San Francisco Department of Children, Youth and Their Families
1390 Market Street, Ste. 900
San Francisco, CA 94102
Tel: 415-554-8415
Website: www.dcyf.org