Search Publications | The Evaluation Exchange | FINE Network | OST Database

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

A Profile of the Evaluation of Fifth Dimension/University-Community Links

Program Description

Overview	The Fifth Dimension/University-Community (UC) Links is an approach to after school programming used by Boys and Girls Clubs, YMCAs and YWCAs, recreation centers, and public schools across America, Mexico, Brazil, Australia, Denmark, Sweden, Spain, and Russia. It provides a way to increase the educational programming of such institutions without substantially increasing the costs of operation. Beginning in 1986, the overarching goals of the Fifth Dimension approach were to: (1) create sustainable activity systems that increase understanding of the cultural mediation of mind and the processes of cognitive and social development, (2) provide contexts for children to master knowledge and skills mediating changes in their everyday practices, (3) deepen understanding of how the social and individual create each other, and to (4) provide a context in which undergraduates from disciplines such as teacher education, developmental psychology, and communications have opportunities to connect theory with practice and deliver services to children. In 1996 the University of California Office of the President provided seed funding to expand this model throughout California. The UC Links network was established to promote the university eligibility and academic preparedness of underserved youth throughout California using the practices developed by Fifth Dimension programs worldwide. The intent was to broaden the base of K–12 learning, by providing access to educational resources to those youth who are not yet on the path to higher education. The statewide program has three major objectives: (1) to improve the educational opportunities and aspirations for diverse California youth from low-income communities throughout the state, so that more of them will become eligible for the University of California in years to come, (2) to improve undergraduate education by providing living laboratories where students can integrate theory and practice, and (3) to pioneer new ways of collaborating that bring together institutions like universities, communities, and K–12 schools in long-term, sustainable programs for underserved K–12 youth.
Start Date	1986
Scope	international
Type	after school
Location	urban, suburban, rural

Setting	community-based organization, public school, recreation center
Participants	elementary, middle, and high school students (ages 5–18)
Number of Sites/Grantees	over 50 worldwide sites (2003)
Number Served	The number of children per site will vary according to setting. In a public school setting, as many as 80 children may be served in any week.
Components	Fifth Dimension programs take place in a computer club that may or may not be associated with the child's school and are located near university campuses since most are affiliated with colleges and universities. Participants are often from low-income homes and many attend on a regular basis through high school. At some sites, high schoolers who attended return and act as mentors for the younger children. Children typically visit a Fifth Dimension on a drop-in basis, although in most public school Fifth Dimensions children attend on a daily basis. Some children spend four to six hours per week of after school time participating in the Fifth Dimension, while others may come once a week for a few hours. Opportunities and constraints vary across locations, seasons, populations, and sites. Many children enter the Fifth Dimension directly after school or homework sessions. The main activities center around off-the-shelf educational programs selected for appropriateness, appeal, and educational value. In a prototype Fifth Dimension system (local names vary), a dozen or more children encounter a large variety of computer games, noncomputer games, and telecommunications activities on a regular basis over the course of a school year. As a rule, Fifth Dimensions contain a variety of kinds of computers at a ratio of one computer for every two to three children. Computer games, such as Carmen San Diego and the Secret Island of Dr. Brain, the Magic School Bus series, and noncomputer games, such as Origami, chess, and Boggle are a part of a make-believe activity system. Task cards that accompany each game or activity enable children and their mentors to negotiate goals for playing games and coordinate activities. Learning is collaborative with children often working together or with adult mentors called Wizard's Assistants. Students with extensive experience are given the opportunity to achieve the status of Young Assistant to the Wizard. The child's participation is voluntary and self-paced; there is no traditional teacher-student structure. Each Fifth Dimension has a site coordinator who is trained to recognize and support Fifth Dimension's pedagogical ideals and curricular materials. Undergraduate students are there to learn and play with the children. The site coordinator monitors the balance of education and play in interactions between children and undergraduates.
Funding Level	varies by local program
Funding Source	The Andrew W. Mellon Foundation provided funding for field-testing and evaluation of the original three programs begun in 1986. In 1996 the University of California Office of the President began funding the UC Links network of after school programs in California. Each local Fifth Dimension program negotiates operating resources from the university, community participants, and other outside funding agencies.

Evaluation

Overview	Evaluators developed collaborations for evaluation at three Fifth Dimension sites: Appalachian State University (ASU), California State University at San Marcos (CSUSM), and University of California at Santa Barbara (UCSB). The ASU team evaluated Fifth Dimension sites that were operated as part of an after school program at four elementary schools in Boone, North Carolina. The CSUSM team evaluated a Fifth Dimension site operating at the Boys and Girls Club in Escondido, California. The UCSB team evaluated a Fifth Dimension site operating at the Boys and Girls Club in Goleta, California. Evaluators wanted to assess changes in children's literacy that can be attributed to participation in the Fifth Dimension, specifically: (1) does learning to use educational software on computers improve a child's mind? and (2) what kinds of cognitive changes, if any, occur over the course of a year in which computer-naïve children learn to use a series of educational programs in an informal and nonthreatening environment? The outcome measures tapped changes in student literacy, broadly defined, including changes in computer literacy, language comprehension, problem-solving strategies, and academic achievement. The primary research method was to assess relevant cognitive skills of students before they began the Fifth Dimension program and after they had extensive experience in the program (e.g., more than 10 or 20 visits over the course of the academic year), and to compare their pretest to posttest changes with those of similar students who did not participate. Separate evaluations were also conducted of the Expedition after school program, a Fifth Dimension program with an archaeological learning framework in Oakland, California. Expedition involves University of California at Berkeley (UCB) faculty, staff, and students directly with sixth graders through a service learning course, Anthropology 128, Archaeological Practice in a Sixth Grade After-School Program. These evaluations sought to measure the program's success in achieving a number of youth outcomes, as well as UCB undergraduate students' goals.
Evaluators	William E. Blanton, Gary B. Moorman, Bobbie A. Hayes, and Mark L. Warner, Appalachian State University Miriam W. Schustack, Rachelle Strauss, and Patricia E. Worden, California State University at San Marcos Richard E. Mayer, Richard Durán, Amy Lavezzo, Roxana Moreno, Jill Quilici, David Sanchez, Rebecca Simon, and Scott Woodbridge, University of California at Santa Barbara Tamara Lynn Sturak, UC Links Program, Graduate School of Education, University of California, Berkeley
Evaluations Profiled	Effects of Participation in the Fifth Dimension on Far Transfer (ASU) Learning About Technology in a Non-Instructional Environment (CSUSM) Cognitive Consequences of Participation in a Fifth Dimension After-School Computer Club (UCSB) What is Learned in an After-School Computer Club? (UCSB) Evaluation of Expedition – Computers and Archaeology After School (UCB) Expedition – Computers and Archaeology After School: Year-End Report, 2000–2001 (UCB)
Evaluations Planned	Evaluations are ongoing.
Report Availability	Note: This is just a sampling of reports available. Many of these reports, as well as others, are available at: www.education.miami.edu/ blantonw/5dclhse/publications/pub1.html. Summary Reports Mayer, R. E., Blanton, W. E., Durán, R., & Schustack, M. W. (1999). Using new information technologies in the creation of sustainable afterschool literacy activities: Evaluation of cognitive outcomes. Available at www.psych.ucsb.edu/~mayer/fifth_dim_website/ HTML/res_reports/research_reports.html. Mayer, R. E., Schustack, M., & Blanton, W. (1999). What do children learn from using computers in an informal collaborative setting? Educational Technology, 39(2), 27–31. Mayer, R. E. (1997). Out-of-school learning: The case of an after-school computer club. Journal of Educational Computing Research, 16, 333–336. Underwood, C., Welsh, M., Emmons, C., Lerner, D., & Sturak, T. (2002). University-Community Links to higher learning: Program impact report. Berkeley: University of California at Berkeley, Office of the President, Educational Outreach Department. Available at: www.uclinks.org. ASU Blanton, W. E., Moorman, G. B., Hayes, B. A., & Warner, M. L. (1997). Effects of participation in the Fifth Dimension on far transfer. Journal of Educational Computing Research, 16, 371–396. CSUSM Schustack, M. W., Strauss, R. & Worden, P. E. (1997). Learning about technology in a non-instructional environment. Journal of Educational Computing Research, 16, 337–352. UCSB Mayer, R. E., Quilici, J., Moreno, R., Durán, R., Woodbridge, S., Simon, R., et al. (1997). Cognitive consequences of participation in a Fifth Dimension after-school computer club. Journal of Educational Computing Research, 16, 353–369. Mayer, R. E., Quilici, J. H., & Moreno, R. (1999). What is learned in an after-school computer club? Journal of Educational Computing Research, 20, 223–235. UCB Sturak, T. L. (2000). Evaluation of Expedition – computers and archaeology after school. Berkeley: Interactive University Project, University of California at Berkeley. Available at www.mactia.berkeley.edu/aop/activity/expedition.pdf (Acrobat file). Sturak, T. L. (2001). Expedition – computers and archaeology after school: Year-end report, 2000–2001. Berkeley: Interactive University Project, University of California at Berkeley.

Contacts

Evaluation	UC Links Mara Welsh Mahmood UC Links Statewide Office 615C University Hall Berkeley, CA 94720-1040 Tel: 510-643-7349 Email: mwelsh@uclink4.berkeley.edu ASU William E. Blanton, Ph.D., Professor of Teaching and Learning University of Miami, School of Education Merrick 324-A 5202 University Drive Coral Gables, FL 33146 Tel: 305-284-5053 Email: blantonw@miami.edu CSUSM Miriam W. Schustack, Ph.D., Professor Department of Psychology California State University 320 University Hall San Marcos, CA 92096-0001 Tel: 760-750-4095 Email: mschusta@csusm.edu UCSB Richard E. Mayer, Ph.D., Professor Department of Psychology University of California, Santa Barbara Santa Barbara, CA 93106 Tel: 805-893-2472 Email: mayer@psych.ucsb.edu UCB Tamara Lynn Sturak, After School and Community Coordinator UC Links Program School of Education University of California at Berkeley 615 C. University Hall Berkeley, CA 94720-1670 Tel: 510-643-7349 Email: tamara@uclink.berkeley.edu
Program	UC Links UC Links Statewide Office 615C University Hall Berkeley, CA 94720-1040 Tel: 510-643-7349 Email: uclinks@socrates.berkeley.edu ASU Walter P. Oldendorf, Ph.D., Coordinator Reich College of Education Appalachian State University Edwin Duncan Hall: EDH 04 730 Rivers Street Boone, NC 28608 Tel: 828-262-7279 Email: oldendrfwp@appstate.edu CSUSM Miriam W. Schustack, Ph.D., Professor Department of Psychology California State University 320 University Hall San Marcos, CA 92096-0001 Tel: 760-750-4095 Email: mschusta@csusm.edu UCSB Lupe Arteaga, Club Coordinator Club Proteo 5701 Hollister Avenue Goleta, CA 93117 Tel: 805-967-1612 Email: marteaga@education.ucsb.edu UCB Tamara Lynn Sturak, After School and Community Coordinator UC Links Program School of Education University of California at Berkeley 615 C. University Hall Berkeley, CA 94720-1670 Tel: 510-643-7349 Email: Tamara@uclink.berkeley.edu
Profile Updated	May 20, 2003

Evaluation 3 (UCSB): Cognitive Consequences of Participation in a “Fifth Dimension” After School Computer Club

Evaluation Description

Evaluation Purpose	To answer the following research questions: (1) Do students who participate in the Fifth Dimension show greater improvement than nonparticipants in their comprehension of arithmetic word problems? (2) Do the benefits of participation in the Fifth Dimension persist over time? (3) Do the benefits of the Fifth Dimension remain for alternative methods of comparison? (4) Do the benefits of the Fifth Dimension persist over time for alternative methods of comparison? Each of the four research questions was handled as a separate “study” of arithmetic word problem improvements, which are, respectively: (1) a matched-by-student characteristics study, (2) a matched-by-student characteristics study of long-term gains, (3) a matched-by-pretest scores study, and (4) a matched-by-pretest scores study of long-term gains.
Evaluation Design	Quasi-Experimental: Approximately 120 students were tested, 16 were treatment students, and 16 were selected to constitute a comparison group. The treatment group consisted of third and fourth grade students who attended Club Proteo, a Fifth Dimension Program run by University of California at Santa Barbara faculty with the Goleta Boys and Girls Club, at least 10 times during the 1994–1995 school year. The comparison group consisted of the treatment group's classmates who did not attend Club Proteo and who were matched to the treatment group for grade and English language proficiency school district test scores. In addition, the comparison and treatment groups were matched for schoolteacher and gender in the matched-by-characteristic studies and for pretest scores in the matched-by-pretest studies.
Data Collection Methods	Tests/Assessments: The achievement assessment tools were pretest and posttest versions of a mathematics comprehension test called the Word Problem Comprehension Test (WPCT), provided in English and Spanish. The WPCT consisted of 12 problems broken down into three types, with four questions each. The first type presented a sentence from a word problem and asked the student to determine which equation corresponded to it. The second type presented a word problem and asked the student to determine which numbers were needed to solve the problem. The third type presented a word problem and asked the student to determine which arithmetic operations were needed to solve the problem. Tests were scored by tallying the number of correct answers selected by each student. A bilingual teacher administered the tests in both in English and Spanish in the children's regular classroom during the school day. The majority of the students in the study had learned Spanish as their first language. Spanish-speaking students who did not know English well were encouraged to complete as much of the English forms of the tests as possible. Similarly, English-speaking students who did not know Spanish well were encouraged to complete as much of the Spanish forms of the tests as possible. In the matched-by-characteristics studies for the English and Spanish WPCT, 16 treatment students and 16 comparison students took the English and Spanish WPCT as a pretest and posttest. In the matched-by-characteristics study of long-term gains on the English WPCT, 11 treatment students and 11 comparison students took the English WPCT as a pretest, posttest, and delayed posttest. These treatment and comparison students are a subset of the original group of 16 treatment students and 16 comparison students; five of the original treatment students were not among those tested in fall 1995 so they and their five matched comparison students were excluded from analysis. A similar analysis could not be conducted for the Spanish form of the WPCT because half of the students declined to take the test in Spanish in fall 1995. In the matched-by-pretest study of the English WPCT, 15 treatment students and 45 comparison students took the English WPCT as a pretest and posttest. In the matched-by-pretest study on the Spanish WPCT, 14 treatment students and 57 comparison students took the Spanish WPCT as a pretest and posttest. In the matched-by-pretest study of long-term effects, the participants were 12 treatment students and 36 comparison students who took the English WPCT as a pretest, posttest, and delayed posttest.
Data Collection Timeframe	The first test was administered at the beginning of the program in fall 1994 (the pretest). The testing was repeated in spring 1995, after treatment students had attended the club at least 10 times, using the posttest versions of each test (the posttest). The testing was again repeated in fall 1996 using the pretest versions of each test (the delayed pretest).

Findings:
Summative/Outcome Findings

Academic

Findings of the matched-by-characteristics studies for the English and Spanish tests indicate that Club Proteo participants scored significantly higher than the comparison group during the course of the academic year. The pretest-to-posttest gain was greater for the treatment group than for the control group on the English WPCT, although the difference was only marginally significant, t(15)=1.68, p<.06. The pretest-to-posttest gain was also greater for the treatment group than for the control group on the Spanish test, t(15)=2.85, p<.01. The pretest-to-posttest gain was greater for the treatment group than for the control group on the combined scores for both English and Spanish versions, t(15)=3.19, p<.01.

In the matched-by-characteristics study of long-term gains on the English WPCT, results provide evidence that the cognitive benefits of participation in the club persist beyond the end of the school year. The groups did not differ significantly on the pretest, t(10)=.22, p=ns and the treatment group outperformed the comparison group on the posttest, t(10)=2.23, p<.025. This difference was still present on the delayed posttest, t(10)=2.36, p<.025.

The matched-by-pretest study results indicate that the matched-by-characteristic study finding that the treatment group had significantly higher test scores over the course of the academic year than the comparison group holds up under an alternative method of comparison. An analysis of variance conducted on the Spanish and English posttest scores with group and pretest score as factors revealed a significant effect (albeit marginally for the English form) for group in which the mean posttest score of the treatment group (M=4.9 for the English form, M=3.8 for the Spanish form) was greater than the mean posttest score of the comparison group (M=3.6), F(1,52)=3.81, MSE= 0.9, p<.06 for the English form, and (M=2.1), F(1,63)=5.98, MSE=3.53, p<.05 for the Spanish form. In addition, the pattern of superiority of the treatment group was consistent across all pretest scores as is indicated by the lack of a significant group x pretest interaction, F(3,52)=1.321, MSE=3.790, p=Ns for the English form and F(3,63)=1.77, MSE=3.53, p=Ns Finally, the ANOVA revealed an effect for pretest score in which posttest scores were greater for students who scored higher on the pretest, F(3,52)=2.87, MSE=8.23 p<.05 for the English form, and F(3,63)=3.11, MSE=3.53, p<.05 for the Spanish form.

The matched-by-pretest study of long-term gain results indicate that the matched-by-characteristic study of long-term gains finding that the treatment group had significantly higher test scores that persisted beyond the end of the school year than the comparison group holds up under an alternative method of comparison. Of students who took the pretest, posttest, and delayed posttest for the English form of WPCT, the treatment group performed better than the comparison group on both the posttest and on the delayed posttest. This observation was examined via an ANOVA on the English WPCT scores with group (treatment vs. comparison) and pretest score as between subject factors and time of test (posttest vs. delayed posttest) as a within subjects factor. Consistent with this observation, there was a main effect for group in which the treatment group outscored the comparison group across both the posttest and the delayed posttest, F(1,40)=7.51, MSE=3.53, p<.01; also consistent with this observation, there was no significant interaction between group and time of test, F(1,40)<1, indicating that the superiority of treatment over comparison group did not differ between posttest and delayed posttest. This finding suggests that the benefits of Fifth Dimension participation persist over the summer, such that the superiority of the treatment group over the comparison group can be seen at both the posttest and the delayed posttest.

A Profile of the Evaluation of Fifth Dimension/University-Community Links

Profile Information