You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.

The Harvard Family Research Project separated from the Harvard Graduate School of Education to become the Global Family Research Project as of January 1, 2017. It is no longer affiliated with Harvard University.

Terms of Use ▼

Program Description

Overview Girls Creating Games (GCG) was an afterschool program for middle school girls in Capitola, California. It aimed to build participants' interest, skills, fluency, and confidence in information technology (IT).
Start Date 2003 (evolved into the Girl Game Company which ran from 2004–2007, and then into an ongoing community-wide effort called Watsonville TEC)
Scope local
Type afterschool, summer
Location urban
Setting public school and community-based organization
Participants middle school
Number of Sites/Grantees 1
Number Served 126 girls (2003–2004)
Components GCG took place after school 2 days per week during the school year and 4 days per week over the summer. Activities were organized into four concurrent strands designed to link activities to expected outcomes. In the first strand, Learning by Design, girls designed and created a computer game using Macromedia’s Flash MX software; they were encouraged to create games to help incoming youth adjust to middle school. In the second strand, Scaffolding and Modeling, instructors supported the development of conceptual understanding of IT by providing girls with the resources to create games and solve problems independently. The third strand, Collaborative Learning, involved activities that built a community of learners, such as having girls work in pairs to design and program a game. Instructors supported the development of effective relationships within and across pairs by modeling behaviors and leading fun activities designed to strengthen communication and mutual decision-making. In the final strand, Identity Formation, girls explored IT careers, interacted with female role models who challenged gender stereotypes, and received public recognition to promote a “tech savvy” identity.
Funding Level $800,000 to develop, implement, evaluate, and disseminate materials
Funding Sources The National Science Foundation’s Program on Research on Gender in Science and Engineering


Overview The evaluation examined GCG’s effects on participants and aimed to identify ways to strengthen the program in future implementations.
Evaluator Jill Denner, ETR Associates
Evaluations Profiled An Innovative Approach to Integrating Technology into Middle School
Evaluations Planned None.
Report Availability Denner, J. (2007). The Girls Creating Games Program: An innovative approach to integrating technology into middle school. Meridian: a Middle School Computer Technologies Journal, 1(10). Available at:



Steve Bean, M.A.
ETR Associates
Senior Program Manager
4 Carbonero Way
Scotts Valley, CA 95066
Tel: 831-438-4060

Evaluation Jill Denner, PhD
Senior Research Associate
ETR Associates
4 Carbonero Way

Scotts Valley, CA 95066 

Tel: 831-438-4060

Profile Updated January 10, 2011

Evaluation: An Innovative Approach to Integrating Technology into Middle School

Evaluation Description

Evaluation Purpose To assess whether GCG participants increased their capacity to pursue and persist with computer technology, and which aspects of GCG could be improved.
Evaluation Design

Quasi-Experimental and Non-Experimental: A sample of 90 GCG participants and a comparison group of 71 girls of the same age from different schools who were either in, or wanted to be in, an afterschool program involving computers, completed pretest and posttest surveys. Attrition varied between groups: Of those who completed pretest surveys (126 in the treatment group and 88 in the comparison group), 28% of the treatment group and 17% of the comparison group did not complete the posttest survey, and thus were excluded from the analysis. Data from the comparison group indicated that those with no computer at home were less likely to complete a posttest survey than those who had a computer at home. In the treatment group, those who did not complete the posttest survey reported significantly higher levels of knowledge about computers at pretest than those who completed the posttest survey (p < .05).

The two groups differed in their race/ethnicity and on their average pretest scores on three variables. Specifically, the percentage of White students was higher in the treatment group than the comparison group (60% vs. 36%). In contrast to the comparison group, the program group reported significantly lower levels of skill (p < .01), more positive attitudes toward computers (p < .01), and lower frequency of teachers thinking that the girls knew a lot about computers (p < .05).

Two sources of qualitative data were also collected from GCG participants: interviews with 31 participants (who were selected to represent the range of grade levels, computer expertise, and race/ethnicity of the larger group) and electronic notebooks completed by participants near the end of the program (most GCG participants responded to questions in pairs, for a total of 81 responses).

Data Collection Methods

Document Review: Electronic notebooks were used to assess GCG participants’ satisfaction.

Interviews/Focus Groups: Interviews included such questions as, “Do you think you have changed at all from being in GCG?” and, “Did you do anything in the program that surprised you or that you hadn’t done before?”

Secondary Source/Data Review: For youth in the GCG group who participated after school, race/ethnicity was obtained from parent reports in school records.

Surveys/Questionnaires: Surveys measured subjective task value, expectations for success, and social support. The comparison group and girls in the GCG group who participated during the summer also reported their race/ethnicity on the survey.

Test/Assessments: Surveys included a number of instruments to measure subjective task value, expectations for success, and social support.

Subjective task value was measured using three scales:

  1. Stereotypes about Computer Workers—Seven items adapted from Lawhead, Wilkins, & Rheningas (n.d.) that describe whether respondents agree with various stereotypes about people who work with computers (e.g., “Is creative,” or “Does not have time for their family”), were rated on a 5-point scale from strongly disagree (1) to strongly agree (5).
  2. Intentions to Study Computers—Four items created by the research team that asked about participants’ plans to take courses like computer graphics and computer programming were rated on a 5-point scale from definitely not (1) to definitely yes (5).
  3. Attitudes Toward Computers—Seven items adapted from Todman & Dick (1993), such as “I like using computers in my free time,” were rated on a 5-point scale from strongly disagree (1) to strongly agree (5).

Expectations for success were measured with the following five scales:

  1. Confidence with Computers—Nine items derived from Levine & Donitsa-Schmidt (1998) and Todman & Dick (1993), such as “I find using the computer easy,” were rated on a 5-point scale from strongly disagree (1) to strongly agree (5).
  2. Computer Skills—Eight items, some derived from Rockman et al. (n.d), such as “Burn a CD,” were rated on a 5-point scale from “I don’t know what this means” (1) to “I can do this so well that I can teach someone how to do it” (5).
  3. Knowledge about Computers—Four items, some derived from Rockman et al. (n.d), assessing what participants’ teachers, other youth, their families, and they themselves thought they knew about computers, were rated on a 4-point scale from nothing (1) to a lot (4).
  4. Problem Solving—One item created by the research team, “If I don’t know how to do something on the computer at school, the first thing I do is….”, had answer options including “try to figure it out myself,” “ask for help,” and “give up.”
  5. Gender Stereotypes—One item adapted from Levine & Donitsa-Schmidt (1998), “Boys my age usually do better than girls when using computers,” was rated on a 5-point scale from strongly disagree (1) to strongly agree (5).

Social support measures were adapted from Mappen (n.d), in which girls were asked to report what percentage of their male and female friends were interested in computers, rated on a 5-point scale from none (1) to all (5); how often their closest friends used computers, rated on a 5-point scale from every day (1) to never (5); and whom they talked to the most about computers in the last 3 months.


Lawhead, P., Wilkins, D. & Rheningas, P. (n.d.). Student survey for girls in science and technology. Oxford, MS: University of Mississippi. Available at:

Levine, T., & Donitsa-Schmidt, S. (1998). Computer use, confidence, attitudes, and knowledge: A causal analysis. Computers in Human Behavior, 14, 125–146.

Mappen, E. F. (n.d.). The Douglass Science Institute Program Series: Encouraging precollege women to persist in math and science studies. New Brunswick, NJ: Rutgers University. Available at:

Rockman et al. (n.d.). Students as agents of change: Transforming the teaching/learning process through technology and African and African-American history and culture. Gary, IN: Gary Community School Corporation. Available at:

Todman, J. & Dick, G. (1993). Primary children and teachers' attitudes to computers. Computers and Education, 20, 199–203.

Data Collection Timeframe Data were collected from January 2003 to June 2004.

Formative/Process Findings

Satisfaction Electronic notebooks contained 130 comments about what participants liked, and 74 comments about what they disliked. What they liked best was using computers (e.g., “It is fun how we get to create games and then play the games that someone else made and to get ideas”) and having mastery experiences (e.g., “I love the feeling of accomplishing something”). What participants disliked the most was the amount of direct instruction (e.g., “…we have to do a lot of work to make the games and when they explain the directions and how to do things it’s really boring listening to them talk”) and having to work with partners (e.g., “…my partner…didn’t really like any of my ideas, but she didn’t come up with any of her own...”).

Summative/Outcome Findings

Youth Development

Computer skill level increased significantly among participants from pretest to posttest (p < .001), and the increase was significantly greater for the program than the comparison group, (p < .001). As one girl described her skills, “I know how to program now and I know how to go on the internet (like Google) and get the graphics and computer animation, like the time frame. And I also know how to put action scripting on a button.”

The program group reported virtually no change in stereotypes about computer workers, while the comparison group reported an increase in negative stereotypes; this difference between the two groups’ pretest to posttest change was significant (p = .05). The following interview quote illustrates participants’ perspectives of GCG’s role in influencing their stereotypes: “Whenever I would think of a computer I would think of a nerd with glasses… since I’ve done this, [I see that] there are all different types of girls here and none of them were nerdy.”

There was no significant change within or across groups in girls’ intentions to take computer courses in the future or their attitudes toward computers. However, in response to the interview question about their plans to take more computer classes, participants’ responses suggested a desire to do so, for example: “I really now understand how important it is because I didn't know any of that stuff and now I know what I want to be; I want to be a computer animator.”

There was an increase in participants’ confidence in using computers from pretest to posttest, which approached significance, but did not significantly differ from non-participants.

The program group reported significant increases from pretest to posttest in what their family (p < .01), they themselves (p < .001), and what other youth (p < .05) thought they knew about computers. The program group’s change in self-knowledge was significantly greater than that reported by the comparison group (p < .05). As one participant put it, “Yes, [I changed] a little because I could teach other people about computers and my mom and my dad and my brother and my cousin and my whole entire family want to learn more about the computer, but I want to learn too.”

A significant difference emerged between the two groups in their view that boys usually do better than girls when using computers. From pretest to posttest, the program group significantly decreased their gender stereotypes in this area (p < .01), while the comparison group increased theirs (p < .05). The difference in changes between program and comparison group was significant (p < .01). However, there was no significant change in ratings for the following items: “Most girls my age find it easy to learn to use computers, “Most boys my age find it easy to learn to use computers,” and “Girls my age usually do better than boys when using computers.”

In response to an item stating, "If I don't know, the first thing I do is…" youth in GCG reported greater pretest to posttest increases in trying to figure it out themselves (from 38% to 61%) compared to those in the comparison group (from 39% to 50%). However, the change did not significantly differ between the program and the comparison group. According to one GCG participant, “Now I don't… hit the computer when I'm mad—I try and find out what's wrong.”

Controlling for pretest ratings, significantly fewer girls in the program than in the comparison group reported at posttest that “There is no one I talk to about computers,” (p < .05, 2% vs. 10%).

© 2016 Presidents and Fellows of Harvard College
Published by Harvard Family Research Project