April 26, 2005
Course: CEP 910 Motivation & Learning
Instructor: Jere Brophy, PhD
Michigan State University; East Lansing, Michigan, USA
a dissertation proposal-in-progress
[I decided to pursue a different dissertation. -Kym, 2005/09/04]
Kym Buchanan (Email: buchan56 AT msu.edu)
Home page: KymBuchanan.org
Appendix: Instruments
It's possible to design and develop a digital game for teaching introductory educational psychology, with limited resources. The game can be enjoyable and challenging, while helping students learn the content.
This proposition will be tested by developing and integrating a digital game in TE 150 Reflections on Learning, an undergraduate course in educational psychology for pre-service teachers. The working title of the game is Extrapolation & Adaptation: A Motivation Simulation.
The game will be piloted in TE 150 in Summer 2005. A full study will occur in Fall 2005.
This study links several areas of inquiry: educational psychology, educational technology, and game design.
This study has a dual interest in educational psychology: teaching using digital games is a promising but enigmatic instructional approach, and the curriculum for this particular study is introductory educational psychology.
Educational psychology is a complex domain because human minds are complex. Research and discovery are ongoing over the breadth and depth of the domain. Educators generally agree that pre-service teachers can benefit from studying theoretical and applied educational psychology. But there is disagreement over appropriate curricula and instructional strategies for pre-service courses like TE 150 (cf. Johnson & Morgan, 2003). To navigate that tension and guide design, this study focuses on several sub-areas of educational psychology: motivation, authentic tasks, problem solving, and complexity.
Motivation is a compelling reason to teach using digital games. Games are a "catch" factor for intriguing learners (Mitchell, 1993; Prensky, 2001). If learners play recreational games outside of courses, teaching using games may connect with learners' personal interest. Even in the absence of personal interest, games may foster situational interest: "temporary interest that arises spontaneously due to environmental factors such as task instructions or an engaging text" (Schraw, Flowerday, & Lehman, 2001, p. 211). Here, "texts" are conceived in the broad, post-structuralist sense. Gee (2003), Spiro and Jehng (1990), and others rightly assert that interactive multimedia like digital games are especially interesting to users (and educational) because they are "writerly" texts (cf. Barthes, 1968/1977; Barthes, 1971/1977). Such texts allow users/learners to co-author their own experience with the content.
But as Mitchell (2003), Brophy, and others argue, "...hands-on activities [like games] will not produce important learning unless they include minds-on features that engage students in thinking about powerful ideas..." (Brophy, 2004, p. 241). After catching learners' interest, activities can hold that interest by "finding variables that empower students" (Mitchell, 2002, p. 426). One promising model for promoting such higher-level engagement is teaching using authentic tasks. Authentic tasks can compel learners to "use their minds and bodies as they would if they were practitioners in a domain" (Honebein, 1996, p. 20) (cf. Dede, 1996; Aldrich, 2004), in a context that simulates (but not necessarily replicates) the domain's real environment. Pre-service teachers can practice integrate ideas from educational psychology into their teaching before teaching students in a classroom. A digital game can create the necessary authentic environment by challenging learners with teaching problems in an interactive context, thus allowing them to by practice thinking and acting like a teacher.
Many educators and institutions are continually examining their relationship with educational technology, especially computers. Some instructors try to integrate new technologies and experience mixed results, while others don't even try. Some technologies may be viewed too intimidating to learn or use, incompatible with instructional goals, or not worth the trouble (Scott, Cole, & Engel, 1992).
Teaching using digital games has already gone through one cycle of fad and backlash: "edutainment," in the 1980s. Apparently there are few successful models of design and course/classroom integration, as there is little research to guide designers and educators (Gredler, 1996; Mitchell & Savill-Smith, 2004). Over the past few years, there has been a resurgence of interest, fueled by significant people and organizations: Henry Jenkins, Jim Gee, the Serious Games Initiative at the Woodrow Wilson Center, the Education Arcade at MIT, and others (Gee, 2003; Gredler, 1996; Dede, 1996; Prensky, 2001; Aldrich, 2004; Elliot, Adams, & Bruckman, 2002). There is also growing mainstream interest in the possible positive and negative effects of games, especially issues of violence, gender, and health and fitness (Video Game Revolution, 2004).
If digital games really have deep educational potential (and I believe they do), design is the great obstacle. Games can clearly entertain, but evidence of their educational utility is scant and mixed (Gredler, 1996; Mitchell & Savill-Smith, 2004). This study must offer a model for integration in courses/classrooms, and it must demonstrate innovative design.
The process of designing and developing an educational game must foreground and mitigate various pressures, in order to avoid distorting the content, marring the gameplay, or otherwise compromising the overall value of the game as a tool for teaching and learning. The content of introductory educational psychology is complex, and thus challenging to teach and learn.
For any content, it's difficult to develop a game which:
Digital game design is complex, difficult, and still emerging as discipline. Some designers and scholars have suggested theories and models for design (Crawford, 2003; Rollings & Adams, 2003), but there are few guidelines for designing games for teaching and learning. Contemporary design is strongly influenced by commercially-successful recreational games and game genres. One of the goals of this study is explicating the process of design, as others have done with their work (Prensky, 2001; Aldrich, 2004; Costikyan, 1994). This study may need to explore new genres, i.e., new territory in the possibility space of game design. The best way to explore new genres for teaching using games is to create and test a game in a real course.
I mediate these three areas of inquiry using my model of co-opting. Co-opting is a strategy for motivation and learning.
Interest promotes learning; apathy stunts it. Retention and recall are usually colored by emotional and sensory experiences (e.g., anxiety, symbols). Knowledge is seldom purely abstract or neutral-affect. Learning is influenced by contexts, including time, place, models, and social relationships (e.g., class time, classroom, diagrams, other people). Therefore, studying a variety of cases and contexts fosters broad and flexible learning.
Ideally, learning is a vivid interaction between a learner and a compelling context. Engagement starts with "catch" factors and persists through "hold" factors (Mitchell, 1993). Learning is the interaction of two unique, dynamic systems: the learner and the context. Educators can do little to influence learners background. So to improve learning, educators should evaluate and modify immediate contexts. For example, better tools foster more vivid and compelling interactions: the system is the solution.
Technology systems can model, preserve, and/or foster the application of knowledge. Many learners are engaged by recreational technologies and media (e.g., games, movies, music). Co-opting is a strategy of identifying and re-purposing such novelties. Co-opting tries to preserve the catch and hold factors of the novelties (e.g., gameplay), while fostering learning of more academic content (e.g., applied educational psychology). A re-purposed novelty can change a learner's interaction with the context(s) of a course.
What are the effects of teaching using a digital game, in an introductory educational psychology course?
1. Does playing the game improve student learning?
2. Are the students engaged?
3. Are the students satisfied?
4. Is the instructor satisfied?
5. How should a game be presented/packaged for teaching (e.g., instructor's guide)?
6. How can a game be both enjoyable and educational?
7. How can this tension be resolved: (a) design a compelling, educational game; and, (b) develop and test it in a relatively short time with limited resources?
8. Which algorithms and heuristics (e.g., models) are necessary and appropriate to design and develop optimal games for teaching?(7 and 8 are complementary questions.)
This study serves and studies instructors and students in TE 150 courses at Michigan State University. TE 150 is a lecture-based course, and includes in-class activities, discussions, and videos. Individually, students complete two short analysis papers. In small groups, students complete one long synthesis paper and presentation. Students take one cumulative, multiple-choice final exam. Students are encouraged to use several study tools to prepare for the final exam, including two web-based practice exams.
TE 150 is a foundation course for Michigan State's Department of Teacher Education, as part of a highly-respected and much-scrutinized pre-service program. Enrollment is not restricted to students planning to continue in TE; some students may take TE 150 as an elective, may change their major, or may not be accepted to the pre-service program. As members of the university community, the students have free access to numerous computers around campus (and at least some have personal computers).
TE 150 is primarily taught by graduate students in the College of Education. There may be 30 sections per semester. Curriculum is aligned across sections. Instructors have some freedom to modify the curriculum, especially lectures and activities. The curriculum is built around Jeanne E. Ormrod's Educational Psychology: Developing Learners (4th ed.) (2003), including textbook-related materials (study guide, case studies, and artifact case studies). The new, fifth edition of the textbook will be used in Fall 2005.
Dr. Linda Anderson, the supervising professor for TE 150, has agreed to cooperate with this study. She has a few stipulations. Only experienced instructors will be allowed to participate, i.e., instructors who have taught at least one semester of TE 150. Instructors won't compelled to participate; they must be recruited individually. (Note that this improves external validity: instructors will be integrating the game into a established course/curriculum.) The game must primarily be an out-of-class activity. (Note that this makes the game a relatively low-threshold opportunity for instructors: there is no need to schedule computer lab time.)
All appropriate procedures will be followed to respect and protect the population, including building consensus with Dr. Anderson, UCRIHS oversight, and protection of subjects' privacy.
Instructors who agree to participate will be asked to give consent for their data (e.g., pre- and post-treatment questionnaire results). Students in their respective sections will be required to play the game as an assignment. Students will be asked to give consent for their data (e.g., pre- and post- questionnaire results, game usage, itemized final exam results). In other words, students may play the game as a course assignment without fully participating in the study.
This study will use two-group, control/treatment design. The only design difference between the control and treatment conditions is access to the game, as an assignment and then as a study tool. Both quantitative and qualitative measures will be used, including usage data, performance on the final exam, questionnaires, and interviews.
All experienced TE 150 instructors will be asked to participate. Instructors will have the right to stop participating at any time. I expect the majority of instructors to participate, because this study is minimally intrusive and the game will be an attractive instructional tool. The pool of participating instructors will be randomly divided in control and treatment conditions.
Within the sections of participating instructors, all students will be asked to participate. Students will have the right to stop participating at any time. I expect the majority of students to participate, because they are studying educational psychology and will be interested in this study. However, students will have access to the game as an assignment and then as a study tool, regardless of whether they agree to participate. Regardless of individual participation, the mean scores on the final exam, by section, will be compared.
Participating instructors will have to answer one or two questionnaires (pre-control, or pre-treatment and post-treatment). Instructors in the treatment condition will have to play the game at least once, assign the game to their students, and transcribe the resulting participation points into their gradebooks. Participating students will have to answer two questionnaires (pre-control and post-control, or pre-treatment or post-control). A subset of instructors and students in the treatment condition will be asked to participate in short, individual interviews (one interview, 10-15 minutes), at the end of the study (i.e., just after the final exam).
Based on the results of this study, all instructors will probably have access to the game in subsequent semesters, but no data will be collected. However, this means that one incentive for participating is improving the game as an instructional tool.
The game will focus on the TE 150 unit on motivation (chapters 11 and 12 in the Ormrod textbook). The focus of the game will be review and application of big ideas, not the introduction of new content. The game will be presented to students as a study tool, similar to the Reading Checks and practice final exams. There are two Reading Checks for the unit, one for each chapter; the Reading Checks are assignments. The practice final exams are not assigned, but students are strongly urged to use them to prepare for the final exam. As an instructional tool, the game will be a hybrid: students will be required to play the game at least once as an assignment, but subsequently they will have unlimited access to the game as a study tool.
The questions on the final exam can be cross-indexed to specific chapters. Hence, the respective final exam items will be the primary measurement of the game's influence on learning. This improves external validity, because the exam is an source of "found" or "in situ" data, not an extra or artificial measurement.
The process of designing the game and conducting this study is dictated by many givens and constraints.
Givens
The game needs to be a study tool, similar to the reading checks. The reading checks are required online self-tests: one for each chapter covered in TE 150. Students must take each reading check, but they can try an unlimited number of times before the due date; each student's most recent score is recorded in the gradebook. The reading checks are ten questions, usually randomly drawn from a larger pool. The reading checks are designed to take less than 10 minutes for a student who has read the chapter, but the student can take up to 30 minutes. To be similar to the reading checks, the game will be playable in a single, short session of 10-15 minutes. It will be replayable.
The reading checks are Web-based (via ANGEL), which allows instructors to assign the reading checks as homework assignments. Likewise, the game will be played outside of class, via the Web. The system requirements will be low, because some students may want or need to play the game in the university's computer labs.
Students receive immediate feedback on their reading checks, including which questions they miss. Their scores are automatically recorded in the gradebook (which is another feature of ANGEL). The game will give immediate feedback to students, and will also save usage data. Instructors will be able to access this data via a simple Web interface, to transcribe into their gradebooks.
The game will be designed such that demonstrating skill with the content is necessary and sufficient for in-game success. In other words, the student can only succeed by demonstrating skill with the content (necessary), and the student can succeed simply by demonstrating such skill, and with or without proficiency in "gaming" skills (reflexes, pattern recognition, etc.) (sufficient).
The game will honor the nature and complexity of the content (applying educational psychology as a teacher). It will try to foreclose on overly-simplistic or otherwise-flawed beliefs about teaching and learning.
The game will be "packaged" and presented to make it appealing to instructors and students, and reasonably easy for instructors to integrate.
Constraints
The game will be developed with limited resources (primarily developed by the researcher, with no budget, in a limited time). This is not a prohibitive obstacle. The process of developing a game (or any instructional tool) is always constrained by resources, like the budget and the developer or developers' time and skills. What matters is how the designer/process addresses such constraints
As mentioned above, instructors must voluntarily adopt game. Therefore, the game must be a comprehensible and appealing tool. It must be presented (or "sold") as worth the relative hassle of integration.
A final constraint is satisfying UCRIHS institutional oversight.
This timeline illustrates the major protocols for this study, including differences in the treatment condition.
| Calendar | Control | Treatment* |
|---|---|---|
| Weeks 1-11 | Standard curriculum on earlier topics | Standard curriculum on earlier topics |
| Week 9 | Instructor Pre-Control Questionnaire | Instructor Pre-Treatment Questionnaire |
| Last class of Week 11 | Student Pre-Control Questionnaire | Student Pre-Treatment Questionnaire |
| Weeks 12-13: Motivation | Standard curriculum | Standard curriculum + Playing the game as homework assignment |
| Weeks 14-15 | Group presentations; studying for final exam | Group presentations; studying for final exam + Continued access to the game |
| Week 16: Finals week | Studying for final exam; final exam |
Final exam + Continued access to the game |
| After final |
* The game will also automatically gather usage data, indexed by student.
I will collect and analyze many initial descriptive statistics about the population, including demographic ("What is your gender?") and subjective data ("How well do you think you will do on the final exam?"). This data will help me identify any systematic background differences in the groups.
I will look for correlations in the initial descriptive statistics, e.g., "What is your gender?" v. "Do you like playing computer/video games?" (1-way ANOVA).
After the study, I will look for correlations in the subjective data, e.g., Treatment v. "How do you feel about the course/curriculum of TE 150?" (1-way ANOVA). I will also look for interaction effects, e.g., Treatment v. "Do you like playing computer/video games?" v. "How do you feel about the specific content about motivation?" (2-way ANOVA).
(Ed. note: I will compile a complete list of planned correlational analyses before submitting this proposal, to avoid what Bill Schmidt calls "dust bowl empiricism.")
Perhaps most importantly, I will look for correlations between Treatment and performance on the final exam (1-way ANOVA).
This table shows the match between the research questions, data, and expected results. All expected results refer to the treatment condition, wherein greater learning, engagement, and satisfaction are predicted.
| Query | Data | Expected results |
|---|---|---|
| 1. Does playing the game improve student learning? |
Items from final exam | More correct individually and as a section |
| Student Post-Treatment Questionnaire | High self-reported learning | |
| 2. Are the students engaged? | Usage data | Usage beyond homework assignment |
| Student Pre- vs. Post- Questionnaires | More interest in content | |
| More time spent studying for final exam | ||
| More liking for course | ||
| Student Post-Treatment Interviews | High self-reported engagement | |
| 3. Are the students satisfied? | Student Post-Treatment Questionnaire | High self-reported satisfaction |
| Student Pre- vs. Post- Questionnaires | More confidence about performance on final exam | |
| Student Post-Treatment Interviews | High self-reported satisfaction | |
| 4. Is the instructor satisfied? | Instructor Post-Treatment Questionnaire | High self-reported satisfaction |
| Instructor Post-Treatment Interviews | High self-reported satisfaction |
This study really isn't about the game. It's about disentangling processes. This study will tell the story of of developing and using an instructional tool. There are several processes involved, including the process of:
This study will be both paradigmatic and narrative. In the field of teaching using digital games, there is a need for accessible models and successful cases. Models are paradigmatic, and this study uses my model of co-opting. Cases are stories to emulate. This study will offer a detailed story of TE 150, including the goals of the course, the attitudes and experiences of the instructors and students, and the complications of using a new tool. Such narrative modes of knowing "are specific, local, personal, and contextualized. We do not speak of the validity of a narrative, but of its verisimilitude. Does it ring true? Is it a compelling and persuasive story" (Shulman, 1992, p. 22)? If this study finds greater learning with the game, it will be a case that others should emulate. The detailed story will show others how to emulate it.
This study will try to create an enjoyable, challenging game. I say "enjoyable" rather than "fun," because fun is a more nebulous and highly subjective idea. Enjoyment and challenge are necessary conditions for flow (Csikszentmihalyi, 1990). A flow experience is not necessarily fun. Learning can be enjoyable and challenging without meeting a learner's criteria for fun.
Assor, Kaplan, and Roth assert that choice and relevance can be highly motivating to students (2002). The game offers both. Choice is the essence of interactivity and gameplay; acclaimed designer Sid Meier describes good game design as "a series of interesting choices" (in Rolling & Adams, 2004, p. 200). For students preparing to become teachers, a simulation of teaching will be very relevant.
The students in TE 150 will benefit from exposure to the game. According to many experts (cf. Gee, 2003; Prensky, 2001; Aldrich, 2004), teaching using games is a major opportunity and challenge in education's near future.
Finally, design can be focusing lens for education reform. The process of integrating a new tool (like a digital game) can foreground and interrogate tensions among instructional and institutional goals, theory and content, practical constraints, and day-to-day instructional practice. This process is an attempt to identify and optimally respond to various pressures, as illustrated below.
Created by Kym Buchanan | http://KymBuchanan.org | This work is licensed under a Creative Commons License.
