Critique of Yoshida, M., Fernandez, C., & Stigler, J. W. (1993). Japanese and american students' differential recognition memory for teachers' statements during a mathematics lesson. Journal of Educational Psychology, 85(4), 610-617.
Students receive most instruction in traditional classrooms. In such classrooms, students are confronted with a variety of stimuli. In a specific lesson, these stimuli include relevant and irrelevant information. Yoshida, Fernandez, and Stigler believe that a student's ability to discriminate between these types of information positively correlates to learning, as reflected in pretest to posttest gains. In other words, if a student can successfully focus on processing only relevant information, then he will perform better on a posttest. Yoshida et al. try to prove this positive correlation in their 1993 article, " Japanese and American Students' Differential Recognition Memory for Teachers' Statements During a Mathematics Lesson." Yoshida et al.'s theoretical perspective is intriguing and potentially powerful, with its emphasis on discriminating between relevant and irrelevant information. Unfortunately, their research design and analysis is inappropriate and inadequate for validating their framework. Their discussion of results is also inadequate.
Yoshida, Fernandez, and Stigler offer a potentially powerful theoretical perspective. They believe that "students participating in a lesson must first construct a mental representation of the events that constitute the lesson." (p. 610) The purpose of this representation is to perform the kinds of tasks the lesson focused on. In other words, learning is constructing conceptual and procedural understanding. For example, consider a math class (as Yoshida et al. did). As the teacher demonstrates the steps to solve a problem, a student constructs a mental representation of the steps. The student then uses this representation to understand how such problems are solved, and to solve similar problems in the future. The quality of the student's representation is its functional effectiveness. According to Yoshida et al., the student will construct a superior representation if he uses ample relevant information and minimal irrelevant information. Irrelevant information reduces the functional effectiveness of the representation. For example, when trying to solve a problem, the student could be frustrated or distracted by a representation that includes irrelevant information.
There are two critical variables in a student's construction of a mental representation: the lesson itself, and the student's discriminatory ability. (Unfortunately Yoshida et al. confound these variables, as discussed below.) First, the lesson will vary in its mixture of relevant and irrelevant information. A lesson or mental representation is highly coherent if it includes much relevant information and little irrelevant information. Yoshida et al. believe that a student will construct a more coherent representation if he's exposed to a more coherent lesson. The second variable is a student's ability to discriminate between relevant and irrelevant information. The student will learn better if he successfully focuses only on relevant information and incorporates such information into his functional representation. Remembering irrelevant information will interfere with constructing a representation, since the student won't be as focused on the relevant information, and may incorporate irrelevant information into the representation.
Yoshida et al. theoretical framework can be illustrated by Diagram 1, which shows how a student uses what the teacher says in class to construct an internal representation for understanding and performance. Yoshida et al. believe that "on the basis of this representation of events, the students must construct the knowledge, both procedural and conceptual, that the teacher intends to convey." (p. 610) A student will construct a superior representation when given ample relevant information and minimal irrelevant information. Relevant information includes statements made by the teacher to guide the students in solving a problem. For example, a clear rationale and explanation for each step is relevant information: "Let's try subtracting the extra 6 children from the 38." (p. 617) Irrelevant information includes statements made by the teacher that offer no guidance in solving a problem. For example, trivial or distracting statements can be irrelevant: "Miriam, you are always answering." (p. 617) Yoshida et al. believe there is a positive correlation between the coherence of a lesson (e.g. what is said in class) and ability to perform (e.g. on a posttest). They also believe there is a positive correlation between a student's ability to discriminate relevant information and the student's ability to perform. The latter correlation is the focus of their study.
Diagram 1: Discrimination for Learning.
Yoshida et al. believe their variables are interrelated: if students are exposed to highly coherent lessons, they develop better discriminatory ability. Such students develop better processing strategies, and they are able to "either ignore irrelevant information or suppress it so as to make processing of the relevant information more efficient." (p. 616) In particular, Yoshida et al. cite evidence that "Japanese lessons, in contrast with American ones, have been characterized as easier to represent coherently by the learner because they are well organized and are typically designed to achieve a single, integrated instructional goal." (p. 610) Yoshida et al. argue that Japanese students should have better discriminatory ability, as a result of prior experience with highly coherent lessons. This cultural difference may have critical implications. Yoshida et al. justify their study by citing evidence that "Japanese students have repeatedly demonstrated superior mathematics achievement when compared with their American counterparts." (p. 610) Improving American students' math achievement is a worthy goal, so this study had high potential value. If Yoshida et al. could prove a positive correlation between discriminatory ability and math achievement, American educators might be able to improve their students' math achievement by improving their students' discriminatory ability. American educators might look to Japan for instructional models for improving information discrimination and lesson coherence.
Despite this ambitious perspective, Yoshida et al. fail to effectively tie their study to relevant theory and prior research. They do cite sufficient evidence for the cultural disparity in math achievement and in instructional practices. They cross-reference their other work on lesson coherence and constructing representations. However, they simply lament the paucity of existing research in classroom instruction and learning. This is bizarre, since a great deal of such research exists and may be relevant. Yet they make no links to other researchers' work on lesson coherence, constructing representations, memory and recall, or educational technology. The absence of citations in educational technology is an especially acute flaw, since this study centers on videotaped instruction.
Yoshida et al. are also somewhat unclear in their research question. They are clear in their belief that that Japanese students are better at discrimination because of their prior experience with highly coherent lessons. Thus, they believe that if they "hold lesson structure constant" among classes of American and Japanese students (by using allegedly-identical videotaped lessons), they will explicate the effect of the other variable: discriminatory ability. (p. 611) Their main hypothesis is that Japanese students have higher discriminatory ability, and consequently learn better than American students. More specifically, they hypothesize that Japanese students will score better on a recognition memory test, that such superior performance indicates a superior mental representation, and that such a superior representation correlates with greater improvement on a math achievement posttest (compared to American students' improvement). This hypothesis depends on a questionable assumption, that a recognition memory test will probe students' discriminatory ability. In other words, Yoshida et al. assume that the ability to remember information is indicative of the ability to discriminate between relevant and irrelevant information. They never explicitly state this assumption (or their hypothesis), so the reader must infer it from the introduction. This assumption is a critical flaw in their hypothesis and subsequent research design.
Yoshida et al.'s research design fails to adequately and appropriately probe the variable of discriminatory ability. They select three classes of students, administer a pretest, expose the students to a videotaped lesson, administer a posttest, and administer a recognition memory test. The memory test is the critical measurement, and it's a flawed approach. Yoshida et al. use it because they believe it will probe discriminatory ability. This is a circuitous approach to measuring this variable. In the process illustrated in Diagram 1, the ability to recall information is an off-shoot. Yoshida et al. argue that discriminating relevance is the critical ability, but they measure recall instead. It's plausible that a student with poor recognition memory could have high discriminatory and constructive ability, and thus demonstrate high performance on the posttest. Recognition memory ability is not the same as discriminatory ability.
Yoshida et al. could have probed discriminatory ability more directly. Perhaps they could have presented students with a random list of statements that were said, and asked students to classify each statement as relevant or irrelevant. (Instead, they ask students whether each statement was actually said in the lesson. Many were not.) Rather than focus on a single pretest to posttest gain, Yoshida et al. could have tried to correlate general academic performance with discriminatory ability. Standardized test scores or recent grades could be used for this correlational analysis.
Yoshida et al. believe they have a powerful opportunity in the expected disparity in discriminatory ability between Japanese and American students. Based on the theory and prior research they initially discuss, it makes sense to juxtapose Japanese and American students. Unfortunately, Yoshida et al. don't do this. Instead, they select participants from two American schools. The American students attend an American university laboratory school, which probably has a far-from-typical culture and curriculum. The "Japanese" students' school is also in America. While it's staffed and taught like a Japanese school, the students are residents of America. The populations of both schools are "mostly middle to upper-middle class." (p. 611) Neither school is very representational of its alleged national culture. Yoshida et al. included both 4th and 6th grade American students, to try to control for cultural differences. But two years of school and life experience may cause significantly different cultural perspectives. Also, as mentioned, the "Japanese" students are American residents, so they represent a culture neither purely American nor Japanese. All these irregularities strongly limit the potential generalizability of this study, despite Yoshida et al.'s later assertions of broad implications.
The study's procedures and materials are also problematic. In this study, Yoshida et al. hope to prove a correlation between discriminatory ability and math achievement. They try to isolate the variable of discriminating relevance, by assuming that Japanese students have superior discriminatory ability, and by allegedly using the same highly-coherent lesson for each population. Unfortunately, the lesson may be significantly less coherent for the American students. A 45 minute lesson was initially recorded in a Tokyo classroom with 38 children. Both the setting and class size are probably somewhat alien to both the American and "Japanese" students in this study. Both populations might also find the lesson less than coherent because Yoshida et al. reduced it to 15 minutes for the purposes of this study. For the American version, they substitute an American teacher into the tape, and this intruder performs a scripted, translated version of the lesson. The American teacher isn't in the Tokyo classroom. Student voices are dubbed into English. Any nuance of classroom flow and social dynamics is compromised. It's difficult to believe that watching this final tape is comparable to sitting through a typical lesson, especially for the American students. Thus, when Yoshida et al. claim their results show poorer discriminatory ability among American students, it's difficult to be sure that's what they were measuring. They point out that some events in the tape "may have caught the attention of American students because they were perceived as novel or unusual." (p. 616)
Allegedly, none of the American students nor the adult screeners noticed the irregularities in the tape. But there's no indication that students were comfortable criticizing the tape, so their acquiescence could have been amusement or confusion. This especially plausible for the American students of a university laboratory school, who may be accustomed to researcher antics. (Discrete, post-experiment interviews could address this possibility.) There's no indication that students had anything to gain or lose in the tests. The posttest shows significant learning, although it's safe to assume that any lesson with a modicum of coherence and practice would improve posttest scores among moderately self-motivated students. Yoshida et al. compare the 4th grade "Japanese" students with 4th and 6th grade American students. They claim that the 4th American grade students have poorer prior math ability than the "Japanese" students, although they offer no proof of this for the specific populations in this study. In fact, the American 4th graders have higher pretest scores than the "Japanese" students, and their poor posttest scores could be a result of their greater confusion during the lesson.
As mentioned, Yoshida et al. show little awareness of theory and prior research in memory. Memory experts might take issue with the nature of the memory recognition test, in which the statements are read by the investigator rather than the teacher, in a monotone voice. Relevant memory research might indicate that the original voice and tone could be critical in recognition. However, even if Yoshida et al. did accurately find poor discriminatory ability among the American students (as they claim), and if the 6th grade American students had approximately the same prior math ability as the "Japanese" 4th graders (as they claim), then it's unclear why the 6th grade American students showed comparable or better gains from pretest to posttest. In fact, this result seems to contradict Yoshida et al.'s hypothesis.
In the interpretation of their results, Yoshida et al. fail to mitigate the flaws in their design. Their results have very limited generalizability. They don't establish the importance of information discrimination in learning math, because they don't adequately probe students' discriminatory ability. Rather, they probe memory recognition. Both the recognition memory test and the lesson it tests have problematic irregularities. Thus, Yoshida et al.'s broad conclusions are inconsistent with their results. For example, they assert that Japanese students have different schemata to guide their information processing. The "Japanese" students in this study allegedly do poorly at recognizing irrelevant statements because they don't remember them well in the first place, because they discriminated against irrelevant information. It's not clear that the recognition memory test accurately measures information discrimination. Furthermore, identifying which statements were actually said by the teacher isn't part of typical math instruction, nor does it necessarily correlate to understanding and performance (e.g. the American 6th graders and "Japanese" 4th graders have disparate memory test scores but similar posttest scores).
Yoshida et al. suggest that, "if asked, American children would be able to discriminate between statements that are relevant and irrelevant to the goal of the lesson." (p. 616) This seems like a much better approach than the bizarre videotape and convoluted recognition memory test. Yoshida et al. also recognize that their study would be greatly strengthened by a comparable study, in which Japanese and American students were presented with "a typical American lesson." (p. 616) But since neither the lesson nor the populations of this study were typical, it's not clear that cross-study inferences could be made. Certainly, creating a similar videotape by dubbing Japanese over American student voices would be just as peculiar.
Despite the design flaws, Yoshida et al. present an intriguing theoretical base for this inquiry. Information discrimination could be a critical part of learning. In the controlled chaos of a classroom, students are challenged to seize relevant information and incorporate it into their internal representations. This study fails to provide useful results for understanding and extending this model of information discrimination. But the model itself suggests similar, better theories and research into how students learn. It suggests very specific ways educators can help students, including creating more coherent lessons, and helping students learn to discriminate between relevant and irrelevant information. With better design to more directly, adequately probe discriminatory ability, such suggestions for practice could be justified.
Created by Kym Buchanan | http://KymBuchanan.org | This work is licensed under a Creative Commons License.
