Beginning last year, JALT publications have featured ongoing discussions of the testing of English language in Japan. In The Language Teacher, Laura MacGregor's (1995a) "My Share" column, "Preparing for the Eiken test," and Shawn Clankies' (1995) "Introduction to commercial English tests in Japan" inspired Greta Gorsuch's (1995) "Opinions & Perspectives" piece warning consumers of such tests to be sure they get some demonstrable reliability and validity for their pains. A somewhat chastened MacGregor (1995b) replied.
James Dean Brown, on his own (1995a, 1995b, 1995c) in The Language Teacher and in collaboration with Sayoko Okada Yamashita (1995a) in the JALT Journal, called attention to problems, inter alia, of reliability and validity in the English portions of tertiary-level entrance exams. Barry O'Sullivan's (1995) "reaction" to the latter posed questions that Brown and Yamashita (1995b) not only embraced but supplemented. Paul Stapleton's (1995) insights about the Confucian ethos of the Japanese educational system, including entrance exams, were explicitly applied to criticize Brown and Yamashita's failure to notice that "the entrance exam . . . places young people into a hierarchy of effort and perseverance: two qualities that have . . . more utility in Japanese society than learning to communicate in English" (Stapleton, 1996, p. 31). Western ELT professionals naturally tend to evaluate English entrance exams in terms of what they feel should be the purposes of such tests, for example, measuring English ability or promoting effective ELT. However, the implicit and more essential purposes of these tests may have little to do with either the English language itself or the teaching/learning of it. Graham Law's (1995) insights on the non-ELT purposes of English instruction in Japan are relevant here.
Most recently, Kimura Shinji and Brad Visgatis compared the difficulty of reading passages in Japanese high school English textbooks with those on junior college entrance exams, finding "examination passages set at a level significantly above that which is expected by the Ministry of Education and reflected in the reading materials found in the approved high school textbooks" (Kimura & Visgatis, 1996, p. 93), a conclusion similar to one of Brown and Yamashita's (1995a, pp. 13-15, 26, 27). All four of these responsible and competent scholars have gone to great lengths to demonstrate that English readings on entrance exams are, generally, objectively so difficult that almost no Japanese high school graduates could reasonably expect to understand them well. There is no reason to doubt these results. However, there are a number of unexamined assumptions in the conclusions these authors draw about the real or potential harm done by such tests.
Kimura and Visgatis (1996, p. 93) infer that junior colleges are "setting standards according to some other benchmark" than Mombusho's, requiring too high a level of English reading comprehension of applicants for admission. For this conclusion to be valid, one would have to demonstrate that comprehension of the reading passages is necessary in order to answer the questions attached to them. This may not be so. Even in her enthusiastic phase, MacGregor's (1995a, p. 29) description of the 2-3 grades Eiken interview test made it clear that it is neither necessary nor desirable to speak (rather than read) one word of English in that test. Since questions follow the order of the short reading passages and may be answered by rereading unprocessed chunks of the passage, not even minimal comprehension is required. Anyone who can read this page could probably pass such a test in Turkish or Basque or any other language written in a Roman alphabet. That would not indicate competence in the language tested. The "speaking" required bears little resemblance to normal human speech, and the "comprehension" demonstrated may be of the structure of the interview rather than of the passage read. In the same way, the "readings" on entrance exams may not be intended to be "read." They may appear simply as locations of material for discrete-point items that can or must be answered based on knowledge unrelated to the readings. When that is the case, the difficulty of the readings is problematic only for examinees who waste time trying to "translate-read" (yaku-doku; see Hino, 1988) rather than immediately attempting to answer the questions. The readings then are "shaggy dog stories," or distractions that render the items "trick questions" really testing memorized "'facts" about English and test-taking strategies. Whatever one thinks of such an approach to testing, to the extent that it exists, it lessens the importance of the readability of the passages.
Other issues affecting the seriousness of the "problem" these tests pose involve the way scores are used. Brown (1995b, p. 11) calls them "a very irresponsible way to decide the future of young Japanese lives." Such a value judgment depends on one's values. It may be that entrance exams are lotteries in which luck in guessing at the answers to absurd questions plays a large, if unacknowledged, role. Is there anything inherently disastrous about selecting students by a lottery in the form of a test? Where is the "fault" in being "egalitarian to a fault" (Brown, 1995a, p. 23)? How are test scores related to entrance? If schools admit a certain number of examinees (presumably those with the highest scores), regardless of the numerical value of the scores, then the difficulty of the test is largely irrelevant to the admissions process. Only if a certain number or percentage of correct answers are required does the exam's objective level of English become a factor in deciding young people's futures. Otherwise, it's just a random condition, like the weather, that may or may not affect results but usually requires no organized compensatory strategies. Why would "rational methods . . . in deciding who will be admitted" (Brown & Yamashita, 1995b, p. 259) necessarily be preferable? Establishing the "grade level" of English involved in a reading passage is less useful if attainment of that level is not necessary to success on the test.
A related question is whether, as Brown (1995a, p. 23) asserts, entrance exams are the only way to be admitted to Japanese universities. Along with more conventional exams, other application processes may also be available, for example, suisen-nyu ("entrance by recommendation"; a term conspicuously absent from Brown's (1995c) vocabulary guide). Interviews and other instruments are sometimes used. It may be that there are in fact several ways in to college. Study of exactly what and how widespread they are is needed in order to determine the real importance of entrance exams to prospective students.
One must also question whether the only purpose of entrance exams is to select students for admission. Assuming that to be the case, the process surely seems irrational and wasteful. However, some students (literally) sit for exams without trying to succeed on them. One sees them sleeping, not answering questions, or "answering" them without looking at them. Such examinees may not be seeking admission to school as much as submitting to a ritual demanded by family or peers. From the point of view of efficient screening of applicants, colleges could base admissions on high school grades, informed by ranking of high schools. There's really no need for a further test of academic ability. The reason this is not done may be because the income from entrance exam fees is necessary for the schools' financial survival. If that is so, is it "ridiculous for each university to develop its own battery of tests" (Brown, 1995b, p. 9)? All the purposes served by tests need to be accounted for, not just those consistent with the researchers' values.
Extending Gorsuch's (1995, p. 37) "educators as consumers" to include students, their families, their future employers, and society in general, consumers of Japanese higher education have far more important areas of concern than the English on entrance exams, even if there is some of Brown's "washback" or Gillian S. Kay's "washforward" (Brown, 1995b, pp. 8-9) effect. Entrance exams will change if society demands value for its money and restructures the educational system accordingly. As long as people are willing to spend millions of yen sending their adult children to two- or four-year day care centers, there will be no need for meaningful change. Where ELT would fit in such change is not obvious. Despite the current importance of English as the principal international language and its long-time hegemony as Japan's favorite foreign language, it might lose out to others geographically and culturally closer, especially Korean and Chinese. Even now, the fact that employers often favor applicants with Eiken certification (MacGregor, 1995b, p. 51) indicates that the "real world" values misleading test scores more than actual competence -- just as much as academia does. Although some Japanese people find more interesting jobs because their English is practical and useful, others may find such a skill disadvantageous at work, adding to their chores but not to their status or their compensation. The motivations currently in place for testing well in eigo are clear. Those for acquiring English are not.
None of the above is intended to defend any aspect of the Japanese system of higher education, only to indicate the complexity of issues involved. Brown and Yamashita (1995b, pp. 259-260) have raised some of these issues, but from a point of view largely irrelevant to the Japanese system. (Their question, for example, "how are norms established . . . ?" [Brown & Yamashita, 1995b, p. 259], assumes that norms should be established.) The methodological issue here is the inherent danger of specialist myopia. The more accurate one's knowledge is in one area, the less chance that knowledge will be applicable elsewhere. If entrance exams to Japanese universities are neither designed nor used as tests in the tradition of the US TESL industry, criticism grounded in that tradition is unlikely to hit its mark.
References
- Brown, J. D. (1995a). English language entrance examinations in Japan: Myths and facts. The Language Teacher, 19 (10), 21-24, 26.
-
Brown, J. D. (1995b). English language entrance examinations at Japanese universities: Interview with James Dean Brown (G. S. Kay, Interviewer). The Language Teacher, 19 (11), 7-9, 11.
- Brown, J. D. (1995c). A gaijin teacher's guide to the vocabulary of entrance exams. The Language Teacher, 19 (12), 25.
-
Brown, J. D., & Yamashita, S. O. (1995a). English language entrance examinations at Japanese universities: What do we know about them? JALT Journal, 17, 7-30.
- Brown, J. D., & Yamashita, S. O. (1995b). The authors respond to O'Sullivan's letter to JALT Journal: Out of criticism comes knowledge. JALT Journal, 17, 257-260.
-
Clankie, S. (1995). An introduction to commercial English tests in Japan. The Language Teacher, 19 (6), 8-10.
- Gorsuch, G. J. (1995). Tests, testing companies, educators, and students. The Language Teacher, 19 (10), 37, 39, 41.
-
Hino N. (1988). Yakudoku: Japan's dominant tradition in foreign language learning. JALT Journal, 10, 45-55.
- Kimura S., & Visgatis, B. (1996). High school English textbooks and college entrance examinations: A comparison of reading passage difficulty. JALT Journal, 18, 81-95.
-
Law, G. (1995). Ideologies of English language education in Japan. JALT Journal, 17, 213-224.
- MacGregor, L. (1995a). Preparing for the Eiken test. The Language Teacher, 19 (7), 29-30.
-
MacGregor, L. (1995b). More on the Eiken. The Language Teacher, 19 (11), 51, 86.
- O'Sullivan, B. (1995). A reaction to Brown and Yamashita "English language entrance exams at Japanese universities: What do we know about them?" JALT Journal, 17, 255-257.
-
Stapleton, P. (1995). The role of Confucianism in Japanese education. The Language Teacher, 19 (4), 13-16.
- Stapleton, P. (1996). A reaction to J. D. Brown's recent inquiry on the English entrance exam. The Language Teacher, 20 (3), 29, 31-32.