Using criterion referenced assessment toward a reorientation in student motivation

Page No.: 
19
Writer(s): 
Jean-Pierre Antonio and Kevin O'Donnell, Suzuka International University

Every April, university instructors of English face a daunting task—to transform a class of English-weary students into enthusiastic learners. It is a challenge that taxes even the best teachers as they encounter students who seem jaded with traditional language-learning methods. Beginning in middle school, and extending through senior high school, English as a living language, and a tool for communication, is transformed by the pressures of looming entrance examinations into a key—one that is necessary only once in a lifetime (Brown, 1995; Frost, 1991; Mochizuki, 1992). Following successful completion of university entrance exams, many students find themselves bored with English—six years of study have served their purpose, and that particular key will not be required again. Therefore, if students cannot develop a new reason for studying English at the university level, improved language proficiency will be limited (Berwick and Ross, 1989). As students begin to study English in the university setting it is vital that instructors guide them towards a personal motivation for learning English.

The following is an explanation of one method we have developed to help students find renewed meaning in the study, and use, of English. At the university level, teachers have the opportunity to create an educational environment that is fundamentally different from exam-oriented pedagogy. Creating such an environment is a challenge for both teachers and students, because the guiding principles for developing communicative competence call for students to take the lead in originating ideas and dialogues, and for the instructor to act as a facilitator and guide, rather than a lecturer. The assessment tools that are appropriate in this context will be defined and discussed, as several criterion referenced assessment tools are introduced and explained using one unit plan centered on getting students to speak with expression.

Communicative Competence

Instructors teaching oral communication will reach their goals most effectively when they use teaching methods that develop communicative competence. Bachman (1990) defines competence as, "both knowledge of or competence in the language, and the capacity of implementing or using this competence" (p. 81). Ellis (1997) points out that:

In many parts of the world [like Japan], learners are exposed to large amounts of grammar teaching in the early stages of their language learning and fail to develop any fluency in the target language. Such learners are likely to benefit from communicative activities rather than more grammar teaching (Ellis, 1997, p. 71).

Nunan (in Ellis, 1997) defines these activities as tasks that, "involve the learners in . . .producing or interacting in the target language while their attention is principally focused on meaning rather than form" (p. 201). Speaking examinations using criterion referenced assessment can offer the instructor one way in which to assess and guide student progress toward communicative competence in English.

Criterion Referenced Assessment

Brown (1990) presents a framework describing two families of testing: norm referenced and criterion referenced assessment tools. Criterion referenced assessment is also referred to as competency based assessment. Brown (1990) shows where norm referenced assessment tools are used appropriately for proficiency and placement testing. Criterion referenced assessment tools are shown to be appropriate at the classroom level for achievement testing of content and for diagnostic purposes. Whereas norm referenced assessment tools are used to interpret student scores in relation to others who have taken the same test, criterion referenced assessment tools are meant to, "measure the degree to which students have developed skills or knowledge in relation to a specific set of [content] objectives" (Brown, 1990, p. 4). Brindley (1994) suggests that among the benefits of this type of assessment, students and instructors alike, "become focused on language as a tool for communication rather than on language knowledge as an end in itself" (p. 52). Evaluation of student progress becomes tied directly to day-to-day teaching.

Co-Verbal Language and its Use in Communicative Competence

One key to students' success is learning to use the tools of language to add affective/emotive content to daily conversation (Keller & Warner, 1979). Co-verbal language, also known as gambits, incorporates the crucial words and phrases described by Brinton (2000) as the language used "to maintain social intercourse (as in greetings or talk about the weather)[, . . . and is] the most important and frequent function of language . . ."(p. 4). Students achieve greater fluency by focusing on the effective use of this aspect of language, and practicing the use of gambits in context.

A Problem of Communicative Inauthenticity

The purpose of Oral Communication I and II is to introduce conversational English and provide students with an environment where they can use the language. The course will both build on the knowledge of English that students bring to class and lay a foundation for their future studies (Suzuka International University, 2002, p.1).

The passage above is part of a course description written for a second-year university oral communication class in an International Cultural Studies program. Prior to implementing the teaching approach discussed in this paper, we established that students brought some prior knowledge of English to the university class. With some proficiency in the language, even lower-level students were able to gain greater confidence in their language acquisition. Students learned new vocabulary and grammar in the classroom, and did, to a certain degree, speak English. However, it must also be acknowledged that the end product fell short in promoting proficiency in natural conversation. Even after several lessons working on original paired dialogues, vocabulary and grammar checks, and practice, student delivery often seemed mechanical and expressionless. Dialogues were, in effect, delivered as monologues, with students speaking their lines, but not relating, or responding, to their partners' contributions. In retrospect, students could not be faulted for their efforts because they had not been taught co-verbal language—the very thing that breathes life into any conversation.

Those classes inspired us to depart from our prior teaching methods. We overhauled the lesson plans, and found that the incorporation of gambits and the use of criterion referenced assessment in evaluation helped students markedly improve their emotive communicative proficiency. During the last academic year, our students explored a variety of situation based responses: expressing surprise, interest, doubt, sympathy for a negative event, congratulations for a positive event, neutral expressions, agreement, asking for favors, problems, and apologies. In contrast to student performances in the past, dialogues were often unique, full of humor, surprise, and drama: students had come to own this language.

Sample Unit

Lesson 1.

This unit begins with the introduction of co-verbal language (Appendix 1). This resource and activity sheet introduces the use of responders, explains the significant role of responders in conversation and lists some of the more frequently used examples in categories for specific situations. Following the explanation there are dialogue samples two lines in length. A's line is provided and the student must choose an appropriate responder for B. After completing the exercise sheet, students work in pairs. One student reads A's statement in random order and the other student, without looking at the paper, replies with an appropriate responder.

Lesson 2.

In the second lesson students write original paired dialogues using at least two of the five responder categories. These are memorized and performed. Three examples of student-generated dialogues, each representative of the wide range of expression that can be found in imaginative interactions are presented below. The first dialogue describes a child's deep-seated fear of parental abandonment that turns into a fantastic dream. The second speaks of the student's strong desire to travel and explore the world. The third illustrates one student's unique sense of humour and fascination with science fiction.

Dialogue I:

A: Hi! How are you?
B: Fine, thanks. And you?
A: Good. I didn't see you for a while. What happened?
B: Yes. I went back to my parent's after a three year absence. Strangers live there now.
A: Really? You must be joking! What did you do?
B: I called my mother right away. I asked, "What happened? Where are you and father staying?" She said, "We moved to the next town two years ago. I forgot to tell you." They sold their house and moved to a new house and I didn't know!
A: No way! Did you go to your parent's house?
B: Yes. I asked for the new address and went right away.
A: What kind of house do your parents live in now?
B: They built a house like Cinderella's castle in Disneyland by themselves. And they have many animals such as dogs, pigs, ducks, and sheep. It's like a small zoo!
A: Wow! I can't believe it!
B: Me too. But it's a great place.
A: I think so.

Dialogue II:

M: Hi! Listen to my amazing story!
Y: What?
M: I won first prize in the neighborhood grand marathon!
Y: Wow! Really?
M: Yes. I got travel tickets for five hundred thousand yen as the prize.
Y: So much?! That's unbelievable!
M: I visited a travel agency to use the ticket and I found an excellent package tour.
Y: That's great!
M: To my amazement there was an around-the-world tour for about two hundred thousand yen!
Y: No way! I can't believe it!
M: I'm going to leave next month with my friend. We'll visit Vietnam first.
Y: Have a nice trip!

Dialogue III:

A: Hi! How are you?
B: Fine, thanks. How about you?
A: Good.
B: What's new recently?
A: I went to Alaska.
B: Really? Alaska? Why?
A: Because I heard there are UFO's there.
B: What? Really? Do you believe it?
A: Yes, I do. I looked for the aliens. I stayed there for three months to find them.
B: Are you kidding?
A: No! No, I'm not. A month ago I finally found them!
B: Wow! I don't believe it!
A: I made friends with them. They said they'll come to Japan next week.
B: What!? WOW!

Lesson 3.

In the last lesson, as a final activity, students create, and are evaluated on, improvised dialogues. After negotiating/explaining how the evaluation will be done using the criterion reference assessment rubric (Appendix 2), students choose two out of the five responder categories in a blind selection and, after a period of preparation with a variety of partners, their test partner is selected. Test pairs are given an appropriate amount of preparation time, and are then asked to present their improvised dialogue. After each test, student pairs are shown their evaluation sheet. Strengths and weaknesses can be easily explained, because students know the evaluation expectations. Assessment must be tied to how well students are able to meet the pre-negotiated criteria. In addition, one advantage for the students is the immediacy of the feedback. Students show great interest in the results of the evaluation and want to know how to improve their score. The criterion reference assessment rubric would appear to be a valuable, even essential, evaluation tool for both instructors and students.

Conclusion

If we can save even one teacher from jumping out the window at the sight of yet another classroom of grammar-gloomy, English-weary students, we will be happy. Just remember that it doesn't have to be a battle of English versus ennui—teachers simply need to help students find a new motivation: To use English in authentic contexts and in a naturalistic manner. One way that this can be accomplished is by incorporating the use of co-verbal language in lessons that emphasize personal communicative aspects of English. Criterion referenced assessment not only assists in accurately assessing students' language proficiency but has also been shown to help motivation. Students received immediate feedback upon completion of speaking activities. Specific strengths and weaknesses were highlighted, offering them direction for focusing on improvement. Over the course of an academic year, our students achieved greater fluency and confidence, particularly in their use of emotive content. As a result, we hope that these students have rediscovered English as a key to open doors throughout their lives, leading them into unexpected and welcome places.

References

Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
Berwick, R. & Ross, S. (1989). Motivation after matriculation: Are Japanese learners of English still alive after exam hell. JALT Journal, 11 (2) 193-210.
Brindley, G. (1994). Competency-based assessment in second language programs: Some issues and questions. Prospect 9 (2), 41-55.
Brinton, L. J. (2000). The structure of modern English: A linguistic introduction. Amsterdam: The Netherlands: John Benjamins Publishing Company.
Brown, J. D. (1990). Where do tests fit into language programs? JALT Journal, 12 (1), 1-10.
Brown, J. D. (1995). English language entrance examinations in Japan: Myths and facts. The Language Teacher, 19 (7), 21-26.
Ellis, R. (1997). SLA research and language testing. Oxford: Oxford University Press.
Frost, P. (1991). Examination hell. In E. Beauchamp (Ed.), Windows on Japanese Education (pp. 137-155). Hartford, CT: Greenwood Press.
Keller, E., & Warner, T. S. (1979). Gambits (responders, closers and inventory). Canada Communication Group, Government of Canada.
Ministry of Education, British Columbia, Canada. (2003). Integrated resource packages. [Online]. Available: www.bced.gov.bc.ca/irp/irp.htm.
Mochizuki, K (1992). The present climate in Japanese junior high schools. In J. J. Shields (Ed.). Japanese schooling (pp 139-157). University Park: State University Press.
Suzuka International University. (2002). Oral communication 1-1, Suzuka International University Course Calendar. Suzuka, Japan: Suzuka International University.

Jean-Pierre Antonio is an assistant professor in the English Studies Department of Suzuka International University. He is interested in materials development.

Kevin O'Donnell is a lecturer in the International Culture Department of Suzuka International University. His research interests include materials development and teacher education.