Student evaluation of teaching is a controversial issue in English language teaching for a variety of reasons. First, there are doubts about the reliability of questions in questionnaire forms where students rate their instructors and in the way students answer them (Baxter, 1991). Secondly, how, where, and why student evaluation data are used may have tremendous impact. In institutions which employ a performance-based appraisal system, such data are included in the instructors' files and can affect their careers. When there are concerns regarding the reliability and credibility of the data collected, their usefulness and validity are questionable.
The aims of this paper are to explore the current status of student evaluation of teaching as a means toward improving quality of teaching and learning, to outline the problems encountered in conducting student evaluations of teaching, and to propose a qualitative model which makes constructive use of student evaluations. The model was tested with students at a Turkish university, and will be described in this context. It is believed that this model is appropriate for other ESL and EFL situations as well.
Current State of Student Evaluation of Teaching
In English language teaching, as well as in other content-based courses, student evaluation of teaching is often conducted on the last day of the course through a questionnaire which asks students to rate their instructors, the course itself, the assessment techniques employed, and the textbooks used. The data are processed by computer and instructors receive a report which shows the average scores for each question and the instructor's rank among his/her colleagues. Some institutions even publish such lists, which potentially creates feelings of resentment and discontent, making teachers less likely to benefit from the informative and diagnostic value of the evaluation process.
Students are crucial sources of information in evaluating the teaching and learning process. Below are five potential reasons that make student ratings of instruction valid (from Scriven, 1994):
- There is a positive and statistically significant correlation between student ratings with learning gains, and students can objectively rate their increased knowledge and comprehension.
- Students are in a unique position in rating factors relevant to competent teaching: the punctuality of the instructor and the legibility of writing on the board; and in evaluating the teaching style, such as the enthusiasm of the teacher, the number of questions the teacher asks, and whether questions are elicited from students.
- Students are in a good position to judge such matters as whether tests covered all the material of the course.
- Students as consumers are likely to be able to report quite reliably on matters such as the cost of the texts, the extent to which attendance is taken and weighted, and the amount of homework assigned.
- Student ratings represent participation in a process often represented as "democratic decision making."
Despite the positive contribution that student ratings can bring to the evaluation process, the main issues to be addressed are identifying reliable and valid methods for data collection and making effective use of the data collected. Some of the potential problems in using a standardized questionnaire to gather student evaluations of teaching are summarized as follows (from Howell, Fox, & Morehead, 1993):
- Using forms that are too long, resulting in students not filling them in or skipping some parts.
- Using forms requiring answers that students suspect will be used to discriminate against them.
- Using forms that are significantly biased towards favorable (or unfavorable) comments.
- Absence of adequate explanations on the importance attached to the students' evaluations and feedback.
- Having the instructors themselves administer the questionnaire, which may influence the way students rate the instructor.
- Delayed processing of the forms, so that by the time instructors receive the results, they are teaching new groups of students and most of the details relating to the previous courses and classes have been forgotten.
- Conducting only summative evaluation which occurs after teaching and learning have taken place and missing out the formative evaluation data which refers to students' feedback while teaching and learning are taking place.
If the above-mentioned points are not considered in designing a student evaluation of teaching model, there exists the risk of conducting a popularity contest with the warm, friendly, and humorous instructor emerging as the winner. In addition, when general evaluative criteria (i.e., a standardized questionnaire) are applied to a specific teaching situation, the results are usually misleading (Millman, 1981).
Having considered the major drawbacks of using a standardized questionnaire, I have identified the following characteristics as desirable in an evaluation model:
- Student evaluation of teaching needs to be carried out in the context of the course rather than in isolation. In other words, student evaluation needs to be an integrated component of the teaching and learning process.
- The ownership and responsibility of student evaluation of teaching should belong to the instructor teaching the course.
- The evaluation of the course needs to be formative in nature to be able to use the findings to diagnose and treat any difficulties that arise.
- Although students contributions are valuable, their perceptions should not be regarded as the final word in the assessment of the course.
- Data collection techniques used in gathering students' perceptions need to be qualitative in nature (i.e., interviews, observations) rather than exclusively quantitative.
A model which incorporates the above characteristics overcomes the difficulties encountered in a summative evaluation system, enabling the instructor to act on the difficulties immediately. As the students see improvements in the teaching and learning situation, they assume ownership of the evaluation and are motivated to participate in it. Also, student evaluation of teaching acts as a tool for the professional self-development of the teacher.
The Model
This model views any English language course in three main components: (1) content of the course, (2) teaching techniques employed in the lessons, and (3) assignments and assessment procedures used. Such a division is necessary since asking the students to evaluate the course as a whole would lead to unmanageable amounts of data. Further, it is necessary to be able to conduct the data collection activity within reasonable time limits during the lesson. It also enables the instructor to ask about one or more of the three components in a specific time frame (a two-week period, for example).
The evaluation model is a cycle of four steps (Figure 1). Conducting a self-observation or a peer observation activity forms the first step in this cycle of professional growth. The aim here is to identify which of the three course components mentioned above will be chosen as the focus of the student evaluation of teaching. The second step in the cycle is to ask the students to evaluate the teaching and learning process using the questionnaire. The third step is to make changes/modifications in the course based on student feedback. The fourth step involves data collection to determine whether the problem has been remedied. To complete the cycle, student feedback is again sought to get their perceptions on the improvements or to identify another point of concern to start another cycle.
Figure 1: The Evaluation Model
|
Step 1: Self or Peer Observation Step 2: Student Evaluation of Teaching Step 3: Change or Modification Step 4: Data Collection |
Implementation of The Model
This four-step model was implemented in an English course which specifically aimed to improve the academic spoken language of freshmen students enrolled in a Turkish university where English is the language of instruction. To demonstrate a real life example, the experiences of the author (as teacher) are reported here following the descriptions of each step.
Step 1: Self or peer observation
The implementation of the cycle starts with identifying which component of the course will be evaluated by the students. Self observation or peer observation are the tools. The teacher chooses among the three points (course content, teaching techniques, or assignments and assessment procedures) or another relevant issue.
In this case, the teacher conducted a peer observation activity, the results of which showed that student participation in the lesson decreased towards the second half of the two-hour teaching block. To address this problem, the teacher and a trusted colleague (who will be referred to as a "critical friend") gathered studentsﾕ perceptions on the teaching/learning activities employed in the lesson.
Step 2: Student evaluation of teaching
The second step in the cycle asks students to evaluate the teaching/learning process. In order to make their comments as concrete and constructive as possible, a three-stage process is used: (1) filling in a questionnaire; (2) pair discussion; and (3) class discussion.
Since this three-stage data collection process requires full comprehension of the questionnaire and discussion of difficulties experienced, it should be done in the native language for beginning-level students. At intermediate levels and higher, it can be carried out in English and conducted as an in-class speaking activity which can be in the form of a discussion, or in interviews with randomly selected students outside class.
In the first stage, the teacher administered a questionnaire during class time, preferably in the second week of teaching, in order to allow the students time to become familiar with the course. Each questionnaire contained a list of statements which are sub-areas of the three main components listed above and students rate them on a scale of 1 to 3, where 1 stands for "appropriate for my learning needs," 2 stands for "satisfactory," and 3 stands for "not appropriate for my learning needs." The aim of the questionnaire was for students to identify the aspects of the teaching/learning process they may not have benefited from. They could also write about areas which did not appear on the list in the blank space provided.
Next, we distributed two questionnaires which focused on the classroom activities employed in the previous two lessons: discussions, role play, reading, and listening activities (the first questionnaire is reproduced in the Appendix).
Students worked with a partner to identify three areas they both had marked "not appropriate for their learning needs." If they had more than three common items, they chose the three most significant. In this specific experience, students were asked to identify three items out of the 26 items on the two questionnaires (there were 14 items on the first questionnaire and 12 items on the second questionnaire).
The second stage involved asking the students to work in pairs and identify one sub-area each and then fill in a second form in which they identify the one that caused the greatest difficulty, a description of the difficulty, and a solution they can offer to overcome this difficulty. At this stage, a pair work activity encouraged the students to benefit from each other's input when describing the difficulty and identifying a remedy for it.
The third stage in the process was a class discussion, where the teacher wrote on the board the points the students had difficulty with in order to identify common ones.
The four most frequently mentioned areas were:
- Looking up vocabulary items in the dictionary (Questionnaire 1, Item 4) was the most frequently mentioned item which the students thought was inappropriate. Firstly, they thought it was an ineffective use of class time and secondly, they reported already being proficient in this skill as they practiced it when they read outside class.
- Writing a final stanza for the song (Questionnaire 1, Item 14) was the second classroom activity which students thought was inappropriate. Since the activity was done at the end of the lesson when they were rather tired, and since it required creativity, they did not have the energy to concentrate and be productive.
- The third item referred to a group decision making activity (Questionnaire 2, Item 3). The students reported that this activity was not appropriate for their learning needs because it involved a lengthy negotiation process as there were many differences of opinions. This led to tension among the group members, and some students reported that they went along with a decision they did not agree with in order to complete the activity within the allocated time.
- Identifying the main idea of a song with a partner (Questionnaire 1, Item 13). Students said that working with a partner consumed too much time compared to working alone.
Step 3: Change or modification
In the third step of the self-development cycle, the teacher acts on the data to bring about a change. Modifications should not be at the expense of covering teaching objectives. Therefore, the aim is not to offer a course which keeps students happy but to create an environment that this conducive to learning by adjusting the teaching process to the needs of students.
The student feedback from these surveys was reevaluated and the following changes were implemented:
- Looking up vocabulary items in the dictionary would not be used as a classroom activity.
- Activities with a creative demand would not be done at the end of a lesson when the students are feeling rather tired. Instead, they would be employed in the initial stages of the lesson when students were feeling fresh and energetic.
- Pair work would be used rather than group work when the aim was to reach a consensus on controversial issues.
- Individual work would be preferred over pair work if the task did not demand participation of two students.
The teacher shared these decisions with the students as the action taken in response to their feedback. This information sharing is crucial to increase student commitment and ownership of the evaluation process.
Step 4: Data collection
After the evaluation data are acted on, the teacher again seeks feedback by collecting data specifically on the modified sub-area. If the data show that the problem persists, other modifications need to be made until it is overcome. Classroom observations by the teacher's "critical friend" can be very helpful at this stage.
Bringing solutions to the problems students mentioned would complete one complete cycle of self-development. To proceed with another cycle, the teacher can repeat the process with a new questionnaire.
Conclusion
This is a constructive and diagnostic model which incorporates student evaluation of teaching into the teaching and learning process. It is useful for a number of reasons:
- It fits the dynamic nature of English language teaching as it offers immediate solutions to the learners. Contrary to any evaluation study conducted at the end of a course, it addresses problems while they are being experienced.
- When students realize that immediate solutions are being provided to their problems, their feeling of ownership of the evaluation increases.
- Since the teacher assumes responsibility of conducting the evaluation and acts on the data to remedy the situation, it acts as a tool for teacher self-development.
- Sharing the data with other colleagues is at the discretion of the teacher and therefore promotes teacher ownership of the evaluation study.
- The evaluation exercise opens communication channels between the teacher and the students as students are given the opportunity to freely express their areas of learning difficulties in the classroom.
- Identifying the problems and acting on them early on in the course minimizes the risk of facing major problems later on.
When compared with content based courses, English language learners usually spend more hours in the classroom and have more direct contact with their teacher Since the input given in each lesson builds on what was presented in the previous lesson, immediate response to students' needs and difficulties carries vital importance in language teaching. The diagnostic and constructive nature of this model therefore serves the needs of language instructors and learners.
References
Baxter, P. E. (1991). The TEVAL experience, 1983-88: The impact of student evaluation of teaching scheme on university teachers. Studies in Higher Education, 16 (2), 151-178.
Howell, K. W., Fox, S. L., and Morehead, M. K. (1993). Curriculum-based evaluation: Teaching and decision making. Pacific Grove, CA: Brooks/Cole Publishing Company.
Millman, J. (1981). Handbook of teacher evaluation. Beverly Hills, CA: Sage Publications.
Scriven, M. (1994). Using student ratings in teacher evaluation. EvaluationPerspectives, 4 (1), 1-3.
Appendix
Questionnaire 1
Dear Students,
Please fill in the questionnaire below considering the lessons we had on November 12. To refresh your memory, the topic of that lesson was Generation Gap.
Please answer the questions very honestly as your feedback will be taken into consideration in designing the future lessons this semester.
You don't have to write your name ! Thanks for your cooperation and time.
Read each item in the questionnaire and tick the appropriate box depending on how much the activity met your learning needs.
Date of the Lesson: November 12
Theme of the Lesson: Generation Gap
| Activity |
+ |
/ |
- |
|
| 1 | Brainstorming on Generation Gap with a partner | |||
| 2 | Whole class discussion on Generation Gap | |||
| 3 | Predicting the content of the reading passage | |||
| 4 | Looking up the vocabulary items in the dictionary | |||
| 5 | Reading the passage on Generation Gap | |||
| 6 | Answering true/false questions after reading the passage | |||
| 7 | Answering the open ended questions about the passage | |||
| 8 | Discussing our answers | |||
| 9 | Discussing how the passage relates to our relationship with parents | |||
| 10 | Going over our predictions of the reading passage and identifying the ideas that came up in the passage | |||
| 11 | Listening to the song Father and Son and filling in the blanks in the worksheet | |||
| 12 | Identifying the common ideas in the reading passage and the lyrics of the song | |||
| 13 | Listening to the song again to figure out the main idea | |||
| 14 | Writing a final stanza for the song |
+ Very Appropriate for my learning needs
/ Satisfactory
- Not appropriate for my learning needs