There are relatively few studies of the use of goals in classroom teaching (but see Griffee, 1994, 1996; Templin, 1995). Goals in ESL are usually conceived of and formulated in terms of course goals (Brown, 1995; Dubin & Olshtain, 1986). Brown defines goals as "general statements concerning desirable and attainable program purposes and aims based on perceived language and situation needs" (Brown, 1995, p. 71). Goals are also of interest in the field of psychology, especially experimental psychology and management theory. Locke and Latham (1990, p. 2) define a goal simply as "an idea of the future, desired end states." In addition, Claypool and Congemi (1983) add that goals must be attainable, measurable, and compatible with other objectives.
The general findings from the psychology literature are that goal setting groups perform better than control groups or "do your best" groups (Morgan, 1987; Punnett, 1986a, 1986b). In particular, an effective goal needs to be specific and difficult (Bryan & Locke, 1967; Latham & Baldes, 1975; Mento, Steel, & Karren, 1987). Feedback or knowledge of results has also been shown to have a positive effect on performance (Erez, 1977; Locke & Latham, 1990). Ford (1984) found that supervisor feedback combined with videotape feedback resulted in improved work performance on teaching skills of paraprofessionals at a mental retardation facility, and Becker (1978) found in an electric conservation study that households who received feedback three times a week combined with a difficult goal (20% reduced electricity consumption during August) conserved significantly more electricity as compared to a control group, a difficult goal group who received no feedback, and two easy goal groups (2% reduced electricity consumption)--one of whom received feedback and one of whom did not. The purpose of this paper is to examine the effects of teacher-created vocabulary goals and feedback on vocabulary homework assignments. The specific research question addressed in this paper is, "Will the use of specific, difficult goals and feedback improve student performance on weekly vocabulary homework assignments and increase the rate of the weekly vocabulary homework worksheets turned in?"
Subjects
The subjects for this study were 27 students in a first-year conversation course at a private university in Saitama Prefecture, Japan. This was the first year of a new curriculum. Data were gathered in May and June, 1996. Sixteen of the students enrolled in the class were majors from the Euro-American Culture Department and the other eleven students were majors from the Early Childhood Education Department. Two students from the Euro-American Culture Department dropped the course leaving a total of 25 students in the study. All students were Japanese nationals in their early 20s. Thirteen students were men and 12 were women.
In April 1996 all incoming freshman were administered the Secondary Level English Proficiency (SLEP) test, a standardized norm-referenced proficiency test designed for North American high school ESL students (Educational Testing Service, 1981). Students in various majors were divided by SLEP test scores into three levels designated A, B, and C with the A level being the highest. The class being described in this paper is an A class that met for 90 minutes two times a week, once on Tuesday and once on Friday from 11:00 to 12:30 on both days. SLEP scores for this class ranged from 38 to 47 with a mean of 41.9. Comparable TOEFL scores would be about 360 to 450 with a mean of 400 (Educational Testing Service, 1991).
Materials
A word list of what was believed to be 3,095 basic words was compiled based on various word lists and learner dictionaries (Osburn, 1996). Of these words, the first 1,295 were considered by the university curriculum committee to have been introduced in high school and were presumed to be known by students previous to university admission. Of the remaining 1,800 words, the lowest 600 words were assigned to C, the next 600 to B, and the top 600 words to A. Of the 600 words assigned to A, 300 were scheduled to be taught in the Spring semester and the remaining 300 words were to be introduced in the following Fall semester. The 300 words for the Spring semester are divided into twelve sets of 25 words each, and worksheets were created (see Appendix A for an example of a worksheet). Each vocabulary set is scored at one point each for a total of 25 points.
Procedures
Preparation of the vocabulary worksheets was done on a rotational basis by three of the teachers assigned to teach the A courses, one of whom is the present researcher. Each teacher prepared a worksheet and then gave it to at least one other teacher for editing. The worksheet typically consisted of a matching exercise (see Appendix A) and a crossword puzzle. For this study only the matching exercise was considered. Teachers were free to hand out worksheets at their discretion, but an approximate schedule of one vocabulary set per week was considered normal since a semester consists of 12 weeks, and 12 vocabulary sets must be covered in the semester. A program-wide decision was made that the worksheets would be done by students as homework with little or no class time allotted to study or discuss the worksheets. Grading of the worksheets by the teacher was optional. Time for three worksheets was allowed to give students time to begin the course and adjust to the assignment. Accordingly, this study begins with worksheet four.
Analysis
A time-series design (Hatch & Lazaraton, 1991, p. 91) was selected because of its simplicity which lends itself to classroom investigation. The essence of a time-series design is that data are collected on a certain number of occasions, a treatment of some sort is introduced, and data are collected on an additional number of occasions. In that way, the class itself serves as its own control for comparison. In this study, worksheets four through six were collected and scored to establish baseline data and an additional three sets were collected and scored to investigate possible change.
A vocabulary worksheet was handed out to the class as homework. Five class meetings were allowed for the return of the vocabulary worksheet homework after which time it was counted as a zero. Returned worksheets were graded by the instructor but not returned to students until after the final due date to insure students did not pass the answers around. In this way, the scores form a pattern which indicates which students are turning in their homework assignments and how satisfactorily they are performing. After worksheet six, a goal of at least 20 (20 correct answers out of 25 possible for each worksheet) was proposed by the teacher and accepted by the class. The results for each vocabulary set were posted each class on the green board at the front of the room for student inspection, but they were not discussed, see Appendix B. Some explanation of Appendix B might be helpful. The first column, dated May 31, shows that 15 students turned in their worksheets and four achieved the goal. The next column, dated June 4, shows that after the second class, two additional students turned in their worksheets. By reading across, you can see the gradual increase in the number of worksheets handed in. All statistics were calculated using StatView 4.5, a statistical program for the Macintosh.
Results
The mean of the vocabulary worksheet scores shows the average for sets four, five, six, and seven to be about 15 points, but sets eight and nine show a dramatic increase to almost twenty-one points as shown in Table 1. Also of interest is the standard deviation which remains at about the same level for sets four through six, but shows a sharp decrease from set seven on. Table 1. Descriptive statistics for vocabulary worksheets
| _______________________________________________________ | ||||||
| Sets | 4 | 5 | 6 | 7 | 8 | 9 |
| _______________________________________________________ | ||||||
| M | 14.20 | 15.00 | 15.32 | 14.44 | 21.56 | 21.32 |
| SD | 8.21 | 8.76 | 8.31 | 6.83 | 6.09 | 5.80 |
| Minimum | 0 | 0 | 0 | 0 | 0 | 0 |
| Maximum | 23 | 23 | 23 | 21 | 25 | 25 |
| Range | 23 | 23 | 23 | 21 | 25 | 25 |
| Median | 17 | 19 | 18 | 16 | 23 | 24 |
This pattern can also be seen in Figure 1, a cell plot of mean scores. Figure 1. Cell Mean Chart
For vocabulary worksheets five, six, and seven, there were typically five or six students who did not turn in worksheets. For vocabulary worksheets seven, eight, and nine only one student each failed to turn in a worksheet as shown in Table 3. For set seven, three students handed in worksheets that were graded zero because each of those three students missed all 25 vocabulary items. Table 2. Students not turning in vocabulary worksheets
| ________________________________________________ | ||||||
| Set | 4 | 5 | 6 | 7 | 8 | 9 |
| ________________________________________________ | ||||||
| Number not handed in | 5 | 6 | 5 | 1 | 1 | 1 |
| Handed in with score of zero | 0 | 0 | 0 | 3 | 0 | 0 |
| ________________________________________________ | ||||||
| Total | 5 | 6 | 5 | 4 | 1 | 1 |
| ________________________________________________ | ||||||
Discussion
This paper began with two research questions. The first question was, "Will the use of goals that are specific and difficult combined with feedback improve student performance on weekly vocabulary homework assignments?" This question can be answered yes because the mean scores for sets eight and nine show a dramatic increase. It might appear as if no change occurred in set seven, but the standard deviation changes from set seven. The standard deviations are probably decreasing because the scores are increasing and bunching up on the positive end of the scale.
The second question was, "Will the use of goals that are specific and difficult combined with feedback increase the rate of return of weekly vocabulary homework worksheets?" It is not enough that the majority of students improve their scores if a small but significant group of students fail. This question can also be answered in the affirmative as indicated by the reduced number of students who did not turn in their worksheets. The baseline data of sets four through six indicated the existence of a core group of students not turning in their worksheets and presumably not studying the assigned vocabulary. During the time in which sets four through six were being assigned, eight students did not turn in at least one of the vocabulary worksheets. Of this group, three students failed to turn in only one worksheet, but five students failed to turn in at least two out of three vocabulary sets. This hard core group of five students was of special interest during this project because they represent hard to reach students who may be at serious risk of failing the course. Membership in this group is defined by not turning in two worksheet assignments from among sets four, five, or six. Table 3 shows the performance of these five students. Again, dramatic change occurred at set seven. Of particular interest were three students who turned in completed worksheets that scored zero. Keeping in mind that the vocabulary worksheets are assigned as homework and are in the possession of the students for a minimum of two days and a maximum of two weeks, it is not normally possibly for any student to miss twenty-five answers out of twenty-five questions. It is surmised that these three students hurriedly filled in their worksheets in a random manner hoping that their hit-and-miss method would produce an acceptable score. Their last minute efforts, however, can be taken as evidence of the effect of the goals and feedback treatment in that previously these students would not have bothered to turn in anything. At set seven they are motivated to at least go through the motions of appearing to do their assignment. The feedback given them at the next class revealed their failure, and they responded by turning in more acceptable work for sets eight and nine. Table 3. The group of five students who do not do their homework and their scores
M = male, F = female, X = worksheet turned in with no correct answers, O = worksheet not turned in.
| ______________________________________________ | ||||||
| Sets | 4 | 5 | 6 | 7 | 8 | 9 |
| ______________________________________________ | ||||||
| 1(M) | 0 | 0 | 0 | 0 | 25 | 21 |
| 2(M) | 0 | 0 | 0 | 15 | 23 | 0 |
| 3(M) | 0 | 0 | 0 | X | 0 | 9 |
| 4(F) | 21 | 0 | 0 | 20 | 18 | 19 |
| 5(M) | 0 | 18 | 0 | 20 | 23 | 25 |
| ______________________________________________ | ||||||
Table 3 shows improved performance indicating that specific goals and feedback may have the potential to motivate even at-risk students.
Conclusion
If, as this small and descriptive study suggests, goals and feedback have a salutary effect on student performance on homework assignments, what might account for the effect? The goal, in this case a score of 20 points or higher on a 25 point exercise, might serve three functions. One might be to establish a benchmark or criterion of performance which students accept because students judge the goal to be reasonable and attainable. A second function a goal might provide is accountability: when students agree to do a task, they are also agreeing to be held accountable for task completion. These two functions may combine to pressure the student towards task completion because the student knows what must be accomplished and is under some pressure to accomplish it. A third function goal acceptance might achieve is to raise student consciousness or awareness. Some researchers (Schmidt, 1990, 1993; Sharwood Smith, 1991) have suggested that awareness promotes learning.
Feedback, in this case the display of a highly visible posting of scores, might serve two functions. One might be triggering peer pressure. The entire class can see the number of (red color-coded) zeros slowly going down on the posting, and this may have an effect on particular students of wanting to demonstrate solidarity with their classmates. A second function of feedback might again be to cause noticing and thus increasing awareness. Researcher log notes provide some evidence along these lines:
June 18, 1996. Just before the beginning of class, two students (one male and one female) were carefully looking at the scores posted on the board.
June 25. S says I mistakenly gave him set 10 instead of set 9 and that is why he doesnít have set 9 today. He gave me set 10 today.
June 28. I saw S on campus yesterday and gave him set 9. He didnít attend class today which is the last day I will accept set 9, but he came to the cafeteria where I was having lunch and gave me set 9. Student S, it should be noted, is a member of the hard core group noted in Table 3. He is listed there as student 3.
A question this research suggests but does not answer is: what would be the effect of goals and feedback as measured by a test where time and memory pressures are present? Additional areas of further research in the area of goals and feedback include using goals and feedback with groups of students with different proficiency or gender make up, as well as student originated goals vs. teacher goals. Finally, if there is a core group who will not turn in assignments, will goals and feedback consistently identify, motivate, and assist those learners?
References
Abacus Concepts, StatView 4.5 (computer software). (1995). Berkeley, CA: Abacus Concepts.
Becker, L. J. (1978). Joint effect of feedback and goal setting on performance: A field study of residential energy conservation. Journal of Applied Psychology, 63 (4), 428-433.
Brown, J. D. (1995). The elements of language curriculum: A systematic approach to program development. Boston, Mass: Heinle & Heinle.
Brown, T., & Perry, F. (1991). A comparison of three learning strategies for ESL vocabulary acquisition. TESOL Quarterly, 25, 655-670.
Bryan, J. F., & Locke, E. (1967). Goal setting as a means of increasing motivation. Journal of Applied Psychology, 51 (3), 274-277.
Claypool, J., & Congemi, J. (1983). Ingredients of setting and achieving goals. Psychology, A Quarterly Journal of Human Behavior, 20 (3/4), 44-46.
Dubin, F., & Olshtain, E. (1986). Course design: Developing program and materials for language learning. Cambridge University Press.
Educational Testing Service. (1981). Secondary Level English Proficiency Test. Princeton, NJ: Educational Testing Service.
Educational Testing Service. (1991). SLEP test manual. Princeton, NJ: Educational Testing Service.
Erez, M. (1977). Feedback: A necessary condition for the goal setting-performance relationship. Journal of Applied Psychology, 62, 624-627.
Ford, J. E. (1984). A comparison of three feedback procedures for improving teaching skills. Journal of Organizational Behaviour Management, 6 (1), 65-77.
Griffee, D. T. (1994). Student generated goals: A pilot study. Seigakuin University Research Institute Newsletter, 3 (2), 20-23.
Griffee, D. T. (1996). Student generated goals and objectives in a learner-centered classroom. The Language Teacher, 19 (12), 14-17.
Hatch, E., & Lazaraton, A. (1991). The research manual: Design and statistics for applied linguistics. Newbury House.
Latham, G. P., & Baldes, J. J. (1975). The "practical significance" of Lockeís theory of goal setting. Journal of Applied Psychology, 60 (1), 122-124.
Locke, E. A., & Latham, G. P. (1990). A theory of goal setting & task performance. Englewood Cliffs, NJ: Prentice Hall.
Mento, A. J., Steel, R. P., & Karren, R. (1987). A meta-analytic study of the effects of goal setting on task performance: 1966-1984. Organizational Behavior and Human Decision Processes, 39, 52-83.
Morgan, M. (1987). Self-monitoring and goal setting in private study. Contemporary Educational Psychology, 12, 1-6.
Osburn, E. D. (1996). Seigakuin English Program (SEP) master vocabulary list. Language Research Institute, Seigakuin University.
Punnett, B. J. (1986a). Goal setting and performance among elementary school students. Journal of Educational Research, 80 (1), 40-42.
Punnett, B. J. (1986b). Goal setting: An extension of the research. Journal of Applied Psychology, 71 (1), 171-172.
Schmidt, R. (1990). The role of consciousness in second language learning. Applied Linguistics, 11 (2), 129-158.
Schmidt, R. (1993). Awareness and second language acquisition. Annual Review of Applied Linguistics, 13, 206-226.
Sharwood Smith, M. (1991). Speaking to many minds: On the relevance of different types of language information for the L2 learner. Second Language Research, 7 (2), 118-132.
Templin, S. A. (1995). Goal-setting to raise speaking self-confidence. JALT Journal, 17 (2), 269-273.
Appendix A
Sample Vocabulary Worksheet
Name ____________________________
Student Number ____________________
Date _____________________________
S.E.P. Program A Spring Semester
Set 9/Worksheet 1
In the space on the left, write the letter (a, b, c etc.) of the sentence that best matches.
| ____201 ancient | a. The family moved to another country and taught their child to be a good .... of that country. |
| ____202 breed | b. The committee made a .... for improving safety standards. |
| ____203 citizen | c. I want to travel to another country and .... the countryside. |
| ____204 continent | d. What he did made the police .... because he was acting oddly. |
| ____205 crop | e. A .... noun is used when speaking about one person or thing. |
| ____206 depress | f. Taking too many tests can .... students and make them unhappy. |
| ____207 domestic | g. Narita Airport is used for international flights and Haneda airport for .... flights. |
| ____208 explore | h. Where are you? Give me your ..... . |
| ____209 flame | i. The animal trainer used a .... to control the animals. |
| ____210 genuine | j. She was an .... of the town for many years. |
| ____211 horizontal | k. That hill is too .... to ride up on a bike. |
| ____212 inhabitant | l. If you can count it, it has ....; if you can't count it, it has quality. |
| ____213 location | m. .... means flat and level. |
| ____214 migration | n. The main .... in China is rice. |
| ____215 moreover | o. The Silk Road was used in .... times as a trade route. |
| ____216 overcome | p. Some birds .... during the cooler months of the year and the eggs hatch later. |
| ____217 professor | q. He teaches in a university where he is a .... of History. |
| ____218 quantity | r. We saw a new .... used in a SF movie; it made a person fly. |
| ____219 recommendation | s. North America is a large .... and so is Australia. |
| ____220 singular | t. He struggled to .... his shyness. |
| ____221 steep | u. My feeling is ....; I really mean it. |
| ____222 suspicious | v. The apartment rent is reasonable and, ...., the location is perfect. |
| ____223 technique | w. Even though he was big and mean, his .... didn't scare me. |
| ____224 threat | x. The .... of the candle was hot. |
Appendix B
Cumulative Scores for Vocabulary Set 7
0 = vocabulary set not turned in, 0* = vocabulary set turned in with no correct answers.
| 5/31 | 6/4 | 6/7 | 6/11 | 6/14 | |
| Students | |||||
| 1 | 21 | 21 | 21 | 21 | 21 |
| 2 | 20 | 20 | 20 | 20 | 20 |
| 3 | 20 | 20 | 20 | 20 | 20 |
| 4 | 20 | 20 | 20 | 20 | 20 |
| 5 | 20 | 19 | 19 | 20 | 20 |
| 6 | 19 | 19 | 19 | 19 | 19 |
| 7 | 19 | 19 | 19 | 19 | 19 |
| 8 | 19 | 19 | 19 | 19 | 19 |
| 9 | 19 | 18 | 18 | 19 | 19 |
| 10 | 18 | 18 | 18 | 18 | 18 |
| 11 | 17 | 17 | 18 | 18 | 18 |
| 12 | 16 | 16 | 17 | 18 | 18 |
| 13 | 16 | 16 | 16 | 17 | 17 |
| 14 | 15 | 15 | 15 | 16 | 16 |
| 15 | 14 | 14 | 16 | 16 | 16 |
| 16 | 0 | 14 | 14 | 15 | 16 |
| 17 | 0 | 13 | 14 | 14 | 15 |
| 18 | 0 | 0 | 13 | 14 | 14 |
| 19 | 0 | 0 | 0 | 13 | 14 |
| 20 | 0 | 0 | 0 | 13 | 13 |
| 21 | 0 | 0 | 0 | 0* | 13 |
| 22 | 0 | 0 | 0 | 0* | 0* |
| 23 | 0 | 0 | 0 | 0 | 0* |
| 24 | 0 | 0 | 0 | 0 | 0* |
| 25 | 0 |