Students' reactions to written test item rebuttals.

Students' reactions to written test item rebuttals. Multiple choice tests have the benefit of having one correct option per question, thus allowing for convenient machine scoring. One disadvantage, however, is that students do not have the opportunity to explain why they selected a distracter dis��tract��eralso dis��trac��tor ?n.One of the incorrect answers presented as a choice in a multiple-choice test. , even though their reasoning could demonstrate content knowledge. I have experimented with allowing students to argue test questions in writing. By reviewing students' remarks, I have the opportunity to understand students' reasoning and make corrections to my test items. ********** Multiple choice tests are ubiquitous in college, especially in large introductory classes (Paxton, 2000). Tests composed of multiple choice items have a number of advantages: ease and objectivity in scoring, potential to cover a large amount of content, and greater difficulty for students in guessing the correct answer than true-false (T-F T-F Time-Frequency ) questions. Despite the benefits of multiple choice testing, difficulties with the method exist. One of the potential problems of the multiple choice test is that although one option per item is keyed as correct, students may select a distracter for legitimate reasons. For example, students' comments during class discussion may have included one of the incorrect answers. Several authors (Dodd & Leal LEAL. Loyal; that which belongs to the law. , 1988; Nield & Wintre, 1986) have examined the use of multiple choice tests, and have proposed similar approaches to the problem of "tricky" multiple choice questions. Nield and Wintre experimented with an "E" option on four-option multiple choice tests. In this format, students could choose one of the four options and if desired, explain a marked answer by also marking the "E" option. First, instructors graded exams with the scoring key; then they reviewed the "E" options marked. Students received a point for a good explanation of a wrong answer, but lost a point for a bad explanation of a correctly marked answer. Dodd and Leal's "answer justification" technique works in a similar manner. While taking the exam, students write justifications for their answers to those questions they see as tricky. Dodd and Leal first grade the exam and then review the students' justifications for the questions they missed. Students responded favorably to the "E" and answer justification options, reported less anxiety in taking multiple choice exams, and considered the "E" option multiple choice exam the least frustrating frus��trate?tr.v. frus��trat��ed, frus��trat��ing, frus��trates1. a. To prevent from accomplishing a purpose or fulfilling a desire; thwart: type of exam to take when compared with true-false, short answer, and essay. These two approaches yield the desired outcome of students demonstrating content knowledge, but are time consuming in a large section class, especially if one does not have a teaching assistant (Dodd & Leal, 1988, note their technique requires less scoring time than the Nield & Wintre, 1986, technique). The technique I describe should take less time than either the Dodd and Leal or Nield and Wintre procedures. My method combines the Nield and Wintre and Dodd and Leal benefits of providing students an opportunity to explain their answer with the notion that tests can serve as learning events (Foley, 1981). Foley permitted students to take exams home to research answers to the questions they did not know and then retake re��take?tr.v. re��took , re��tak��en , re��tak��ing, re��takes1. To take back or again.2. To recapture.3. To photograph, film, or record again.n.1. the test (Toma & Heady, 1996, describe a similar technique). Foley' s approach w as popular with students, but it seems impractical to create new items for every new class. What I have done is encourage students to review the tests in my office, on library reserve, or during a specified class and prepare written rebuttals to questions in which they explain their rationale for selecting an incorrect option. I review these rebuttals to determine if students' responses justify awarding more points. I have used this method for several years and have collected responses on a brief survey to assess student's reactions. Method Participants Participants were 238 students enrolled in my classes over a three year period. These undergraduate classes included lower- and upper-division courses in statistics, industrial psychology, organizational behavior, tests and measures, and history/systems of psychology. Total number of students in these classes was 287, thus the percentage of students who completed surveys was 83%. Materials and Procedure At the beginning of the quarter, students received within their syllabus A headnote; a short note preceding the text of a reported case that briefly summarizes the rulings of the court on the points decided in the case.The syllabus appears before the text of the opinion. an explanation of the opportunity and approach to write item rebuttals. I asked students who wanted to rebut To defeat, dispute, or remove the effect of the other side's facts or arguments in a particular case or controversy.When a defendant in a lawsuit proves that the plaintiff's allegations are not true, the defendant has thereby rebutted them. TO REBUT. items to turn in their rebuttals, with documentation for their distracter selections, by the next test. Documentation could take the form of citations from the textbook or copies of the lecture notes. Students examined tests on library reserve, during exam periods, or during office hours office hours,n.pl See business hours. . At course-end, I asked students to complete a brief survey (1) about their reactions to the use of rebuttals. I content coded the open-ended responses and recorded students' final grades (percent of points possible). The author and a graduate student independently reviewed and coded the surveys according to according toprep.1. As stated or indicated by; on the authority of: according to historians.2. In keeping with: according to instructions.3. a prior categories. On 17 elements [of 487], the raters disagreed. The two raters met, discussed, and resolved their differences. Results Of the 238 completed surveys, 83 (35%) students reported having written a rebuttal rebuttaln. evidence introduced to counter, disprove or contradict the opposition's evidence or a presumption, or responsive legal argument. . Seven of these 83 did not answer any other questions on the survey. Of the 76 who did, 65 (86%) had successfully argued for additional points. Did students get the points they believe they deserved? Seventeen of the 75 responding to the question (23%) believed they received fewer points than to which they were entitled. Only 11 of 17 explained why they deserved more point s--5 respondents believed they had made a good argument; another 5 believed they should be rewarded for their time and effort; and one student said he made a mistake in preparing his rebuttal. Of the 73 who responded to the question about presenting their cases in person, 17 (24%) believed they would have been more successful if they had been permitted to give oral arguments. Six said they were more skilled in oral discourse than written; another 10 said they could have countered the instructor's response with argument and thus been more persuasive (one student said he made a mistake in preparation of his rebuttal). Three major categories of explanations emerged when students described why they did not write rebuttals. Of the 154 who gave an explanation, 77 (50%) conceded that keyed answers were better than their choices, 38 (25%) indicated they were too busy to take the time to prepare a rebuttal, and 16 (10%) were satisfied with their test grade or believed there was sufficient opportunity to make up lost points through other means (e.g., extra credit). The remaining 15% were a mixture of the three categories described or unique. Although those who did not file a rebuttal because of satisfaction with their grades did attain the highest scores (expressed in percentages), an analysis of variance did not reveal a statistically significant effect, F(3,150) =1.92, p =. 13. Students who wrote rebuttals spent about 19 minutes (Mn = 18.59)preparing each rebuttal with a range of 1 to 120 minutes (median = 11 minutes). There was a negligible relationship between the length of preparation time and student success in gaining points, [r.sub.pb] = .08; n=70. Were students who prepared written rebuttals better students? A correlation between total percent earned in the class and the categorical That which is unqualified or unconditional.A categorical imperative is a rule, command, or moral obligation that is absolutely and universally binding.Categorical is also used to describe programs limited to or designed for certain classes of people. variable of writing a rebuttal yielded [r.sub.pb] = .22, p <.05 (n = 232). A t-test [t(230) = 3.49, p < .001] indicated a significant difference between those who did a rebuttal (Mn = 81.51%, SD = 9.12) versus those who had not (Mn = 76.03%. SD = 12.19). Thus, better students were more likely to have written a rebuttal (2). There was not a significant relationship, however, between total percentage of points in the class and whether a rebuttal was successful ([r.sub.pb] = .04). Discussion By reviewing students' remarks, I can determine if I placed too much emphasis on an issue that led them astray a��stray?adv.1. Away from the correct path or direction. See Synonyms at amiss.2. Away from the right or good, as in thought or behavior; straying to or into wrong or evil ways. in test-taking or an item was worded poorly. I can make corrections to my tests or lectures and students have an option to recoup recoupTo sell an asset at a price sufficient to recover the original outlay or to offset a previous loss. some of the points they lost on the first scoring of the test. This procedure is also a mechanism to encourage students to do more writing, an activity that has become required less often as class sizes have grown. The item rebuttal approach goes beyond the "E" option and answer justification methods as students must write short essays with proper language to attain credit (Paxton, 2000). Good writing is key to a successful rebuttal attempt as poor grammar or spelling negatively affects the probability of recovering points. Further, I avoid face-to-face confrontations that sometimes rely less on logic than on emotion. Students have reported that they sometimes abandoned their item rebuttal effort when they realized their choice was simply not as correct as the keyed response. Most important, however, is that students spend more time with the course material. Instructors who want to use this technique but feel squeezed by time may wish to limit the number of rebuttals accepted. I do not yet restrict the number of rebuttals, spending at most one hour per exam evaluating rebuttals with five different items rebutted on a typical exam. I need to expend ex��pend?tr.v. ex��pend��ed, ex��pend��ing, ex��pends1. To lay out; spend: expending tax revenues on government operations.See Synonyms at spend.2. a little extra time to use this method, but I share Foley's (1981) contention that exams can be learning events and believe that students can learn through this technique. Author Note Portions of this paper were presented at the Western Psychological Association Annual Meeting in Seattle, Washington This page is protected from moves until disputes have been resolved on the .The reason for its protection is listed on the protection policy page. , on April 26, 1997. The author thanks Diane Pfahler and Cheryl Rickabaugh for their helpful comments on an earlier version of this paper. Footnotes (1) A copy of the survey for interested readers is available from the author. (2) The total number of points possible in my undergraduate classes is typically 500. Students who have been the most successful in writing rebuttals have been awarded 10 additional points (or 2% of the possible points in the class). Most students who have written rebuttals have achieved 3 to 5 points (or 1% of the possible points). References Dodd, D.K., & Leal, L. (1988). Answer justification: Removing the "trick" from multiple choice questions. Teaching of Psychology, 15, 37-38. Foley, D. P. Instructional potential of teacher-made tests. (1981). Teaching of Psychology, 8, 243-244. Nield, A.F., & Wintre, M.G. (1986). Multiple-choice questions with an option to comment: Student attitudes and use. Teaching of Psychology, 13, 196-199. Paxton, M. (2000). A linguistic perspective on multiple choice questioning. Assessment and Evaluation in Higher Education higher educationStudy beyond the level of secondary education. Institutions of higher education include not only colleges and universities but also professional schools in such fields as law, theology, medicine, business, music, and art. , 25, 109-119. Toma, A.G., & Heady, R.B. (1996). Take-two testing. College Teaching, 44, 61. Index Akinade, E.A., Risk-taking behaviour and substance abuse vis-a-vis HIV HIV(Human Immunodeficiency Virus), either of two closely related retroviruses that invade T-helper lymphocytes and are responsible for AIDS. There are two types of HIV: HIV-1 and HIV-2. HIV-1 is responsible for the vast majority of AIDS in the United States. transmission in African societies, 28(1), 3-8. Arfaniarromo, A., Toward understanding of Latino gang members in U.S. Schools: Psychosocial psychosocial/psy��cho��so��cial/ (si?ko-so��shul) pertaining to or involving both psychic and social aspects. psy��cho��so��cialadj.Involving aspects of both social and psychological behavior. and Sociocultural so��ci��o��cul��tur��al?adj.Of or involving both social and cultural factors.soci��o��cul development factors., 28(3), 123136. Astleitner, H., Designing emotionally sound instruction: An empirical validation An empirical validation of a hypothesis is required for it to gain acceptance in the scientific community. Normally this validation is achieved by the scientific method of hypothesis commitment, experimental design, peer review, adversarial review, reproduction of results, of the FEASP-approach, 28(4), 209-219. Beale, E.W., Analysis of state standards for paraprofessionals, 28(4), 244-248. Carter, T., Hardy, C.A. & Hardy, J.C., Latin vocabulary acquisition: An experiment using information-professing techniques of chunking and imagery, 28(4), 225-228. Chow, P. & Wood, W., Comparing Canadian and United States United States,officially United States of America, republic (2005 est. pop. 295,734,000), 3,539,227 sq mi (9,166,598 sq km), North America. The United States is the world's third largest country in population and the fourth largest country in area. High School Students on Cognitive Dissonance cognitive dissonanceMental conflict that occurs when beliefs or assumptions are contradicted by new information. The concept was introduced by the psychologist Leon Festinger (1919–89) in the late 1950s. Test Scores, 28(3), 137-40. Constantine, J.L., Integrating thematic-fantasy play and phonological awareness Phonological awareness is the conscious sensitivity to the sound structure of language. It includes the ability to auditorily distinguish parts of speech, such as syllables and phonemes. activities in a speechlanguage preschool environment, 28(1), 914. Desai, M.S., Computer anxiety and performance: An application of a change model in a pedagogical ped��a��gog��ic? also ped��a��gog��i��caladj.1. Of, relating to, or characteristic of pedagogy.2. Characterized by pedantic formality: a haughty, pedagogic manner. setting, 28(3), 141-149. Dominguez, P.A. & Ridley, D.R., Assessing distance education courses and discipline differences in their effectiveness, 28(1), 15-19. Dykeman, B.F., School-based cognitive interventions Cognitive Interventions are a set of techniques and therapies practiced in counseling. This form of counseling is the practice of Cognitive Psychology.The range of Cognitive Interventions are: A-B-C-D Analysis Disputation Desibels Redecision Work for children with postconcussional disorder, 28(1), 20-4. Ediger, M., Education and the societal arena, 28(1), 25-30. Ediger, M., The school principal: State standards versus creativity, 28(2), 79-83. Ediger, M., Assessment, geography, and the Teacher, 28(3), 150-4. Ediger, M., What makes for a quality science curriculum?, 28(4), 241-243. Fisher, D., Alternatives to the reading resource room for struggling adolescent readers, 28(4), 234-240. Foote, C. J. & Vermette, P.J., Teaching Portfolio 101: Implement the teaching portfolio is introductory courses, 28(1), 31-7. Gadzella, B.M. & Baloglu, M. Confirmatory factor analysis In statistics, confirmatory factor analysis (CFA) is a special form of factor analysis. It is used to assess the the number of factors and the loadings of variables. and internal consistency In statistics and research, internal consistency is a measure based on the correlations between different items on the same test (or the same subscale on a larger test). It measures whether several items that propose to measure the same general construct produce similar scores. of the Student-life Stress Inventory, 28(2), 84-93. Iannone, R., An educator in exile, 28(1), 38-43. Jason, L.A., Kennedy, C.L., & Taylor, R.R., Development and evaluation of a Web-based classroom, 28(3), 155-160. Johnson, H.R., Administrators and mentors: Keys in the success of beginning teachers, 28(1), 44-9. Jonas, P.M., Weimer, D., & Herzer, K., Comparison of traditional and nontraditional (Adult Education) undergraduate business programs., 28(3), 161-170. Katayama, A.D., Bi-modal instructional practices in educational psychology: Mentoring and traditional students, 28(3), 17 l- 177. Keim, J., Warring, D.F. & Rau, R., Impact of multicultural training on school psychology and education students, 28(4), 249-252. Koszalka, T.A., Effect of computer-mediated communications Computer-Mediated Communication (CMC) can be defined broadly as any form of data exchange across two or more networked computers. More frequently, the term is narrowed to include only those communications that occur via computer-mediated formats (i.e. on teachers' attitudes towards using web resources in the classroom, 28(2), 95-103. Kottke, J.L., Students' reaction to written test item rebuttals, 28(4), 256-258. Neal, J. & Frick-Horbury, D., The effects of childhood parenting styles Parenting style is a psychological construct representing standard strategies parents use in raising their children.One of the best known theories of parenting style was developed by Diana Baumrind. on attachment as a predictor of interpersonal relationships This article or section may contain original research or unverified claims.Please help Wikipedia by adding references. See the for details.This article has been tagged since September 2007. , 28(3), 178-183. Petress, K., The ethics of student classroom silence, 28(2), 104-7. Poole, S.W., Designing and measuring educational outcomes utilizing student portfolios for business management education, 28(1), 50-8. Rodriguez, G., Johnson, S. W., & Combs, D.C., Significant variables associated with assertiveness assertiveness/as��ser��tive��ness/ (ah-ser��tiv-nes) the quality or state of bold or confident self-expression, neither aggressive nor submissive. among Hispanic women, 28(3), 184-190. Rosenthal, McKnight, & Price, Who, what, how, and where the typical psychologist is ... The Profession of Psychology Scale, 28(4), Sankaran, S.R. & Bui, T., Impact of learning strategies and motivation on performance: A study in Web-based instruction, 28(3), 191-8. Simplicio, J. S. C., A lesson on racial differences, 28(2), 108-110. Simplicio, J. S. C., How to recognize and counteract student inattentiveness in��at��ten��tive?adj.Exhibiting a lack of attention; not attentive.inat��ten in the classroom, 28(3), 199-201. Snell Snell, George 1903-1996.American geneticist. He shared a 1980 Nobel Prize for discoveries concerning cell structure that enhanced understanding of the immunological system, resulting in higher success rates in organ transplantation. , J.C. & Mekies, S., Replaceable Professors: An observation, 28(2), 1112. Stevenson, J.M. & Goldenberg, I.I., Educational policy practice: Facing competition against complacency com��pla��cen��cy?n.1. A feeling of contentment or self-satisfaction, especially when coupled with an unawareness of danger, trouble, or controversy.2. An instance of contented self-satisfaction. , 28(1), 59-60. Supon, V., The Student Teacher Literacy Project, 28(2), 113-116. Vodanovich, S.J. & Piotrowski, C., Internet-based instruction: A national survey of psychology faculty, 28(4), Wilder, D.A., Flood, W.A., & Stromsnes, W., The use of random extra credit quizzes to increase student attendance, 28(2), 117-120. Williams & Alawiye, Assessment lessons learned from a year-long undergraduate teacher education pilot program, 28(4), Winter, E. C. & Kilpatrick, R. Special needs resource roles: A cross-jurisdictional comparison, 28(1), 61-7. Young, C.Y. & Wright, J.V., Mentoring: The components for success, 28(3), 202-6. Janet L. Kottke, Department of Psychology, California State University Enrollment . Correspondence concerning this article should be addressed to Janet L. Kottke, Department of Psychology, California State University, 5500 University Parkway, San Bernardino San Bernardino, city, United StatesSan Bernardino(săn bûr'nədē`nō), city (1990 pop. 164,164), seat of San Bernardino co., S Calif., at the foot of the San Bernardino Mts.; inc. 1854. , CA 92407-2397; or e-mail jkottke@csusb.edu.