Report on an E-Mail Survey
Concerning the Use of Non-Essay Assessment Methods
in Introductory Philosophy Classes
by Stephen Palmquist (stevepq@hkbu.edu.hk)
In May of 1996 I composed a simple survey concerning the suitability of using objective questions on final examinations for introductory philosophy classes. In hopes of collecting empirical data demonstrating a variety of views on this issue—data that could be presented during an upcoming departmental discussion—I submitted the survey to five electronic mailing lists (Philos-L, Philosop, Bridge-L, Kant-L, and the NAKS E-Mail Directory). Eugene Kelley encouraged me to submit a report of the survey results to this Newsletter. In the end, my report turned into a full-length article—too lengthy to be published here. The present essay is a summary of that article, intended to entice interested readers to consult the full treatment, to appear as “Philosophers’ Views on the Use of Non-Essay Assessment Methods: Discussion of an E-Mail Survey”, in volume 21 (1997-98) of Teaching Philosophy.
After a brief introduction, defining as “objective” any question of the “multiple choice, fill in the blank, matching” variety, the questionnaire requested yes-or-no responses to the following four statements:
1. I have been a student in an introductory (or other) philosophy class in which the teacher included objective questions on the final examination.
2. As a student, I believed that the only legitimate kind of final examination for any philosophy class (other than logic) would be an essay exam.
3. I have included objective questions on the final examination of an introductory (or other) philosophy class taught by me.
4. I believe it would be proper for a philosophy department to ban the use of objective questions on all final examinations other than those dealing exclusively with logic.
Respondents were encouraged to supplement their yes/no responses by explaining/defending their views, either immediately after each question or in a special section for comments at the end. The web page at http://www.hkbu.edu.hk/~ppp/srp/examquest.html has links to four files listing respondents’ comments in full (with names, addresses and other personal remarks deleted).
Over a period of approximately one week, 145 people submitted responses, representing universities in at least 15 countries. A summary of the 141 statistically relevant responses (excluding four that did not give a definite answer to any of the questions) is presented in the table below, which includes all responses with at least one definite answer. (“N/A” stands for “no answer” or “not applicable”.) Several interesting observations can be made about these statistics. First, respondents were evenly split on questions 1-3, showing the use of non-essay assessment methods is widespread, though a significant proportion of respondents (about half of those who gave yes/no answers) prefer not to use them. Second, the breakdown of the answers to question 2 (indicating how the respondents for each option answered question 1) shows that a large majority of those who rejected the validity of non-essay assessment methods had never experienced them as students, whereas most who experienced them thought they were acceptable. Third, a clear majority of respondents (76% of the 124 yes/no answers) rejected the idea of banning non-objective assessment methods (many adding strongly-worded statements of disapproval). The full article (mentioned above) discusses these and other points in more detail.
The supplementary comments were so interesting and varied that in a short report such as this I can do no more than summarize the most significant points. First, several respondents drew attention to ambiguities relating to my use of the term “objective”; these have been clarified in the article (e.g., by replacing “objective” with “non-essay”). Second, various respondents raised issues concerning the relativity of assessment methods to specific class objectives and to the general educational system: teachers who view the purpose of philosophy classes exclusively in terms of teaching reasoning skills are more likely to disapprove of non-essay assessment methods, as are those teaching in universities with a cumulative assessment system (having only one set of exams, at the end of a whole degree program); those who believe a certain amount of “basic knowledge” should be taught and who teach at universities using continuous assessment methods (having exams for each individual class) are more likely to accept the use of non-essay questions. Third, many well-reasoned arguments were given both for and against using non-essay assessment methods, though (as explained in the article) I believe the former are far more persuasive: several limitations of using essay exams were pointed out by those seeking to defend the legitimacy of using non-essay methods; some specific suggestions were also proposed for creative ways of designing non-essay questions in order to avoid the chief objection (namely, that they do not require a sufficient degree of reasoning skill to merit being called “philosophical”). Finally, two important provisos were mentioned by many who were otherwise skeptical about using non-essay questions: that such questions are acceptable for logic classes, and that short answer questions are a good compromise between full-length essays and other forms of “objective” question.
In conclusion, the responses ended up not only confirming my fundamental hypothesis, that a wide variety of views exists on the relevant issues, but also suggesting a number of surprisingly definite conclusions. In addition to the points mentioned above, for instance, there appears to be widespread agreement that if non-essay questions are used, they should not be the sole component of a final exam. These and numerous other conclusions are brought out in the more thorough discussion of the respondents’ comments contained in the article mentioned above, wherein I attempt to synthesize as many of the comments as possible so that interested readers may gain an accurate idea of the spectrum of possible views on this important pedagogical issue.
Stephen Palmquist
Department of Religion and Philosophy
Hong Kong Baptist University
This etext is based on a prepublication draft of the published version of this essay.
Send comments to: StevePq@hkbu.edu.hk
My Web Counter identifies you as visitor number
For the page of published articles (and all the articles linked thereto), use:
to this page, last updated on 1 June 2001. Please come again!
Back to the listing of Steve Palmquist's published articles.