Short Answer Questions: A Great Middle Ground

by Susan Codone

Short answer questions, no matter how well-formulated, cannot measure divergent thinking and subjective or imaginative thought.

Allan Ornstein, The ClearingHouse, 1992

Did he really say that?  Did Allan Ornstein really just savage a staple of the test arsenals of many college professors, the short answer question?  Stronger than multiple choice, yet not quite as revealing (or time consuming to grade) as the essay question, the short answer question offers professors a great middle ground – the chance to measure a student’s brief composition of facts, concepts, and attitudes in a paragraph or less.  In spite of what Allan Ornstein says, the short answer question is indeed capable of measuring divergent thinking and both subjective and imaginative thought and is used widely by many professors.

Leave it to Wikipedia to offer a pithy definition of the short answer question – couching it as an “extended-response question with more than one right answer” or “more than one way of expressing the right answer."  The University of Wisconsin Teaching Academy calls short answer questions “constructed response”, or “open-ended questions that require students to create an answer.”   The Center for Teaching Excellence at the University of Illinois at Urbana Champaign says that short answer questions allow students to present an original answer.  Going further, Karen Scouller of the University of Sydney, in an 1997 Advancing International Perspectives article, says that the greatest difference between multiple choice and short answer or essay questions is the measurement of surface vs. deep learning, indicating that longer-response questions tend to assess learning that is deeper and more tightly held in long-term memory.

Elizabeth Badger and Brenda Thomas, writing in Open-ended Questions in Reading, Practical Assessment: Research and Evaluation (1992), claim that a gradual change has resulted in teaching, with professors moving from teaching content alone to instead helping their students learn the ability to use and interpret knowledge critically and thoughtfully.  They recommend that assessment measure not just the product of learning (the content), but also the process students go through to acquire information.  Alexander Maxwell, a history professor from New Zealand, states in a 2010 article that methodological diversity in assessment is helpful, and says that in history, at least, the short answer question is very valuable (Assessment Strategies for a History Exam, or Why Short Answer Questions are better than In-Class Essays, The History Teacher).  The short answer question is well-positioned to capture both subjective and imaginative thought that occurs as students learn, as well as provide professors with a glimpse into the writing skills and idea expression of students.

Like all assessment items, a short answer question should clearly assess a specific learning objective.  It should ask students to select relevant facts and concepts and integrate them into a coherent written response. Question 1, below, is a typical example of a short answer question requiring such a constructed response.

This question, while a little long, sets up a scenario with an expert role, a community history, and an environmental problem and asks the students to use a specific problem-solving strategy -- the 4 A's (attain facts, alternatives, assessment, action) to frame a response, which can most likely be completed in 4-6 sentences, or one paragraph.

Question 2, below, is slightly more problematic because of a very common error in constructing short answer questions.

This question, while well-intended, actually asks two questions.  This likely will leave the student confused as to which question is more important.  Additionally, the student will have to write a longer response to answer both questions, leading this particular test question more toward an essay response than a short answer.  Short answer questions should always ask one clear question, rather than confusing the issue with multiple queries.

Finally, one strategy many professors use is to post a rubric or scoring guide in the test so that students will know how points will be distributed based on their answer.  Question 3, below, both shows such a rubric and demonstrates another common problem in short answer question development.

Note in this question, a scoring distribution is provided to the students -- not only do they know the question is worth six points, but they also know immediately that three points will be awarded for fully answering the question and two points for legibility, with the final point for spelling and grammar.  This tells them that not only do they need to answer the question completely -- they must also be neat and watch spelling and grammar.  Question 3 also demonstrates another common error -- writing questions that close off the extent of a student's potential answer.  Notice the second sentence -- "Do you think that two accidents..." -- what might the student's answer be?  Yes or No?  This is not the answer for which the professor is looking, but it might be the one given.  A better question would ask "How might two accidents be an acceptable level of risk...", in order to promote a more meaningful answer.

Ornstein (1992), who was more a proponent of essay questions than short answer in his article Essay Tests: Use, Development, and Grading, delineated a hierarchy of question verbs, as follows:

Type 1: Open  -- Why, How, Predict
Type 2: Directed -- Explain, Discuss, Examine
Type 3: Closed -- Identify, Compare, Contrast

According to Ornstein, Type 1 questions result in more meaningful answers, while Type 3 generate more specific, less creative responses.  So, whenever possible, even short answer questions should begin with an open verb, inviting students to be thoughtful and deliberative in their answers.

Short answer questions are a great middle ground for professors.  They are easier to develop than multiple choice and generate a more in depth answer as well.  Because of their brevity, they are easier to grade and they encourage student to integrate information into a coherent written answer, revealing much about what the student knows and how they express responses.  They can measure many types of knowledge when phrased correctly -- even divergent thinking and subjective and imaginative thought. Best of all, they can provide professors with a open window into student learning -- the real purpose of assessment.