Are you looking to develop questions that go beyond recall for your Science and Engineering class sessions? Would you like to explore poll question formats other than multiple choice to expand the ways students can engage with course content? Do you want to assess in real-time how well all your students are answering questions to inform your next instructional decision?
In this guide, we share how you can develop and incorporate poll questions into your classroom that engage students in higher-order cognitive processes, such as applying concepts or evaluating hypotheses, to assess your students’ understanding. This guide combines the cognitive framework Bloom’s Taxonomy and the audience response system Poll Everywhere that is available to Columbia instructors and licensed by CUIT.
This guide is particularly useful for Science and Engineering instructors who are interested in using or are already using Poll Everywhere. Columbia instructors looking for Poll Everywhere support can contact the CTL Learning Designer assigned to their department or school. To learn more about audience response systems in general and how you can use them in your teaching, please refer to our Audience Response Systems resource.
On this page:
- Why Poll?
- Using Bloom’s Taxonomy to Target Different Cognitive Processes
- Asking Higher-Order Questions and Getting Real-Time Feedback with Poll Everywhere
- Step-by-Step Guide to Developing Higher-Order Poll Questions
- Accessibility Considerations
- Final Tips
- Example Lesson Scenarios
- Additional Resources
- References
Cite this resource: Columbia Center for Teaching and Learning (2020). Developing Poll Questions to Engage and Assess Student Thinking in Science and Engineering Courses. Columbia University. Retrieved [today’s date] from https://ctl.columbia.edu/resources-and-technology/resources/poll-questions-stem/
Why Poll?
Polling is an active learning technique that can be used to engage students in thinking about course content as well as assess their opinions, knowledge, and/or skills in real time and with low or no stakes.
Polls can serve a wide array of purposes in your course, including:
- Engaging students in problem-solving, analysis, idea generation or refinement, or metacognition individually or in pairs/small groups before polling them for their responses.
- Assessing student knowledge or understanding in or after class to determine whether to review course material, employ a different pedagogical approach, or progress to the next learning objective.
- Gathering student perspectives on a certain concept or idea and using the results of the poll to engage students in a class-wide discussion.
- Enabling students to learn more about each other, find commonalities with their peers, and contribute to a collaborative learning environment.
- Encouraging students to relate course content to their interests and lived experiences by crowdsourcing examples from their daily lives.
- Soliciting feedback from students on ways to enhance the course to further their learning.
Polls can be used on their own or as part of a larger in-class activity, such as a discussion or case study. Methods of implementing polling in the classroom range from having students raise their hands to vote for an option to having students use handheld devices to submit responses to open-ended questions and upvote responses from peers.
While there are many ways to use polls to engage and assess students, this guide focuses on developing poll questions to engage students in higher-order cognitive processes (e.g., analyzing arguments, generating ideas) through a variety of poll types (e.g., open-ended, clickable image) and the discussions they spark.
Using Bloom’s Taxonomy to Target Different Cognitive Processes
Remembering information is an important part of the learning process. Having students recall course content they encountered previously improves their ability to retrieve that information again in the future by strengthening its association with contextual cues1. In addition to recall, there are other important cognitive processes that students need to practice in order to gain expertise in their discipline2,3. For example, students in a physics class should be able to analyze a given problem for the physics principles that are needed to solve it. This act of analyzing is a higher-order cognitive process that students can perform and practice as part of an in-class poll.
To develop poll questions that require students to do more than remember previously encountered information, we turn to a useful framework called Bloom’s Taxonomy4. Originally developed in the 1950s to serve as a common language for discussing learning objectives and to guide their development, Bloom’s Taxonomy has also been used to create questions that promote the use of cognitive processes other than recall2,5.
In 2001, an updated Bloom’s Taxonomy was published to reflect contemporary understanding of cognition and learning4. This revised Bloom’s Taxonomy comprises two dimensions—the knowledge dimension and the cognitive process dimension. The knowledge dimension is divided into four types—Factual, Conceptual, Procedural, and Metacognitive (see Table 1)—and is loosely arranged from concrete to abstract4. The cognitive process dimension is divided into six categories—Remember, Understand, Apply, Analyze, Evaluate, and Create (see Table 2)—and is loosely hierarchical in terms of increasing complexity4. For this guide, we consider cognitive processes other than Remember as higher-order cognitive processes.
Table 1: The Knowledge Dimension of Bloom’s Taxonomy4
| Knowledge Type | Definition | Example |
| Factual | Specific pieces of information | The names and structures of the four different types of DNA nucleotides |
| Conceptual | Relationships among elements within a larger structure/context | The relationships between DNA, RNA, and proteins in gene expression |
| Procedural | Methods and processes | The process for predicting the amino acid sequence based on a DNA strand |
| Metacognitive | Knowledge of cognition in general and personal cognition | The efficacy of retrieval practice when studying for a biology test |
Table 2: The Cognitive Process Dimension of Bloom’s Taxonomy4
| Cognitive Process | Definition | Possible Action Verbs6,7 | Example Question |
| Remember (i.e., Recognizing, Recalling) |
Recall relevant information | Define, Find, Identify, Label, List, Match, Name, State, Write Down / Type | State Newton’s First Law of Motion |
| Understand (i.e., Interpreting, Exemplifying, Classifying, Summarizing, Inferring, Comparing, Explaining) |
Construct meaning from material | Classify, Compare, Describe (in your own words), Deduce, Discuss, Explain, Express, Share (your own example), Paraphrase, Predict, Summarize | Share an example from your daily life that demonstrates Newton’s First Law of Motion. |
| Apply (i.e., Executing, Implementing) |
Carry out or use a procedure in a given situation | Apply, Calculate, Determine, Estimate, Perform, Plot, Solve, Use | Use Newton’s laws of motion to calculate the additional force that you would need to exert on the block so that it does not accelerate. |
| Analyze (i.e., Differentiating, Organizing, Attributing) |
Deconstruct ideas and find connections among different parts with respect to an overarching structure | Analyze, Deconstruct, Differentiate, Distinguish, Examine, Organize, Outline, Structure | Of the pieces of information given, distinguish the ones that are necessary to solve the given problem. |
| Evaluate (i.e., Checking, Critiquing) |
Make judgments based on criteria and standards | Assess, Appraise, Check, Choose, Critique, Evaluate, Grade, Inspect, Judge, Rank, Score, Select | In the following sample response from a hypothetical student, check the problem-solving steps to identify if and where the student went wrong. |
| Create (i.e., Generating, Planning, Producing) |
Construct (or reorganize elements into) a new structure / product | Adapt, Build, Brainstorm, Compose, Construct, Create, Design, Develop, Generate, Hypothesize, Propose | Design a simulation in 2D that shows the net force on a point object when two or more forces are acting on it. |
The value of using Bloom’s Taxonomy to guide the development of poll questions lies not only in recognizing the different cognitive processes and knowledge types, but also in intentionally engaging students in practicing these cognitive processes and assessing their performance.
In addition to describing the cognitive processes of Bloom’s Taxonomy, Table 2 also lists corresponding action verbs that you may find helpful for developing or categorizing questions. These verbs do not necessarily have to be used within the question itself, but can instead be used to explain your rationale behind the question. An example question for each cognitive process is given in the right-most column.
While higher-order questions may require students to use multiple cognitive processes, it is helpful to focus on the main cognitive process that you want students to develop. To further illustrate how different questions can be designed to target different cognitive processes in Bloom’s Taxonomy, consider the following examples:
A multiple-choice poll question that asks students to select the correct definition of a word from a list of definitions could belong to the category Remember (recalling a previously seen definition), whereas a multiple-choice poll question that asks students to select the best description of a concept from a list of descriptions with nuanced differences could belong to the category Evaluate (determining the best description based on criteria).
An open-ended poll question that asks students to state the boiling point of pure water at standard atmospheric pressure focuses on the ability of students to recall that piece of information. This question thus addresses the cognitive process of recalling, which belongs to the category Remember. Alternatively, an open-ended poll question that asks students to design an experiment to determine the boiling point of pure water at standard atmospheric pressure focuses on the ability of students to produce new or original work. This question thus addresses the cognitive processes of planning and design, which belong to the category Create.
In addition to considering the cognitive processes, it is also helpful to consider what types of knowledge the poll question requires your students to work with. You may find that some knowledge types pair more readily with some cognitive processes than others. The cognitive processes of Remember, Understand, and Apply typically involve factual, conceptual, and procedural knowledge, respectively.
While it is certainly possible for students to Analyze, Evaluate, or Create any of the knowledge types, you may find that you want your students to Analyze, Evaluate, or Create based on, in terms of, or using the different knowledge types instead. Here are some examples:
- Structure (Analyze) an argument for or against genetically-modified foods based on what you’ve learned about CRISPR technology (factual and conceptual knowledge)
- Critique (Evaluate) the design of this car engine in terms of efficiency (conceptual knowledge)
- Write (Create) a program to display the current temperature using for loops (factual, conceptual, and procedural knowledge)
- Develop (Create) your study plan for the final exam based on effective study strategies (metacognitive knowledge)
One knowledge type that often gets overlooked is Metacognitive knowledge, which is knowledge about one’s own thoughts and cognitive processes as well as the cognitive regulation involved in directing one’s learning. For example, asking students to reflect on which aspect of a question they struggled with most can help them identify the gaps in their current knowledge or skills. If you’d like to learn more about Metacognition and how to teach both for and with metacognition, please refer to the CTL’s Metacognition resource.
Assess your understanding of Bloom’s Taxonomy!
|
What levels of Bloom’s Taxonomy do the following questions address? The boiling point of water in a beaker was measured to be 98°C.
Bloom’s level(s) addressed: ________________________________ |
Asking Higher-Order Questions and Getting Real-Time Feedback with Poll Everywhere
While multiple choice polls can be designed to target some of the higher-order cognitive processes8,9,10, some questions benefit from other types of polls, such as those that allow students to submit free responses or click on an image.
Poll Everywhere is an online tool available to the Columbia community that allows an instructor to electronically—through the use of web-enabled devices, such as a phone, tablet, or laptop computer—collect responses from all of their students and present the results back to them in real-time.
Table 3 lists some basic types of polls that Poll Everywhere supports. For a quick graphical overview of these poll types and how to create them, refer to Poll Everywhere’s Getting Started guide.
Table 3: Basic Poll Types in Poll Everywhere
| Poll Type | How Students Respond | How Poll Results are Presented |
| Multiple Choice | Students select from a provided list of options | Aggregate |
| Rank Order | Students rank the provided list of options | Aggregate |
| Clickable Image | Students click on pre-selected areas or anywhere on a given image | Individual or Aggregate |
| Word Cloud | Students submit a word or short phrase to produce a graphic of the most-used words | Aggregate |
| Open-ended | Students respond freely in text and all responses appear on screen | Individual |
| Q&A | Students respond freely in text, with the option of up- and down-voting any submitted response | Individual / Most popular |
With these different poll types, Poll Everywhere gives you more flexibility when developing poll questions to engage students in using higher-order cognitive processes. In addition, the responses you get from your students in real-time can be highly informative. Consider highlighting individual or aggregate responses with students to give them an idea of how their peers think. Depending on how students respond, you can follow up the poll by having students volunteer to share their thought process, engage in peer instruction11, begin an in-depth class-wide discussion on the topic, or review a concept before moving on to new topics.
For example, a Word Cloud poll question at the start of a class session asking students to recall previously-encountered ideas and concepts that relate to the day’s topic can be a useful way to activate their prior knowledge and make connections to new material. You can refer to concepts and ideas shared on the Word Cloud and ask students to elaborate on the connections to the day’s topic to check their understanding.
With this real-time assessment data, you can make more informed pedagogical choices to best meet the needs of your students and engage them in higher-order cognitive processes.
Step-by-Step Guide to Developing Higher-Order Poll Questions
Now that you are familiar with Bloom’s Taxonomy and Poll Everywhere, here is a step-by-step guide you can use to develop and effectively implement your own higher-order poll questions (Figure 1). Each step is followed by reflection questions you can use to help plan your approach. We begin by focusing on the desired concept and cognitive process before considering the appropriate poll type.
Figure 1: Step-by-step process to creating questions in Poll Everywhere that target higher-order cognitive processes
1. Select Concept
Begin by selecting a single concept that you want students to learn within a single class session. For poll questions, prioritize conceptual knowledge that would benefit from synchronous instructor-student and student-student interactions over concepts that students can more easily pick up on their own. For example, a poll question that addresses a common misconception could generate responses that lead to a robust discussion of the concept among students. Depending on the complexity of the concept, you may have to break it down into more manageable components for students to work their way through.
Reflection questions:
- What concept do my students have the most difficulty with or require the most support with?
- What related knowledge and skills would help my students master this concept?
2. Select Bloom’s Cognitive Process
Review the cognitive process dimension of Bloom’s Taxonomy (Table 2) and select one cognitive process that you would like your students to use on your chosen concept in a single class session. Selecting the cognitive process early will help focus the development of the poll question(s). Referring to your course or module learning objectives that apply to this concept can help inform your decision.
Important factors to consider when selecting a cognitive process include:
- the level of your course
- the class session duration
- how much time you will allocate to the poll (at least five minutes)
- student familiarity with the material and disciplinary skills
- students’ prior experience using different cognitive processes in your course.
Higher-order cognitive processes typically involve a wider knowledge base and more complex skill sets. To help students successfully engage with these cognitive processes, provide your students multiple opportunities to expand and deepen their knowledge base and hone specific skills. As your students progressively work their way through these opportunities, they will be more prepared to respond to your higher-order poll questions.
Reflection questions:
- For your chosen concept, what cognitive process would most benefit from your students practicing it with peers and in your presence?
- What prior experiences with this cognitive process would students have had before this class session?
3. Develop Poll Question
Here are two possible pathways for developing poll questions to target a higher-order cognitive process. You can either begin with the concept itself or use existing questions that already address this concept as a starting point.
3a. Build directly from concept
Flesh Out Concept
Now that you have selected a concept and the cognitive process you want your students to use, flesh out the concept to clarify aspects that you could base your poll question on.
Questions you can use to guide this process include:
- What elements are present in this concept, and how do they relate to one another?
- What possible ways can this concept be used?
- Apply the concept to an unfamiliar context?
- Use the concept to explain an observation?
- Compare this concept with a different one?
- View the concept from a different perspective?
Table 4: Possible aspects of your selected concept to consider and the cognitive processes they may pair well with
|
Possible aspects of your selected concept |
Cognitive processes they may pair well with |
|
Understand |
|
Apply, Evaluate, Create |
|
Analyze |
You can also consider related knowledge types (factual, procedural, and metacognitive) from the knowledge dimension of Bloom’s Taxonomy (see Table 1).
- What related factual knowledge do you want your students to remember?
- What related procedural knowledge do you want your students to apply?
- What related metacognitive knowledge do you want your students to discover?
Create question targeting concept and cognitive process
Now that you’ve fleshed out the concept, you can begin drafting poll questions based on the different aspects of the concept that you identified and your desired cognitive process. Consider focusing on the aspects that students tend to have the most difficulty with or require the most support with.
For example, if students are having difficulty determining when to apply your selected concept, they could benefit from practicing the cognitive process Evaluate with respect to the underlying assumptions for your chosen concept. In response, you could develop a poll question that describes a scenario and asks students to suggest the concept they think best applies and why.
Depending on the cognitive process that you selected, you should also consider the underlying knowledge and cognitive processes that students need in order to successfully answer your question. This will allow you to scaffold the process for students using complementary poll questions or other activities.
For example, students first need to Understand how an equation is related to its roots before they can Apply appropriate techniques to solve an equation for its roots and explain (Understand) what the results mean. A corresponding series of poll questions could be:
- Describe in your own words how an equation is related to its roots. (Open-ended poll)
- What are the roots of the following equation? (Multiple choice poll)
- What technique did you use to find the roots of the equation? (Q&A poll)
- Explain how the roots you found are related to the equation. (Open-ended poll)
Depending on how students respond to each poll question, you could intersperse within these poll questions a variety of learning activities such as having students explain to a partner the steps they took to find the roots of the question or small group discussions to discuss the pros and cons of each technique for finding the roots of the given equation.
As a reminder, poll questions that focus on the cognitive process of Remember can be used to prepare students for answering questions that target higher-order cognitive processes.
3b. Build upon existing questions
Categorize existing questions
Existing questions can be a useful starting point in developing higher-order questions for polling. Begin by listing questions you have already created that address your chosen concept. Next, categorize each question using Bloom’s Taxonomy by determining the main cognitive process it requires students to perform. You may find that some questions require students to engage with multiple cognitive processes. If you do, note these cognitive processes as you may later need to break down such questions into sub-questions should you decide to use them for polling. Although the time needed for each poll is dependent on how you use it in your course, you would generally need to break down big questions into smaller component questions so that students would be able to answer each in approximately five minutes.
Now that you’ve categorized your existing questions, note the cognitive processes that they address. Do these align with your course learning objectives?
Create complementary question targeting cognitive process
Depending on the existing questions you have, you may find that you already have questions that target your desired cognitive process for your students.
If you do, your next step is to consider if the question is sufficiently focused such that it can be used directly as a poll question, or if you need to break a complex question down into a set of more focused poll questions that build upon each other.
For example, the question “How would you design a water filtration system optimized for a sample of muddy water?” requires students to use the cognitive process Create on the concept of optimization. While this question might be too complex to be used as a single poll question, it could be broken down into more focused poll questions that may involve other cognitive processes, including:
- What factors will you need to consider when designing this system? (open-ended)
- How would you prioritize the following design constraints? (rank order)
- Given these five filters and their costs, which two would you select for your filtration system? (multiple choice)
- What outcome variables will you use to evaluate your water filtration system? (word cloud)
- What tradeoffs might you have to make when designing this system? (open-ended)
A culminating activity could be a minute paper for students to reflect on the optimization choices they made when designing their water filtration system. You can always collect these responses using an open-ended poll to review later without having to show responses to the class.
If you do not already have a poll question that targets your desired cognitive process, your next step is to find an existing question that you can build your question upon.
Often, questions that focus on the cognitive process Remember can be complemented with subsequent questions that require students to engage higher-order cognitive processes. For example, a recall question on the definition of Newtonian fluids (Remember) could be followed by video clips or animations of different fluids in motion and asking students to judge whether each fluid exhibits Newtonian behavior and explain their reasoning (Evaluate).
Likewise, a question that targets a different higher-order cognitive process may be complemented by a question targeting your desired cognitive process. For example, a question that requires students to apply Poiseuille’s Law (Apply) to find the change in flow rate of blood through a dilated vessel could be preceded by a question asking students to describe in their own words the significance of Poiseuille’s Law (Understand).
Reflection questions:
- How well does your poll question engage your students in your selected cognitive process?
- What types of responses / discussion do you expect from students when they answer your poll question?
4. Create Poll
Once you have developed the higher-order question, select the poll type that best suits the selected cognitive process and would give you the type of reponses you want from students (Table 5).
Table 5: Basic Poll Everywhere poll types organized by how students respond and how poll results are presented
| Poll Types | How Poll Results are Presented | ||
| Aggregate | Individual | ||
| How Students Respond | Predetermined options |
Multiple Choice Rank Order Clickable Image |
Clickable Image |
| Open-ended |
Word Cloud Clickable Image |
Open-ended Q&A* Clickable Image |
|
Note: For short descriptions of these poll types, refer to Table 3.
*Q&A includes the option for students to up- and down-vote any submitted response.
For example, if the question asks students to each generate their own hypothesis to explain a given phenomenon (shown in a demo, video, etc.), either the Open-ended or Q&A polls could be appropriate. Alternatively, if a question asks students to list the names of concepts relevant to analyzing a given scenario or solving a given problem, both the Q&A (with its up- and down-vote feature) and the Word Cloud polls could be more effective than an Open-ended poll.
You may find that you need to modify the wording of your questions in order for them to best fit the poll types you choose. For example, when using the Clickable Image poll, the question “Trace the path of the light ray from the lens to the screen” could be clarified to “Click the spot where you predict the light ray will reach the screen.” Consider consulting a colleague to ensure that what you’re asking students to do in your questions is clear.
For resources focused on developing multiple-choice questions in general, please refer to the Additional Resources section.
Reflection questions:
- What response format (multiple choice, single word, phrase, sentences, location) and visual representation (aggregate, individual, most popular) would be most effective for your purpose?
- What additional instructions would students need in order to answer the poll question?
5. Implement Poll
Consider the most effective moments during the class session to ask the poll questions. Polls do not have to be standalone and can be used to complement other learning activities in the class.
For example, a poll question targeting the cognitive process of Analyze could be asked at the start of the lesson to focus your students as they learn about the different factors that impact an outcome, e.g., Which of the following factors do you think has the largest impact on the rate at which the patient receives nicotine from the nicotine patch? This same poll question could be asked again at the end of the lesson to assess how student thinking changed after learning more about the topic.
Alternatively, a poll question that focuses on the cognitive process of Apply could be used immediately after introducing a new problem-solving strategy, e.g., Find the roots of the following equation. This gives students the opportunity to work through applying the strategy and self-assess their current ability to do so.
Reflection questions:
- At what point during the class session would asking the poll question be most effective?
- What preparation would students need prior to answering the poll question?
6. Use Student Responses
It is important for students to see that answering the poll question is not just an assessment but also serves as a learning activity. After your students have responded to the poll, be transparent with how you will use the data to inform your next pedagogical moves.
Here are some examples of how you can use student responses:
- Launch a mini review to address misconceptions and other mistakes highlighted in the poll responses
- For poll responses that indicate widespread disagreement, have students partner up to try to convince each other / share their thought process before re-polling
- Highlight interesting individual or aggregate responses to spark a class-wide discussion
- Provide aggregate feedback to students based on their responses (e.g., if you answered “b,” please be sure to read Chapter 5.3 of the textbook or watch this video.)
Reflection questions:
- How might you use the poll results to adjust your teaching and support student learning?
- What additional support and resources can you offer or refer students to based on their individual or aggregate responses?
Accessibility Considerations
Regardless of whether you are using polls in your in-person or online classroom, you should consider ahead of time reducing any barriers for students to fully participate in your poll.
For example, responding to Poll Everywhere polls requires students to each have access to a web-enabled device such as a phone, tablet, or laptop computer. To ensure that all students are able to participate in the poll, check with your school or department for devices that students may borrow prior to an in-class session and that the classroom has adequate wifi connection.
If you are teaching via video conferencing, consider the time needed for students to switch between the video conferencing and polling software. This is especially the case for students who are using their smartphones to do both.
Having students download and log in to the Poll Everywhere app ahead of time could help minimize the need to troubleshoot technology during any class session.
Other good practices to reduce barriers include reading the question and options out loud in addition to displaying the question, and giving students sufficient periods of silence to think through and respond to the question.
Reflection Questions:
- How will you ensure that all students are able to access and respond to your polls?
- What is your contingency plan for any student who is unable to access and respond to your polls?
To learn more about making your class session accessible to all learners, please refer to our Accessibility in Teaching and Learning resource.
Final Tips
- Creating an inclusive classroom environment is essential for encouraging all students to actively participate in polls and the discussions they spark. Strategies include learning student names, providing students with opportunities to get to know you and each other, and collaboratively establishing community agreements about class participation and discussions. To learn more about creating an inclusive learning environment, please refer to our Guide for Inclusive Teaching at Columbia
- Let students know that poll questions are great opportunities for them to practice higher-order cognitive processes with the support of you and their peers as well as an opportunity to self-assess their knowledge and skills.
- Be sure to create other opportunities for your students to practice and demonstrate these cognitive processes inside and outside the class sessions, including on homework assignments, projects, presentations, reflections, and exams.
Example Lesson Scenarios
The following example scenarios show how you can incorporate different types of poll questions targeting a variety of cognitive processes from Bloom’s Taxonomy into your science and engineering classroom:
Each example scenario includes the instructor’s goals for the class session and how the poll questions are implemented within it.
Additional Resources
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into practice, 41(4), 212-218.
- This article provides an overview of the revised Bloom’s taxonomy described in Anderson and Krathwohl (2001).
Revised Bloom’s Taxonomy. (n.d.) Center for Excellence in Learning and Teaching. Retrieved April 27, 2020, from https://www.celt.iastate.edu/teaching/effective-teaching-practices/revised-blooms-taxonomy/
- This resource provides examples of each cognitive process-knowledge type pairing.
Resources on developing multiple-choice questions
Bowen, J.A. (2012). Teaching naked: how moving technology out of your college classroom will improve student learning. San Francisco, CA: Jossey-Bass
- Pages 155-160 of this book provides examples of how certain multiple choice questions can be used to engage different cognitive processes by purposeful wording of the questions.
Butler, A. C. (2018). Multiple-choice testing in education: Are the best practices for assessment also good for learning?. Journal of Applied Research in Memory and Cognition, 7(3), 323-331.
- This resource shares best practices for developing multiple choice questions.
Burton, S. J. (1991). How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty. Brigham Young University Testing Services and The Department of Instructional Science.
- This guide shares how to prepare multiple-choice questions, including ones that target higher-order cognitive processes.
Paniagua, M. A., and Swygert, K. A. (Ed.) (2016). Constructing Written Test Questions For the Basic and Clinical Sciences (4th ed.). National Board of Medical Examiners (NBME), Philadelphia, PA
- This comprehensive manual focuses on developing questions for basic and clinical sciences, including the use of clinical vignettes to test application of knowledge.
References
- Karpicke, J. D. (2012). Retrieval-Based Learning: Active Retrieval Promotes Meaningful Learning. Current Directions in Psychological Science, 21(3), 157–163.
- Lord, T., & Baviskar, S. (2007). Moving students from information recitation to information understanding-Exploiting Bloom’s Taxonomy in creating science questions. Journal of College Science Teaching, 36(5), 40.
- Zoller, Uri. (1993). Are lecture and learning compatible? Maybe for LOCS: Unlikely for HOCS. Journal of Chemical Education, 70(3), 195. https://doi.org/10.1021/ed070p195
- Anderson, L.W. and Krathwohl, D.R. (2001). A Taxonomy for learning, teaching, and assessing : a revision of Bloom’s Taxonomy of educational objectives, Abridged Edition. Boston, MA: Allyn and Bacon.
- Crowe, A., Dirks, C., & Wenderoth, M. P. (2008). Biology in bloom: Implementing Bloom’s taxonomy to enhance student learning in biology. CBE-Life Sciences Education, 7(4), 368–381.
- Armstrong, P. (n.d.). Bloom’s Taxonomy. Center for Teaching, Vanderbilt University. Retrieved April 27, 2020, from https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/
- Verbs for Bloom’s Taxonomy. (n.d.). Eberly Center for Teaching Excellence, Carnegie Mellon University. Retrieved April 27, 2020, from https://www.cmu.edu/teaching/resources/Teaching/CourseDesign/Objectives/BloomsTaxonomyVerbs.pdf
- Butler, A. C. (2018). Multiple-choice testing in education: Are the best practices for assessment also good for learning?. Journal of Applied Research in Memory and Cognition, 7(3), 323-331.
- Cheesman, K. L. (2009). Writing / Using Multiple-Choice Questions to Assess Higher-Order Thinking. College Science Teachers Guide to Assessment. Chapter 5 . National Science Teachers Association (NSTA) Press, Arlington, Virginia.
- Burton, S. J. (1991). How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty. Brigham Young University Testing Services and The Department of Instructional Science.
- Crouch, C. H., & Mazur, E. (2001). Peer Instruction: Ten years of experience and results. American Journal of Physics, 69(9), 970–977. https://doi.org/10.1119/1.1374249