QuizCon: Multiple Choice Quizzing for Learning
QuizCon is a tool that was developed by the Columbia University Center for Teaching and Learning to extend the utility of online multiple choice quizzing. Using Confidence Weighted Multiple Choice questions, which is discussed in more detail below, QuizCon aims to improve the multiple choice quizzing experience for both students and teachers. If you are an instructor who uses multiple choice quizzing in your classes, you may want to consider using QuizCon as it can have a better effect in long term retention of information, provides students flexibility and the opportunity to earn partial credit, and allows for you, the instructor, to get a more accurate view of student understanding of the content.
This resource provides an overview of traditional multiple choice quizzing, highlights the benefits of confidence weighted multiple choice questions to student learning, and introduces QuizCon, which is a CourseWorks connected platform that facilitates quizzing using confidence weighted multiple choice questions.
Benefits of Multiple Choice Quizzes
Multiple choice quizzes can be an effective means to evaluate student learning, but they can also be used as a learning tool. Research has found that multiple choice quizzes can lead to better long term retention of information than simply reviewing material (Butler & Roediger, 2007), as well as prompt productive retrieval processes (Little, 2011). Specifically, when the multiple choice questions have competitive distractor options, and students take the time to evaluate all the different options for their answer, they are more likely to retain the information and even answer related questions correctly (Little & Bjork, 2015). Distractors are the incorrect choices beside the correct answer that are shown to students on the multiple choice question. When they are more competitive, it encourages students to evaluate between the different choices and spend time thinking about them, which leads to a more meaningful thought process in answering the question, thus they are more likely to retain the feedback that they will receive on that question after submission.
As an assessment tool, there is often a negative perception around multiple choice quizzing. Many believe multiple choice quizzing can only be used to evaluate knowledge and the memorization of facts and do not evaluate key critical thinking skills. However, well-crafted multiple choice questions that push students to apply or evaluate key concepts and ideas, can in fact access higher-order thinking (Snyder & Snyder, 2008; Kim, Patel, & Uchizono, 2012).
Introducing Confidence Weighted Multiple Choice Questions: How does it work?
Originally designed 1989 by James E. Bruno, a former professor from UCLA, Confidence Weighted Multiple Choice Questions use a triangle to create three spectrums of confidence level. This means that each question will have one correct answer choice, and two incorrect distractors. Each of these three options can be found at the corners of the triangles. The spectrum formed on each side of the triangle represents the level of confidence of choosing one option over the other.
If a student is confident in a specific answer, they can choose that option outright by selecting the corner of the triangle. If they are correct, they get full credit, but if they are wrong they get no credit (or even negative points, depending on the scoring scheme chosen).
However, if a student is stuck between two options A and B, they can choose on the side of the triangle how confident they are of one over the other. For example: if the student is leaning towards A, with some doubt that it may be B, they can choose a spot on the side of the triangle that is closer to A, but not at the corner. If the correct answer is indeed A, they would get most of the possible points for the question (e.g. 4 out of 5 points), but if they are wrong, they will receive partial credit (e.g. 1 out of 5 points), instead of no credit.
Lastly, there is an “I don’t know” option that students can select and still receive partial credit (or not receive negative points depending on the scoring scheme).
The original concept of the confidence-weighted triangle for multiple choice questions from Bruno (1989).
Added Benefits to Confidence Weighted Multiple Choice Questions
Confidence weighting brings the following benefits compared to traditional multiple choice questions.
- The “I don’t know” option discourages guessing (Sparck, Bjork, & Bjork, 2016).
- Confidence spectrums on the triangle makes it so that student responses can more accurately represent their understanding (Sparck, Bjork, & Bjork; Cisneros-Pahayahay & Pahayahay, 2017).
- Since student understanding is more accurate and student guessing is discouraged, as a teacher you get more accurate information and can identify misconceptions (Sparck, Bjork, & Bjork, 2016).
Research around confidence weighted multiple choice questions shows improved learning outcomes and that students, after an initial adjustment period, generally prefer them over traditional multiple choice questions. The benefits that directly impact students include the following:
An example of a confidence weighted multiple choice question, with consequence/negative points from Sparck, Bjork, & Bjork (2016).
- Students have noted that they like that these questions offer the opportunity to earn partial credit, and that the added flexibility can lower their anxiety levels (Sparck, Bjork, & Bjork, 2016).
- Students have also performed better on subsequent tests on the same content, whether they are also multiple choice or cued-recall questions, where students are given cues to help remember previously quizzed content (Sparck, Bjork, & Bjork, 2016).
- Students are more likely to retain information about distractor options after getting feedback, and consequently, be able to correctly answer related questions to those distactors (Sparck, Bjork, & Bjork, 2016).
What is QuizCon and How Can You Use It?
QuizCon is the first online tool to use Confidence Weighted Multiple Choice Questions. Designed to be practical for classroom use, it can be used as both a learning tool and as an assessment tool. You can use QuizCon as a learning tool to create knowledge checks throughout the semester that give your students the opportunity to practice their skills, improve knowledge retention, and give you insight into how your students are doing and any misconceptions they may have. QuizCon has a built-in auto-feedback feature that you can use to give timely feedback, and you can use the data from student performances to make adjustments for lesson planning. You can also use QuizCon as an assessment tool that provides a more reliable measure of students’ understanding and an opportunity for fair grading.
QuizCon quizzes are built and edited on the QuizCon website. QuizCon integrates with CourseWorks, so quizzes can be directly embedded into your CourseWorks site, graded automatically, and scores transferred to the Gradebook. An analytics page on the QuizCon website allows you to get an overview of class performance, including sorting to identify the questions with which students had the most difficulty.
You can choose from different scoring schemes (consequences) based on the goal of your quiz (whether it is for learning or assessment).
How Can I Add QuizCon to My Course?
Contact the QuizCon team at email@example.com and provide the following information:
- Name of course
QuizCon will be added to your course menu within 24-48 hours. Once QuizCon has been added to your course, a CTL Learning Designer will assist you to get started with QuizCon and help you create your first quiz.
For more information about QuizCon, you can visit the website’s help page.
General Assessment Resources
While multiple choice quizzes can be an effective means for evaluating student learning and can also be used as a learning tool, it is recommended that it is supplemented with other modes of assessment. To explore more of these options, please see our Resources for Assessing Student Learning resource.
Bruno, J. E. (1989). Using MCW-APM test scoring to evaluate economics curricula. Journal of Economic Education, 20, 5–22
Butler, A. C., & Roediger III, H. L. (2007). Testing improves long-term retention in a simulated classroom setting. European Journal of Cognitive Psychology, 19(4-5), 514-527.
Cisneros-Pahayahay, M., & Pahayahay, G. (2017). Level and Quality of Knowledge Using Confidence-Weighted NRET Scoring Method in Multiple Choice Test. Advanced Science Letters, 23(2), 885-889.
Kim, M. K., Patel, R. A., Uchizono, J. A., & Beck, L. (2012). Incorporation of Bloom’s taxonomy into multiple-choice examination questions for a pharmacotherapeutics course. American journal of pharmaceutical education, 76(6).
Little, J. L. (2011). Optimizing multiple-choice tests as learning events (Doctoral dissertation). University of California, Los Angeles.
Little, J. L., & Bjork, E. L. (2015). Optimizing multiple-choice tests as tools for learning. Memory & cognition, 43(1), 14-26.
Snyder, L. G., & Snyder, M. J. (2008). Teaching critical thinking and problem solving skills. The Journal of Research in Business Education, 50(2), 90.
Sparck, E. M., Bjork, E. L., & Bjork, R. A. (2016). On the learning benefits of confidence-weighted testing. Cognitive research: principles and implications, 1(1), 1-10.