QuizCon: Multiple Choice Quizzing for Learning
QuizCon is a tool that was developed by the Columbia University Center for Teaching and Learning to extend the utility of online multiple choice quizzing. Using Confidence Weighted Multiple Choice questions, which is discussed in more detail below, QuizCon aims to improve the multiple choice quizzing experience for both students and teachers. If you are an instructor who uses multiple choice quizzing in your classes, you may want to consider using QuizCon as it can have a better effect in long term retention of information, provides students flexibility and the opportunity to earn partial credit, and allows for you, the instructor, to get a more accurate view of student understanding of the content.
This resource provides an overview of traditional multiple choice quizzing, highlights the benefits of confidence weighted multiple choice questions to student learning, and introduces QuizCon, which is a CourseWorks connected platform that facilitates quizzing using confidence weighted multiple choice questions.
Benefits of Multiple Choice Quizzes
Multiple choice quizzes can be an effective means to evaluate student learning, but they can also be used as a learning tool. Research has found that multiple choice quizzes can lead to better long term retention of information than simply reviewing material (Butler & Roediger, 2007), as well as prompt productive retrieval processes (Little, 2011). Specifically, when the multiple choice questions have competitive distractor options, and students take the time to evaluate all the different options for their answer, they are more likely to retain the information and even answer related questions correctly (Little & Bjork, 2015). Distractors are the incorrect choices beside the correct answer that are shown to students on the multiple choice question. When they are more competitive, it encourages students to evaluate between the different choices and spend time thinking about them, which leads to a more meaningful thought process in answering the question, thus they are more likely to retain the feedback that they will receive on that question after submission.
As an assessment tool, there is often a negative perception around multiple choice quizzing. Many believe multiple choice quizzing can only be used to evaluate knowledge and the memorization of facts and do not evaluate key critical thinking skills. However, well-crafted multiple choice questions that push students to apply or evaluate key concepts and ideas, can in fact access higher-order thinking (Snyder & Snyder, 2008; Kim, Patel, & Uchizono, 2012).
Introducing Confidence Weighted Multiple Choice Questions: How does it work?
Originally designed 1989 by James E. Bruno, a former professor from UCLA, Confidence Weighted Multiple Choice Questions use a triangle to create three spectrums of confidence level. This means that each question will have one correct answer choice, and two incorrect distractors. Each of these three options can be found at the corners of the triangles. The spectrum formed on each side of the triangle represents the level of confidence of choosing one option over the other.
If a student is confident in a specific answer, they can choose that option outright by selecting the corner of the triangle. If they are correct, they get full credit, but if they are wrong they get no credit (or even negative points, depending on the scoring scheme chosen).
However, if a student is stuck between two options A and B, they can choose on the side of the triangle how confident they are of one over the other. For example: if the student is leaning towards A, with some doubt that it may be B, they can choose a spot on the side of the triangle that is closer to A, but not at the corner. If the correct answer is indeed A, they would get most of the possible points for the question (e.g. 4 out of 5 points), but if they are wrong, they will receive partial credit (e.g. 1 out of 5 points), instead of no credit.
Lastly, there is an “I don’t know” option that students can select and still receive partial credit (or not receive negative points depending on the scoring scheme).
The original concept of the confidence-weighted triangle for multiple choice questions from Bruno (1989).
Added Benefits to Confidence Weighted Multiple Choice Questions
Confidence weighting brings the following benefits compared to traditional multiple choice questions.
- The “I don’t know” option discourages guessing (Sparck, Bjork, & Bjork, 2016).
- Confidence spectrums on the triangle makes it so that student responses can more accurately represent their understanding (Sparck, Bjork, & Bjork; Cisneros-Pahayahay & Pahayahay, 2017).
- Since student understanding is more accurate and student guessing is discouraged, as a teacher you get more accurate information and can identify misconceptions (Sparck, Bjork, & Bjork, 2016).
Research around confidence weighted multiple choice questions shows improved learning outcomes and that students, after an initial adjustment period, generally prefer them over traditional multiple choice questions. The benefits that directly impact students include the following:
An example of a confidence weighted multiple choice question, with consequence/negative points from Sparck, Bjork, & Bjork (2016).
- Students have noted that they like that these questions offer the opportunity to earn partial credit, and that the added flexibility can lower their anxiety levels (Sparck, Bjork, & Bjork, 2016).
- Students have also performed better on subsequent tests on the same content, whether they are also multiple choice or cued-recall questions, where students are given cues to help remember previously quizzed content (Sparck, Bjork, & Bjork, 2016).
- Students are more likely to retain information about distractor options after getting feedback, and consequently, be able to correctly answer related questions to those distactors (Sparck, Bjork, & Bjork, 2016).
What is QuizCon and How Can You Use It?
QuizCon is the first online tool to use Confidence Weighted Multiple Choice Questions. Designed to be practical for classroom use, it can be used as both a learning tool and as an assessment tool. You can use QuizCon as a learning tool to create knowledge checks throughout the semester that give your students the opportunity to practice their skills, improve knowledge retention, and give you insight into how your students are doing and any misconceptions they may have. QuizCon has a built-in auto-feedback feature that you can use to give timely feedback, and you can use the data from student performances to make adjustments for lesson planning. You can also use QuizCon as an assessment tool that provides a more reliable measure of students’ understanding and an opportunity for fair grading.
QuizCon quizzes are built and edited on the QuizCon website. QuizCon integrates with CourseWorks, so quizzes can be directly embedded into your CourseWorks site, graded automatically, and scores transferred to the Gradebook. An analytics page on the QuizCon website allows you to get an overview of class performance, including sorting to identify the questions with which students had the most difficulty.
You can choose from different scoring schemes (consequences) based on the goal of your quiz (whether it is for learning or assessment).
How Can I Add QuizCon to My Course?
QuizCon is available in all Columbia CourseWorks courses and can be easily activated from the course Settings. To access the Settings, simply go to the bottom of your Course Menu and click on the Settings option.
Once on the Settings page, click on the Navigation tab at the top. This will take you to a page where you can manage the items in your Course Menu. Scroll to the bottom of the page and locate the option called “Quizzing With Confidence”. Then, click the three vertical dots button on its right and select Enable. Finally, don’t forget to save your changes by scrolling to the bottom of the page and clicking the Save button.
After activation, you will now be able to access the “Quizzing With Confidence” tab through your Course Menu. Note that this tab is only accessible to Instructors and TAs as a tool to create quizzes. To import a quiz into your CourseWorks course, you can refer to the Help page on the platform for guidance.
Get Assistance or Set up a Consultation
If you find yourself in need of assistance with using QuizCon or would like advice on best practices in using the tool, you can contact us at quizcon@columbia.edu. You can either request an email reply, or set up a one-on-one consultation with one of the CTL Learning Designers.
General Assessment Resources
While multiple choice quizzes can be an effective means for evaluating student learning and can also be used as a learning tool, it is recommended that it is supplemented with other modes of assessment. To explore more of these options, please see our Resources for Assessing Student Learning resource.
References
Bruno, J. E. (1989). Using MCW-APM test scoring to evaluate economics curricula. Journal of Economic Education, 20, 5–22
Butler, A. C., & Roediger III, H. L. (2007). Testing improves long-term retention in a simulated classroom setting. European Journal of Cognitive Psychology, 19(4-5), 514-527.
Cisneros-Pahayahay, M., & Pahayahay, G. (2017). Level and Quality of Knowledge Using Confidence-Weighted NRET Scoring Method in Multiple Choice Test. Advanced Science Letters, 23(2), 885-889.
Kim, M. K., Patel, R. A., Uchizono, J. A., & Beck, L. (2012). Incorporation of Bloom’s taxonomy into multiple-choice examination questions for a pharmacotherapeutics course. American journal of pharmaceutical education, 76(6).
Little, J. L. (2011). Optimizing multiple-choice tests as learning events (Doctoral dissertation). University of California, Los Angeles.
Little, J. L., & Bjork, E. L. (2015). Optimizing multiple-choice tests as tools for learning. Memory & cognition, 43(1), 14-26.
Snyder, L. G., & Snyder, M. J. (2008). Teaching critical thinking and problem solving skills. The Journal of Research in Business Education, 50(2), 90.
Sparck, E. M., Bjork, E. L., & Bjork, R. A. (2016). On the learning benefits of confidence-weighted testing. Cognitive research: principles and implications, 1(1), 1-10.