Do Multiple Choice Questions Lead to Long-Term Retention?

Shari Fisher Hinds, RN, MSN, FNP

With the increasing trend of medical students using question banks as a study method, there is a concern that the multiple choice question might not be producing long-term retention of knowledge. The “Educational Advisory Panel I: Multiple Choice Questions for Learning: A Critical Analysis of Effectiveness,” held May 14 at the AUA 2021 Annual Meeting, explored the effectiveness of multiple choice question methods for learning and whether it is really as effective as believed.

Keith Baker, MD, PhD, from Massachusetts General Hospital, moderated the panel and the discussion began with a presentation by Ann Harman, PhD, Chief Assessment Officer of the American Board of Anesthesia (ABA).

In 2014, Dr. Harman began the process of reevaluating the Maintenance of Certification Program’s personalized web-based platform used to keep medical knowledge current. Her hope was to create a next-generation continuing education program.

In 2016, the ABA launched MOCA Minute®, a longitudinal assessment that replaced the previous exam. Research has shown that using MOCA Minute® yielded higher test scores and resulted in less disciplinary actions against diplomats who used it for continuing education in their specialty of practice. The MOCA Minute® assessment tool provides a knowledge assessment report to show areas of weakness, as well as a CME explorer to find the most appropriate area for continued learning. This platform uses 3-option, single answer, multiple choice questions that are designed to be relevant to the diplomats’ medical practices. It is designed to promote lifelong learning by using principles of the Adult Learning Theory.

The next presenter Randall Schell, MD, Professor of Anesthesiology at the University of Kentucky, discussed the use of question banks, and whether they are a fad or a paradigm shift. The popularity of using question banks has increased significantly with medical students because it provides quick and efficient exposure to information. He stated that 80% of anesthesia residents use question banks more than 50% of their overall study time. The concern with this type of learning is that there seems to be more dispersed knowledge and a lack of deep understanding of concepts. His suggestion was for teachers to help learners build skills of integration and connection, so that the concepts learned from question banks can be deepened for better understanding.

Lastly, Edward Nemergut, MD, of the University of Virginia, discussed his thoughts on deep versus superficial learning. The concern is that the use of multiple choice question banks uses generative retrieval, which is the acquisition of factual knowledge, and may not provide long-term retention. In order to gain a deep understanding of a concept, there must be opportunities to use creative thinking, as well as a chance to evaluate, analyze and apply concepts, he explained. Superficial learning does not get to this level, and stops with a basic understanding and remembering, or memorizing, of the information. In order for question banks to be effective and enhance deep learning, they must stimulate higher order thinking and create links so that dispersed knowledge can be sewn together.           

Question banks and multiple choice testing methods don’t appear to be losing popularity, so finding ways to gain the most benefit from them is key, the panelists asserted. Helping learners build the skills of integration and connection can help link the puzzle pieces together, providing a deeper understanding of concepts in return.