Getting off to a good start: Student-generated multiple-choice questions as a way to facilitate learning in Pediatric Orthopedic Surgery
This project proposes that students are capable of creating questions that are as good as questions written by the teacher and that these questions can play an important part in pre-course preparation by enhancing student learning, and engagement.
Course and context
Since 2015 I have been the coordinator and teacher for a 4-day course in Pediatric Orthopedic surgery for residents in Orthopedic surgery (SK-course) financed by the National Board of Health and Welfare. The residents have in average approximately 4 years of clinical practice since they became licensed physicians.
Previous generations of students have, as a mandatory part of their preparation for the course, a month before the course starts, received an email with 10 questions (designed by me) related to the topic of the course. The intention of the questions is not primarily to test the level of knowledge, but to challenge and support the students to understand key concepts that represent important ideas that I want them to retain after the course. The questions are deliberately challenging and students who are not prepared are not likely to answer well enough.
The aim of this project was to compare student’s subjective perception of the quality of questions written by the teacher with questions written by students. The second aim was to evaluate if answering questions before the course is helping the students in their preparation. The third aim was to test if PeerWise is a suitable online platform to answer and rate questions.
Material and Methods
PeerWise allows students to answer a database of multiple choice questions. Once the students have answered a question, they will be given feedback on their answer, as well as an explanation (1-3).
The participants in the projects were 35 students attending the 2016 year’s 4-day course in Pediatric Orthopedic surgery for residents and three orthopedic residents doing their clinical rotation at the Department of Pediatric Orthopedic Surgery, Astrid Lindgren‘s Hospital.
The project had seven defined stages:
- The 35 students of the course held in March, 2016 answered the 10 questions that I sent them as a mandatory preparation for the course.
- In the end of the course the students answer the question “The questions was a help in the preparation for this course”.
- At the end of the course, when students had knowledge about the content of the course and the learning outcomes, they were asked to create multiple choice questions and provide answers. The students wrote in total 30 questions.
- The questions (and an explanation to the correct answer written by me) was entered in PeerWise. This created a database of questions that can be accessed online.
- Three orthopedic residents, doing their clinical rotation at the Pediatric Orthopedic department, Astrid Lindgren‘s Hospital, received an email with instructions how to access the 30 questions in PeerWise.
- The three residents answered and commented on the questions in the PeerWise database and rated them according to difficulty and quality, thus ensuring the questions had face validity.
The residents comment the questions in the PeerWise database and rated them according to difficulty and quality.
The residents furthermore rated the 10 mandatory questions written by me and used in the first stage of the project according to difficulty and quality
The residents also evaluated the online platform Peerwise answering the question “Using PeerWise to answer and rate questions was easy?” using a seven-point Likert scale (1=strongly disagree, 7=strongly agree).
- The best 10 questions according to quality were kept in the revised version of the question database in PeerWise.
The primary outcome of the project was to select the 10 best questions out of the 30 student- generated questions and to compare the quality of these questions to the 10 teacher-generated questions. The Mann Whitney U-test demonstrated no statically significant difference in the quality of questions generated by the teacher or the students [a median of 3.7 student, interquartile range (3.7-4.0) and a median of 4.0 teacher, interquartile range (3.9-4.5) (P = .089). The 10 student-generated questions were nearly identical to the teacher-generated questions with respect to the level of difficulty.
Students used a 7-point Likert scale to rate the usefulness of answering questions before the course started. On average, this exercise was rated as very useful for their course preparation [median = 6.5, interquartile range (6.0-7.0)].
Three residents, who were former students, were asked to respond to the following item, “Using Peer-Wise items to answer and rate questions was easy?”, using an another 7-point Likert scale (range from 1=strongly disagree, 7=strongly agree). On average, these residents reported a 5/7.
This project supports the concept of using student-generated questions as a way to activate students.
The benefit of student-generated questions in PeerWise is that it enables students to decide which subjects to focus on, how to phrase their questions and see which questions others have created. The concept of writing and answering Peer-wise questions also supports” learning by teaching” by encouraging students to focus on course content and extract the most relevant items. The rating of student-generated item difficulty and quality illustrates to students that what items they create are visible and of interest to others.
However, it is important that students are provided sufficient time to answer and prepare questions. Thus, I recommend beginning the course with a half day for reading and answering questions so that students take the task seriously. Answering the 10 questions in PeerWise is a mandatory preparation for the 4-day course in Pediatric Orthopedic surgery for Orthopedic Surgery Residents in 2017 and these students will be preparing new questions in the end of the course for the following course as a part of their examination.
This project has a number of strengths. PeerWise assessment is free and very easy to use. Students are presented with a simple, intuitive interface and instructors can easily view student content and monitor participation. The project had specific, measureable outcomes and the study group consisted of real students.
It was not the intent of the project to evaluate whether or not to author and answer multiple-choice questions, is associated with enhanced academic performance. Although a limitation is the small sample of residents evaluating the questions. However, the residents evaluating the questions were all participants of the course 2016.
In conclusion, the project demonstrates that there was no difference in the quality of the 10 best student generated questions compared to the teacher-generated questions and confirmed that the students perceived answering questions as part of course preparation as helpful, and had a positive experience of using PeerWise.
Biggins D, Crowley E, Bolat E, Dupac M, H D. Using Peerwise to Improve Engagement and Learning. The European Conference on Education 2015:1-16.
McQueen HA, Shields C, Finnegan DJ, Higham J, Simmen MW. PeerWise provides significant academic benefits to biological science students across diverse learning tasks, but with minimal instructor intervention. Biochem Mol Biol Educ. 2014;42(5):371-81.
Walsh JL, Denny P, Smith PE. Encouraging maximal learning with minimal effort using PeerWise. Med Educ. 2015;49(5):521-2.