For closed-ended questions, you have to take into account the guess factor: the probability that students can guess the right answer without any knowledge of the material. For a question with 4 answer alternatives, a student without any knowledge will guess an average of 25% of the answers correctly. These points are given away to the student, so to speak, and therefore say nothing about the extent to which they have mastered the material. This generates noise in the measurement of your learning objectives. To get a reliable picture of the extent to which students have mastered the material, there are two things you must do when using closed-ended questions: ask enough questions, and apply a correction-for-guessing formula.

How many closed-ended questions is enough?

You need to have enough ‘genuine’ answers left after correcting for the guess factor to get a reliable indication of whether the student has mastered the material. In general, the more questions the better, but there are also guidelines to ensure reliability. These assume a minimum of 160 answer options:

Question type	Minimum number of questions
Four alternatives	40
Three alternatives	54
Two alternatives	80

A multiple-choice exam with fewer questions than these minimal guidelines does not give a reliable indication of the extent to which the student has mastered the material.

NB: Sometimes there is reasoned deviation from these guidelines if the learning objectives in question are also assessed in other ways within the course, for example in the case of several partial exams or if there are also a number of open-ended questions in the exam. Check the specific guidelines for your programme with the Examinations Board.

How do you correct for guessing?

Digital assessment programmes such as TestVision and ANS Exam can automatically apply a correction-for-guessing formula; otherwise you will have to do it manually. This correction for guessing works as follows. For an exam with 40 questions with four alternatives, the guess factor is 25%, so on average 10 questions are guessed correctly. You therefore work out the grade over the remaining 30 questions. For a 55% cut-off point, this is 16.5 questions. The student must then answer 10 + 16.5 = 26.5 out of 40 questions correctly to get a 5.5. Another example: for an exam with 100 questions with two alternatives, the guess factor is 50%. So you calculate the marks over the remaining 50 questions. To pass, you then need to answer 50 + 28 = 78 of the 100 questions correctly.

It can be laborious to calculate the marks manually if you administered the exam on paper. Fortunately, there are tools available to help you. The website cijfersberekenen.nl allows you to generate a scoring table for your exam. For the first example above, you would then choose the following settings (in Dutch): 5,5 voldoende; 40 punten (totaal te behalen); Normeringsterm: 27 punten non-lineair (you have to scroll quite a bit for this). The downside is that you still have to calculate the cut-off point yourself.

Example (screenshot)

The TLC has created an Excel document that allows you to convert students’ scores into grades, automatically calculating and correcting for the guess factor (download here). For this you need an overview of student numbers and scores on the exam, but of course you can also make a list of all possible scores to get a scoring table. There are instructions on how to fill it in at the top of the document.

Example (screenshot)

The banner image for this article was created using generative artificial intelligence.