For closed-ended questions, the correct answer is always given, so you have to take guessing into account: the probability that students can guess the right answer without any knowledge of the material. For a question with 4 answer alternatives, a passerby on the street without any subject knowledge will guess an average of 25% of the answers correctly (by choosing 1 out of the given 4 alternatives at random; 1/4 = 25%). These points are freebies, so to speak, and therefore say nothing about the extent to which students have mastered the material. This generates noise in the measurement of your learning objectives. To get a reliable picture of the extent to which students have mastered the material, there are two things you must do when using closed-ended questions: ask enough questions, and apply a correction-for-guessing formula.

How many closed-ended questions is enough?

You need to have enough ‘genuine’ answers left after correcting for the guess factor to get a reliable indication of whether the student has mastered the material. In general, the more questions the better, but there are also guidelines to ensure reliability. These assume a minimum of 160 answer options:

Question type	Minimum number of questions
Four alternatives	40
Three alternatives	54
Two alternatives	80

A multiple-choice exam with fewer questions than these minimal guidelines does not give a reliable indication of the extent to which the student has mastered the material.

NB: Sometimes there is reasoned deviation from these guidelines if the learning objectives in question are also assessed in other ways within the course, for example in the case of several partial exams where grades compensate, or if there are also a number of open-ended questions in the exam. Check the specific guidelines for your programme with the Examinations Board.

How do you correct for guessing?

Digital assessment programmes such as ANS Exam can automatically apply a correction-for-guessing formula. The guessing correction ensures that the score is based on demonstrable knowledge rather than chance. For an exam with 40 questions with four alternatives, the guess factor is 25%, so on average 10 questions will be guessed correctly. You therefore work out the grade over the remaining 30 questions. For a pass mark of 55% (standard in most faculties), that amounts to 16.5 questions. The student must therefore answer 10 + 16.5 = 26.5 of the 40 questions correctly to achieve a grade of 5.5.

Another example: for an exam with 100 questions with two alternatives, the guess factor is 50%. So you calculate the marks over the remaining 50 questions. To pass, you then need to answer 50 + 27.5 = 77.5 of the 100 questions correctly.

In practice, for digital assessments, the precise calculation is usually carried out automatically through Ans Exam, taking into account the type of questions and the applicable pass mark for the course.

It can be laborious to calculate the marks manually if you administered the exam on paper. The TLC has created an Excel document that allows you to convert students’ scores into grades, automatically calculating and correcting for the guess factor (download here). For this you need an overview of student numbers and scores on the exam, but of course you can also make a list of all possible scores to get a scoring table. There are instructions on how to fill it in at the top of the document.

Example (screenshot)

Increasing the required level of knowledge (pass mark)

Sometimes, instead of applying a guessing correction, you might want to simply raise the pass mark for the exam to make the required level of knowledge higher. For example, for an exam with 40 multiple-choice questions, you could set a pass mark of 70% instead of the usual 55% (without a guess correction). Students would then need to score 70% of the 40 points = 28 points to obtain a 5.5. This amounts to a slightly ‘stricter’ pass mark than the calculation example with guessing correction above (26.5 points = 5.5).

For exams consisting solely of one question type (e.g. only questions with four alternatives), this can provide a useful approach, but if your exam contains a variety of question types (e.g. a mix of questions with 2, 3 or 4 alternatives, or more complex question types), the above calculation does not take this into account. For example, with ranking, matching or hotspot questions in ANS, the chance of guessing correctly is almost zero. With fill-in-the-blank questions (where students must fill in a missing word), there is no chance of guessing at all. You can double-check the calculations to be sure, but applying a fixed higher pass mark that does not account for the specific probability of guessing correctly for the different question types often disadvantages students.

It is fairer to apply the guess correction on a case-by-case basis for each exam than to raise the pass mark across the board, as this ensures you are aiming for an equal level of knowledge for each exam. This reduces the likelihood of discrepancies between subjects within a degree programme, which can be confusing for students.

“Negative marking”

A common misconception among students is that guess correction as it is applied in the Netherlands is the same as “negative marking”, a strategy used to prevent students from guessing, which is often used in countries such as Belgium and the US. This is emphatically not the case: under the guess correction system as we know it in the Netherlands, students are not “punished” for guessing by deducting points.

Under the ‘negative marking’ strategy, students must determine for each question how certain they are of their answer. If they are unsure, they may choose not to answer that question. They receive 1 point for a correct answer, 0 points if nothing is filled in, and -1 point for an incorrect answer. This encourages students not to guess answers (this is usually also explicitly stated in the instructions).

However, the effectiveness of this is questionable because it causes stress, and how students cope with it is determined by personality and even gender: male students are more likely to guess, even when they have partial knowledge, meaning that you are not (only) measuring the intended knowledge, but also irrelevant factors. As a result, the grade gives a less reliable picture of what the student is capable of. (See, for example, Betts et al., (2009). Does correction for guessing reduce students’ performance on multiple-choice examinations? Yes? No? Sometimes? Assessment & Evaluation in Higher Education, 34(1), 1–15. https://doi.org/10.1080/02602930701773091)

Can you also choose to not take the probability of guessing into account for closed-ended questions?

This would mean the grades are not a reliable indication of what a student actually knows or is capable of. It is likely that virtually all students will pass, some of them undeservedly.

What is the policy on guess correction?

Most degree programmes require guess correction, for example using the ANS Exam calculation method or their own formula. Increasing the pass mark to raise the percentage of correct answers is less commonly used. Negative marking is not used at the UvA, as far as we know.

Please consult the Rules and Regulations of your programme’s Examination Board to check what applies to you.

The banner image for this article was created using generative artificial intelligence.