Before you design your assessment, it can be helpful to ask yourself what goal your assessment serves. Do you want your students to practice? Help them stay on track? Or do you want to determine whether students have a sufficient grasp of the material? Your answer to these questions partly determines the requirements your assessment should meet.
The first step in designing effective assessment of learning is articulating what you want to achieve in your course. Limit yourself to goals you can actually measure: for instance, ‘raising interest for a subject’ is a legitimate goal for your teaching, but it’s impossible to measure. ‘Understanding of certain schools of thought in the history of film’ is something that can be measured.
Learning outcomes describe what you expect students to be able to do after completing your course. In principle, learning outcomes cannot be seen in isolation, because the combined learning outcomes of your programme should cover the exit qualifications for the degree. Even so, many instructors have the freedom to write their own course learning outcomes. An effective learning outcome specifies, as concretely as possible, what knowledge, understanding, attitude and/or skills the student must acquire during the course. If students are required to show certain behaviour, this is described in measurable activities.
Bloom’s taxonomy
The wording of the learning outcome should also indicate how you will assess whether the goal has been reached. A well-phrased learning outcome is concrete and describes measurable activities, for example: “Upon completion of this course, students will be able to explain the difference between X and Y.” Here is an example of an unmeasurable learning outcome: “Students will develop a sense of X.” Read more about how to write good learning outcomes here.
Effective learning outcomes help you determine what you want to measure in your assessment. In turn, this helps you determine what students need to do to meet your learning outcomes. In short, before designing your assessment, you need to establish three anchor points: your intended learning outcomes, teaching activities, and assessment. This is known as Constructive Alignment (Biggs & Tang, 2011).
Constructive alignment model
Constructive Alignment is a design principle that can help you ensure your students’ learning activities are aligned with your intentions for the course. You can integrate this in your course design by considering:
If you want to teach students to form an opinion, you can choose to practice debating during class. Your assessment should also measure students’ abilities to form an opinion. If your assessment measures something different, for instance factual recall, chances are high that students will memorize facts instead of practicing their debate skills, also during class. In other words, there should be a logical connection between the learning outcomes, teaching and learning activities and assessment methods in your course. Constructive alignment is recommended by the UvA as teaching design principle.
When we talk about assessment, we usually mean summative assessment. This kind of assessment counts towards the final course grade (also known as high-stakes assessment). But there are also forms of assessment that help students practice and learn – then they are formative, sometimes known as low-stakes or ungraded assessment. This form of assessment is primarily used as a self-evaluation tool, so students can gauge their progress at various points throughout their studies.
Examples of formative assessment include homework assignments, a diagnostic test, or an outline or first version of a paper or thesis. Formative assessment contributes to the learning process, so it is at this stage that good feedback is most valuable; this is when students gain insight into what needs to happen before they can meet the intended goals.
Clear distinction
It’s good to keep the distinction between formative and summative assessment clear in your course. Sometimes formative assessment, such as a homework assignment, counts towards the final grade in order to motivate students to take it seriously. However, this frustrates the purpose of formative assessment, namely to practice. If you want students to practice, they should be allowed to make mistakes, so they can learn and adapt their learning strategies in preparation for summative assessment.
Once you have determined exactly what you want to measure, you can decide which activities best reflect whether students have reached the learning outcome. In other words, which assessment methods will you choose? Make sure you choose the appropriate cognitive level for your assessment. For instance, if the learning outcome involves ‘analyzing’, you won’t be asking factual recall questions. As you choose which assessment methods to use, keep in mind the four quality standards for assessment: validity, reliability, feasibility and transparency (see below).
An additional aspect to consider is whether your chosen assessment method is susceptible to unauthorized use of genAI. Information and advice on safe assessment in times of genAI is available here.
Assessment method | Suitable for | Comments | ||
Exam with multiple choice / closed-ended questions | Remembering, understanding, sometimes applying | Efficient for large groups (>100)
Preferably digital (automatic grading) Not safe for remote assessment (high risk of cheating) |
||
Exam with open-ended questions | Remembering, understanding, applying | Inefficient for large groups (grading is time-consuming)
Digital or on paper Not safe for remote assessment (high risk of cheating) |
||
Exam with longer open-ended questions | Applying, analyzing, evaluating, creating | Grading is time-consuming, especially for large groups
Digital or on paper Not safe for remote assessment (high risk of cheating) |
||
Open book exam
(open-ended questions with access to course materials) |
Applying, analyzing | Suitable for assessing whether students are able to use course material to answer application or analysis questions.
Not safe for remote assessment (high risk of cheating)
|
||
Take-home exam (longer open-ended questions with access to course materials) | Applying, analyzing, evaluating | Risk of misuse of generative AI. | ||
Short written assignment | Applying, analyzing, evaluating | Often formative, for instance as preparation before class | ||
Essay, paper, thesis | Applying, analyzing, evaluating, creating | Writing skills are often explicitly included in learning outcomes | ||
Oral presentation | Applying, analyzing, evaluating, creating | May be combined with a written assignment or paper; speaking or writing skills may be explicitly included in learning outcomes | ||
Oral exam * | Remembering, understanding, applying, analyzing, evaluating | Efficient for smaller groups (<40)
Examining students individually is time-consuming, but there is virtually no grading time You can ask follow-up questions |
||
Creative assignment: video, blog, website, poster, podcast, etc. | Applying, analyzing, evaluating, creating | Very suitable for presenting material/research to a non-academic audience | ||
Portfolio | Higher cognitive levels, but also competences, attitudes, skills | Suitable for holistic assessment of learning outcomes and reflecting on the learning process |
The final questions to consider before you begin designing your assessment have to do with quality: your assessment should be impartial and fair. For instance, is your assessment representative of the course material? In addition, students should know what to expect beforehand. If the exam situation is entirely unfamiliar to them, the results won’t be reliable. You should also consider whether your chosen assessment method is feasible for you as the instructor, but also for students. These quality standards for assessment – validity, reliability, feasibility and transparency – should be kept in mind as you design your assessment. Read more about each of the four standards below.
You can also download a checklist with guiding questions for each of the four quality standards here (Word).
Validity can be defined as the extent to which assessment measures what it is meant to measure, or in other words, does the content of the assessment correspond to the intended learning outcomes? There are several aspects of validity to keep in mind. ‘Content validity’ means the assessment must cover the learning material. ‘Criterion validity’ means the assessment corresponds to the intended level, which can be seen when students score similarly on two assessments covering the same material. To get a sense of the validity of your assessment, it can be helpful to use an assessment planning chart, also known as a test blueprint or specification model. This charts course material or learning outcomes against assessment types or question complexity (usually following Bloom’s taxonomy). This helps you – and your colleagues and students – see at a glance whether all the course material has been covered and the level of complexity of the questions. Read more about assessment planning charts in stage two of the assessment cycle.
Guiding questions to check the validity of your assessment:
Assessment is considered reliable if repeated testing yields consistent results. However, in university teaching, this idea of repeated testing is theoretical. Therefore, reliability can best be thought of as meeting a number of requirements. First of all, assessment should be objective and impartial. Questions (and answer options, in the case of multiple choice) must not be subject to ambiguity or confusion, and grading must be consistent (regardless of who is grading, for instance). In addition, assessment results should make a clear distinction between students who have mastered the material and those who have not. Finally, tests must consist of enough questions: scoring a high grade shouldn’t be a matter of luck.
Guiding questions to check the reliability of your assessment:
In addition to quality standards for content, it’s important to keep in mind whether your assessment is feasible in practice, for both instructors and students. Make sure your chosen assessment methods fit in the hours you have been given for your course. For instance, it’s unrealistic to assign a long paper in a course with many students if you won’t have time to grade it. In addition, consider whether your course assessment is feasible for students. Don’t assess them more than you need to, and remember that students also have time commitments for other courses.
Guiding questions to check the feasibility of your assessment:
Make sure your students are adequately informed about what to expect in terms of assessment and how they will be graded. It is helpful, for instance, to provide representative sample questions, practice tests or mock exams. For exams or tests, make sure you include clear instructions, providing information about how many points each question is worth, required answer length, time limits, etc. For assignments, provide students with a clear deadline and explain what happens when deadlines are missed. All these points are usually covered in your course manual or syllabus.
Guiding questions to check the transparency of your assessment:
Designing | How do I choose a form of assessment that accurately measures my learning outcomes? | |
The next step: Constructing | How do I construct effective questions and assignments? | |
Administering | What should I keep in mind while administering an exam? | |
Grading | How can make sure my grading is efficient and reliable? | |
Analyzing | How do I evaluate and improve assessment quality after the fact? | |
Reporting | What should I keep in mind when returning grades and feedback? | |
Evaluating | How do I improve my assessment next year? |