1. Designing
Learning outcomes and assessment
Before you design your assessment, it can be helpful to ask yourself what goal your assessment serves. Do you want your students to practice? Help them stay on track? Or do you want to determine whether students have a sufficient grasp of the material? Your answer to these questions partly determines the requirements your assessment should meet.
The first step in designing effective assessment of learning is articulating what you want to achieve in your course. Limit yourself to goals you can actually measure: for instance, ‘raising interest for a subject’ is a legitimate goal for your teaching, but it’s impossible to measure. ‘Understanding of certain schools of thought in the history of film’ is something that can be measured. This is how to write measurable learning outcomes:
How to write good learning outcomes
Learning outcomes describe what you expect students to be able to do after completing your course. In principle, learning outcomes cannot be seen in isolation, because the combined learning outcomes of your programme should cover the exit qualifications for the degree. Even so, many instructors have the freedom to write their own course learning outcomes. An effective learning outcome specifies, as concretely as possible, what knowledge, understanding, attitude and/or skills the student must acquire during the course. If students are required to show certain behaviour, this is described in measurable activities.
Bloom’s taxonomy
The wording of the learning outcome should also indicate how you will assess whether the goal has been reached. A well-formulated learning outcome is concrete and describes measurable activities, for example: “Upon completion of this course, students will be able to explain the difference between X and Y.” Here is an example of an unmeasurable learning outcome: “Students will develop a sense of X.”
What do your students need for this?
Effective learning outcomes help you determine what you want to measure in your assessment. In turn, this helps you determine what students need to meet your learning outcomes. In short, before designing your assessment, you need to establish three anchor points: your intended learning outcomes, teaching activities, and assessment. This is known as Constructive Alignment.
Constructive Alignment
Constructive alignment model
Constructive Alignment is a design principle that can help you ensure your students’ learning activities are aligned with your intentions for the course. You can integrate this in your course design by considering:
- what you want your students to learn
- what they need to do in your course to learn this, and
- how you will assess their learning.
If you want to teach students to form an opinion, you can choose to practice debating during class. Your assessment should also measure students’ abilities to form an opinion. If your assessment measures something different, for instance factual recall, chances are high that students will memorize facts instead of practicing their debate skills, also during class. In other words, there should be a logical connection between the learning outcomes, teaching and learning activities and assessment methods in your course. Constructive alignment was developed by educational theorists John Biggs and Catherine Tang and is recommended by the UvA as teaching design principle.
Formative and summative
Some forms of assessment help students practice and learn – then they are formative, sometimes known as low-stakes or nongraded. Assessment that counts towards the final course grade is known as summative, or high-stakes assessment. It’s good to keep this distinction sharp in your course. Allocating grades to a mock exam or practice test does not encourage students to practice, as this is actually high-stakes assessment. Practicing means being allowed to make mistakes.
This form of assessment, also known as low-stakes or non-graded assessment, is primarily used as a self-evaluation tool, so students can gauge their progress at various points throughout their studies. Examples of formative assessment include homework assignments, a diagnostic test, or an outline or first version of a paper or thesis. Formative assessment contributes to the learning process, so it is at this stage that good feedback is most valuable; this is when students gain insight into what needs to happen before they can meet the intended goals. Sometimes formative assessment, such as a homework assignment, counts towards the final grade in order to motivate students to take it seriously. However, this frustrates the purpose of formative assessment, namely to practice. If you want students to practice, they should be allowed to make mistakes.
Choosing an assessment method
Once you have determined exactly what you want to measure, you can decide which activities best reflect whether students have reached the learning outcome. In other words, which assessment methods will you choose? Make sure you choose the appropriate cognitive level for your assessment. For instance, if the learning outcome involves ‘analyzing’, you won’t be asking factual recall questions. An important question is whether your assessment is feasible for you as an instructor – does it not mean too much grading?
Assessment quality standards
The final questions to consider before you begin designing your assessment have to do with quality: your assessment should be impartial and fair. For instance, is your assessment representative of the course material? In addition, students should know what to expect beforehand. If the exam situation is entirely unfamiliar to them, the results won’t be reliable. You should also consider whether your chosen assessment method is feasible for you as the instructor, but also for students. These quality standards for assessment – validity, reliability, feasibility and transparency – should be kept in mind as you design your assessment.
Validity can be defined as the extent to which assessment measures what it is meant to measure, or in other words, does the content of the assessment correspond to the intended learning outcomes? There are several aspects of validity to keep in mind. ‘Content validity’ means the assessment must cover the learning material. ‘Criterion validity’ means the assessment corresponds to the intended level, which can be seen when students score similarly on two assessments covering the same material. To get a sense of the validity of your assessment, it can be helpful to use an assessment planning chart, also known as a test blueprint or specification model. This charts course material or learning outcomes against assessment types or question complexity (usually following Bloom’s taxonomy). This helps you – and your colleagues and students – see at a glance whether all the course material has been covered and the level of complexity of the questions. Want to learn more about this?
Assessment is considered reliable if repeated testing yields consistent results. However, in university teaching, this idea of repeated testing is theoretical. Therefore, reliability can best be thought of as meeting a number of requirements. First of all, assessment should be objective and impartial. Questions (and answer options, in the case of multiple choice) must not be subject to ambiguity or confusion, and grading must be consistent (regardless of who is grading, for instance). In addition, assessment results should make a clear distinction between students who have mastered the material and those who have not. Finally, tests must consist of enough questions: scoring a high grade shouldn’t be a matter of luck.
In addition to quality standards for content, it’s important to keep in mind whether your assessment is feasible in practice, for both instructors and students. Make sure your chosen assessment methods fit in the hours you have been given for your course. For instance, it’s unrealistic to assign a long paper in a course with many students if you won’t have time to grade it. In addition, consider whether your course assessment is feasible for students. Don’t assess them more than you need to, and remember that students also have time commitments for other courses.
Make sure your students are adequately informed about what to expect in terms of assessment and how they will be graded. It is helpful, for instance, to provide representative sample questions, practice tests or mock exams. For exams or tests, make sure you include clear instructions, providing information about how many points each question is worth, required answer length, time limits, etc. For assignments, provide students with a clear deadline and explain what happens when deadlines are missed. All these points are usually covered in your course manual or syllabus.
Frequently Asked Questions:
| Assessment method | Suitable for | Comments | ||
| Exam with multiple choice / closed-ended questions | Remembering, understanding, sometimes applying | Efficient for large groups (>100)
Preferably online |
||
| Exam with open-ended questions | Remembering, understanding, applying | Inefficient for large groups (grading is time-consuming)
Online or on paper |
||
| Exam with longer open-ended questions | Applying, analyzing, evaluating, creating | Grading is time-consuming, especially for large groups
Online or on paper |
||
| Open book exam
(open-ended questions with access to course materials) |
Understanding, applying, analyzing | Not suited for at-home completion (risk of cheating) | ||
| Take-home exam (longer open-ended questions with access to course materials) | Applying, analyzing, evaluating | Suited for at-home completion: students have several hours or even days to complete the exam | ||
| Short written assignment | Applying, analyzing, evaluating | Often formative, for instance as preparation before class | ||
| Essay, paper, thesis | Applying, analyzing, evaluating, creating | Writing skills are often explicitly included in learning outcomes | ||
| Oral presentation | Applying, analyzing, evaluating, creating | Often combined with a written assignment or paper; speaking or writing skills may be explicitly included in learning outcomes | ||
| Oral exam | Remembering, understanding, applying, analyzing, evaluating | Efficient for smaller groups (<40)
Examining students individually is time-consuming, but there is virtually no grading time You can ask follow-up questions |
||
| Creative assignment: video, blog, website, poster, podcast, etc. | Applying, analyzing, evaluating, creating | Very suitable for presenting material/research to a non-academic audience | ||
| Portfolio | Higher cognitive levels, but also competences, attitudes, skills | Suitable for holistic assessment of learning outcomes and reflecting on the learning process |
Validity and reliability are both quality requirements for assessment. Validity refers to the extent to which assessment measures what it is supposed to measure. In practice, reliability is mostly concerned with the objectivity of assessment.
The Assessment Cycle
| Designing | How do I choose a form of assessment that accurately measures my learning outcomes? | |
| The next step: Constructing | How do I construct effective questions and assignments? | |
| Administering | What should I keep in mind while administering an exam? | |
| Grading | How can make sure my grading is efficient and reliable? | |
| Analyzing | How do I evaluate and improve assessment quality after the fact? | |
| Reporting | What should I keep in mind when returning grades and feedback? | |
| Evaluating | How do I improve my assessment next year? |