On 3 July 2024, the TLC Science organised a 2-hour hands-on workshop for Science lecturers in collaboration with TLC Central, TLC FGw and TLC FdR. The workshop focused on the impact of generative AI on assessment. How can you make sure your assessment is more AI-resilient? The participants listened to several student and lecturer examples, received didactical tips and tricks and, above all, got to work on their own assessment.
The primary concerns of the participants regarding GenAI and assessment were that the learning objectives are no longer being assessed, that students’ critical thinking is not being assessed, that students don’t learn to write, that students no longer learn anything, and that students don’t use formative assessment effectively.
With unsupervised written assignments, students can easily misuse GenAI. For example by following these steps:
If students do this, it is possible to get a high grade for the assignment in 30 minutes without learning much about the topic. Especially the most competent students figure out how to work well with GenAI and engage less and less with the actual content of the assignments.
Adapting your assignment to GenAI will require an investment in time and effort. Generally, no single solution will completely safeguard your assessment. It will always involve a mixed bag of solutions that depend on your specific situation.
Are you not sure whether your course assessment is vulnerable to unauthorised use of GenAI? Then you can first follow the steps from this checklist: Checklist for GenAI vulnerabilities in assessment at course level.
Broadly speaking, you have two options to make your assessment more AI-resilient:
See below for more information.
Students are generally not that interested in cheating, but in learning and preparing for their future. However, situations arise where students will look for shortcuts if they are not motivated to engage with an assessment, or they have other time pressures where they feel they have to prioritise one assessment over another.
Students are more inclined to do effortful work themselves if they are motivated and at ease. Below are some general tips to help you decide if any elements of your assessments should be adjusted to improve students motivation:
We need to reconsider whether unsupervised written assignments still fit the learning outcomes at a time when many students are using GenAI to help them.
In some faculties and programmes, the learning outcomes assessed in written assignments are primarily content-based, with less focus on the actual academic writing skills. In such cases, it might not matter if students have used GenAI to help them correct structural or language errors, as long as the ideas, results and analysis are their own (although this is by no means a certainty in times of GenAI, even if you monitor the research and writing process). In fact, some argue that GenAI can even help make writing assignments more equitable for students, by helping to level the playing field for students who are not writing in their first language, for instance.
However, this must be reflected in how you grade the assignment.
Tips & tricks:
However, be aware that future GenAI models are likely to also be able to do this. AI can understand context better and better, so don’t lean entirely on that.
A short term change you can look at is to adjust the weighting of different aspects in the rubric to emphasize those skills that GenAI can’t do well. For example, to put more emphasis on the higher-order cognitive skills such as analysis, argumentation, and use of sources, and to put less emphasis on the lower-order cognitive skills such as language, style, and lay-out. So the actual assessment criteria don’t change, but the emphasis and weight put on them does.
If you do this, always go back to your learning objectives. What are you assessing? If it is writing skills, it’s unlikely that an unsupervised assessment is the right way to assess this, due to how proficient GenAI is in this area. A solution would be to put more emphasis on the whole writing process, rather than just the end product.
Focusing on the process a student goes through for an assessment is a great way to safeguard your assessment. Here are some ways you can start to think about this:
At each step of the learning process, you can do one of two things:
Some additional ideas for how you can separate the learning process from the end product and incorporate tasks along the process path:
While assessment focused on writing can be an excellent method of assessing your students, go back to your learning outcomes and have another look: Are there other ways of assessing whether students have met them?
Varying assessment formats is advisable for equity reasons, and to ensure a more complete sense of whether the student has met the learning outcomes.
Possible alternative assessment formats:
However, some alternative options can still be impacted by the use of GenAI, so make sure you consider the learning process when constructing these assessments.
See below for the solutions that the participants came up with to make their assessment more AI-resilient. They indicated that they found the workshop especially useful for thinking critically about what they actually want from their students. What is the learning objective and how can it best be assessed?
Pay more attention to students’ intrinsic motivation. Because students feel more inclined to do effortful work themselves if they are motivated and at ease. For example, by giving them more freedom in the assignments.
Replace the final presentation with a defense to test their knowledge. In that case, students have to respond directly to questions from lecturers and/or students, which is more difficult to have generated by GenAI.
In the rubric of your course, give more weight to the final presentation (in particular the answers they give to the questions afterwards).
Replace the take-home writing assignment by a writing assignment in class. But be aware that in this case, you might (also) assess other skills of your students.
Offer more supervision during a long-term assignment. For example, twice a week instead of once a week.
Connect the assignment to their internship.
If you have any questions or are interested in this workshop, please send an email to tlc-science@uva.nl.