Adapting your thesis assignment to GenAI
What are you going to read?
The thesis assignment is one of the classical capstones of educational programmes, measuring a student’s ability to collect and synthesise information, as well as to articulate a critical perspective. The advent of Generative AI (GenAI) has put pressure on the validity of this assessment form, however. In this article, we will break down which changes you can consider for your thesis assignment, to be better equipped for the challenges offered by GenAI. The main points of consideration are:
- The student’s learning process: How can you guide and support students during the thesis process so that they gain authentic and valuable learning experiences?
- Valid assessment: How do you ensure that your assessment demonstrates the learning objectives validly and reliably, in the face of potential GenAI use?
- Weighting and grading: How does GenAI affect the grading process?
After reading this article, you’ll know how to make GenAI agreements explicit, focus on the process and ask for matching evidence, and rethink your rubric towards reflection, validation, and reasoning. In the end, the thesis can remain a valuable learning experience for students, testing and honing their ability to process complex information.
Introduction
In both bachelor and master programmes, the final thesis is supposed to be an aptitude test; it captures a comprehensive process of research in which knowledge and academic skills come together and are applied into one integrated whole. Students demonstrate critical thinking skills, gain research and problem-solving skills and develop their academic voice. In addition, working on a thesis challenges their organisational and planning skills, helping them become better project managers. While all these elements could be trained and assessed using different forms, the thesis neatly brings them together and can provide tangible evidence that a student is ready for the professional field.
The thesis is, however, an example of a “Lane 2” assessment form, in which students work mostly unsupervised. This means that they can, if they decide to do so, use GenAI tools in their ideation, analysis and writing processes. This in turn has two distinct, but related consequences:
- GenAI use can disrupt the learning process. Tasks that in the past would bring students to hone particular cognitive skills, now may be delegated, causing these skills to remain untrained.
- GenAI use can lower the test validity of the thesis as an assessment form. If the thesis is used to measure a student’s capacity to structure a text, but that student uses GenAI for this task, the thesis is no longer a measure of student proficiency.
To deal with this, a multi-pronged approach is necessary, ranging from rethinking intended learning outcomes (ILOs) to changing grading criteria and from the structure of the thesis assignment to the way students are supervised and coached.
Analyse your ILOs
Any course (re)design starts with a critical reflection on its ILOs. What are the goals the module should be setting? Can these be formulated as active, demonstrable measures of behaviour shown by the student? Adapting a thesis assignment to GenAI may involve the removal or addition of ILOs.
For each ILO of your thesis assignment, consider the following:
- Why is it in place? How does it contribute to the programme’s goals?
- Does your thesis assignment indeed contribute to student learning in order to achieve the outcome?
- Could a student use GenAI to demonstrate achievement of the ILO? Would that be problematic?
- If so, what happens if you remove the ILO altogether?
For example, a thesis assignment may have an ILO that states that students ‘can clearly structure their ideas in the form of a journal article’, which contributes to the programme’s exit qualification about scientific communication. Structuring ideas has become trivial with the help of GenAI, so a student could indeed use the technology to (seemingly) achieve the ILO.
It is a judgment call whether this is problematic. You could argue that a student using GenAI is still demonstrating their ability to structure a text, albeit with the use of GenAI. You could also argue that supplying a well-structured text was supposed to be a proxy for thinking about the subject material, prioritising certain arguments over others, considering the audience to explain a point, etc. In the former case, you could decide to keep the ILO (although you might want to alter the weight of its assessment, see below). In the latter case, you could decide to drop the ILO and replace it by something that more closely (and validly) captures the cognitive processes you are interested in.
If you drop an ILO, do consider how this affects the overall setup of the curriculum. Is a similar ILO being assessed elsewhere in the curriculum? Could it? It is good practice to discuss the removal of an ILO with the relevant programme director before making a decision.
The possibility of GenAI use may also warrant the addition of ILOs. This may be as a substitution of ILOs you decided to remove, or because GenAI brings in new requirements for students.
For example, thesis assignments might benefit from new ILOs such as:
- The student can justify key decisions in their research design
- The student can transparently report on GenAI use in the writing of their thesis
- The student can critically discuss the content of their thesis
- The student displays a curious and critical attitude during the development of their thesis
Note that these additional ILOs may also require the addition of new assessment forms, such as oral defences or process assessment.
Process, rather than product
GenAI allows students to generate deliverables, but may also see them bypass the learning process you are trying to achieve. By emphasising this process over the final product, you can mitigate the risk of GenAI causing students to miss out on valuable learning opportunities. This goes both for learning activities and assessment.
For example, rather than discussing the thesis itself at various stages of writing, you can use meetings with student to discuss what they have been working on or what was challenging for them. This can be student-centered, by discussing exactly those tasks that the student finds difficult. It can also be content-centered, for example by guiding students through critically reviewing research papers. Emphasising process means focusing your supervision on the how of thesis-writing.
This approach changes the calculus for students in two ways. First of all, to the extent that overreliance on GenAI is motivated by feelings of uncertainty on how to go about, process guidance can make such unwanted GenAI use less likely. Secondly, if the discussions are at a proper (challenging but doable) level, they provide an incentive for the student to remain engaged with the content, so that they come to meetings prepared.
Assessment-wise, you could opt to grade particular elements of the meeting(s). The extent to which a student displays original thinking, critical questioning, meeting preparation or any other aspects you deem important can be graded on the basis of the meetings. Some care should be taken when structuring this: a student should not feel like every move they make is being judged, and there should be room for error and learning, too. However, assessing process can allow you to measure exactly those cognitive or affective processes you may no longer infer from the thesis itself.
In case meetings are sparse, or it proves difficult to have supervisors monitor the process reliably, you can also ask students to submit regular reflection reports. In these reports, students describe how their research is progressing, which problems they encounter, and how they are solving them. These reports can then be starting points for progress meetings.
Think of questions such as:
-
- How do you connect the objectives of your thesis with your personal values and plans for the future?
- What impact do you want your research to have, both on yourself and on academic and wider society?
- What research skills are you currently and how will they help you in your further career?
- Which literature and knowledge inspired you most during your research and why?
- What did you discover about yourself in terms of strengths and growth opportunities during this process?
One thing to be mindful of is that this approach only works if students are clearly instructed on what the reflection assignment entails, there are not many similar assignments throughout the programme (otherwise some form of “reflection fatigue” will set in) and the reports are actually put to use, either in conversation or via written feedback.
Similar to reflection reports, these can be elements that prompt discussion during progress meetings, but they require less additional work from the students. The idea behind work documents is that they provide tangible evidence of the process, so that you can discuss it, give suggestions and simultaneously assure that the learning process is how you want it to be. Think of logs, annotation lists of literature read, draft versions of their written work, datasets, or mind maps.
Students are not always sure whether they can or cannot use GenAI for a particular task. Make sure your course manual lists what you find acceptable use cases for your thesis assignment, and where you would draw the line. You can also ask students to offer the same transparency in return, by requiring them to disclose their GenAI use. Advanced students can also be asked to justify their decisions about GenAI use. Disclosure reports measure compliance with a norm, while justifications provide a measure of GenAI literacy.
Some things to note:
- Disclosure reports can become an administrative burden for both students and teachers, especially if there are many different use cases, prompts and GenAI tools used in the project. Rather than asking students to document everything, you can also ask them to pick two instances of GenAI use that worked particularly well or did not work at all, and elaborate on those.
- If you choose to assess GenAI disclosure or justification, you may inadvertently set a norm for GenAI use. After all, a non-user will have nothing to disclose and may feel left out of an assessment opportunity.
- Assessing GenAI disclosure/justification also implies that you set a corresponding ILO and teach students how to reach that ILO.
- To ensure honesty in reporting, grading should not be negatively affected by the type of disclosure. If they disclose undesirable use that is still something to discuss with them, but penalising that by lowering their grade will incentivise secrecy.
- Disclosure statements are controversial, see for example this case against using them.
Beyond the thesis
If the thesis no longer suffices as a valid assessment form for your particular ILOs, you can also choose to complement it by “Lane 1” assessment — assessment that is supervised. These additional assessment forms may cover one or more ILOs that are no longer validly aligned with only the thesis.
You can complement the thesis by having students present their research question and findings to a panel of teachers. Teachers question the students about their findings in a dynamic interaction, to see whether the student has mastered the learning objectives. You can also consider presenting in symposium form, or giving a presentation in the company where they worked on their research. More information about oral assessments is found in this article.
When using this approach, focus the grading criteria on behaviour you can measure while being present with the student. For example, quality of slide decks is determined by work done outside of your supervision, but Q&As give a good view of what students can do without GenAI use. Sometimes, ILOs that you originally formulated for a written thesis can be translated to this oral setting. For example, the ILO “can provide a clearly structured text” is mostly valuable because it measures the ability to provide cogent argumentation, which could also be operationalised as an oral explanation.
Depending on the exact constructive alignment of your course, the oral assessment does not need to carry a lot of weight grade-wise. The master Forensic Science, for example, is piloting an interactive oral assessment that students join after a symposium presentation of their Literature Thesis. In this oral assessment, the examiner and assessor of the project ascertain the student has thorough understanding of the thesis content and then give a pass/fail grade that cannot be compensated. They can do this because there are not ILOs that are exclusively measured by the oral assessment.
Rather than a thesis, you can also opt to make the end-product ‘form-free’, where students choose for themselves how they can demonstrate competencies. This makes the assessment form more meaningful to the student, can boost motivation, and takes into account the context in which the student learns (is this an internship, research project, part of programming education, etc.).
The way to do this is to have student and examiner agree at the start of the course what would be a good measure for each ILO of the module, and what the corresponding deliverable would look like. Only if both parties agree can the student set out to demonstrate the ILOs.
If a particular ILO of your thesis assignment needs to remain in place, but can no longer be validly assessed by the thesis alone, you can also add an in-class writing assignment to the module, if appropriate for the particular ILO. This in-class writing assessment could measure the ability to structure a text or summarise a paper, but could also look at more high-level skills. For example, the research master Brain and Cognitive Sciences contains a Literature Thesis in which one of the ILOs asks students to synthesise sources from the academic literature. To assure that students can do this without using GenAI, they are piloting in-class assessment in which students receive research papers (disclosed only at the moment of assessment) and are tasked to evaluate and synthesise them.
Change grading criteria and assessment structure
When changing ILOs or focusing on process, the assessment criteria themselves should also shift.
Text structure, spelling, tone of voice: all of these are easily deployed to GenAI. You may still want to clarify their importance by grading them, but in Lane 2 (unsupervised) assessment you should not weigh these elements heavily. You can also opt to not grade them at all, or to set them as a precondition for assessment in the first place (a go/no-go moment).
If, as suggested above you put more emphasis on the process than on the product, you may find that you wish to assess affective criteria, such as curiosity when studying materials, engagement with tasks or listening and responding during meetings. These may traditionally not have been present in your rubrics or grading criteria, but adding them sends a clear signal that process itself is important.
These affective criteria may fit easily with the ILOs you already have, but may also warrant formulation of affective ILOs. Please visit the Visible Learning Trajectory website to find more information about affective learning goals and how they might look in practice.
According to Boud and Cohen (2014), peer learning is about sharing knowledge, ideas, and experiences between students, which is of added value for both parties. This creates a human connection in a thesis project that can sometimes feel alone. Think about how you will include this in your assessment, but also think about the application of peer learning in the project, for example, when there are feedback moments, how and when do students work together (e.g. in ‘graduation groups’) and when peer learning is or is not supervised. For a convenient integration of peer learning, use the Peerceptiv tool.
Thesis assignment for large cohorts of students
Many of the above tips can also be applied when you have a large cohort of students, but some do not: you want to find a balance between ensuring the reliability and validity of the learning objectives and managing the workload for teachers (and students). Here are some extra tips for supervising large groups of students in the thesis process:
- Make sure you have a rubric that all your supervisors can work with. Consider calibration sessions or training, or info sessions for both internal and external supervisors.
- A larger cohort lends itself well to peer feedback or peer review. This can be done under the supervision of TAs or junior teachers, with the examiner as the person ultimately responsible.
- Add some kind of oral assessment, which doesn’t always have to be at the end of the process. Also, an oral assessment does not have to take long.
- You can randomly check some (interim) products for improper use of GenAI. Be transparent about this with your students.
- See if more emphasis can be placed on the process versus the product in thesis assignments carried out by external companies/supervisors. Think of making interim products or conversations more formal to find out what the student’s contribution is.
Finally
Supervising and assessing the thesis process are an essential part of academic learning. This is not only about awarding a final grade that is as fair and careful as possible, but especially about the broader role of the teacher and supervisor. Your added value lies not only in transferring knowledge, but also in personal guidance, giving targeted feedback, brainstorming about different choices, human interaction, and acting as a role model. This makes a difference in the education of students, far beyond the assessment alone.
So get started right away. Start by making explicit agreements about the use of GenAI in your thesis assignment. Revise your rubric to put more emphasis on reflection and validation, and schedule regular check-ins to monitor your students’ progress. By following these steps, you can ensure that the thesis remains a valuable and authentic learning journey for your students.
Would you like more information about unsupervised writing assignments? Then read more in this TLC article.
Or would you like more information about assessment in general? Then check the TLC Assessment page for more information.