On January 16th 2023, the Programming Teaching Lab and the Teaching and Learning Centre Science organised a 4-hour ChatGPT workshop for lecturers. ChatGPT is a language model with chat capabilities and it seems like it can do almost anything: Answer all of your questions, come up with “new” ideas, write your essay for you, and whatnot. Lecturers all over the world shudder in fear: Can we still trust the content of our students or is everything generated by AI?
However, there are also a lot of opportunities with this new technique. During the workshop, we wanted to focus on the positives: What can lecturers take away from ChatGPT? How can we use ChatGPT to our own advantage? Should we think about new ways of assessing? Can it generate new assignments, embed its use into homework? We would argue that the sky is the limit here.
Jelle Zuidema, associate professor in Natural Language Processing, Explainable AI and Cognitive Modelling at the Institute for Logic, Language and Computation, explained that ChatGPT didn’t come out of the blue, but there has been a continuous development of natural language processing technology in recent years. There is no explicit source where it gets all its information from, but it makes abstractions from all the available online information. However, that doesn’t mean that the information it provides is always correct. It produces a lot of content that sounds right because it has learned the grammar and different types of writing genres and can reproduce those structures. However, the content can be (and regularly is) actually nonsense. As ChatGPT itself explains it:
It is a pure language model and produces content based on what are plausible continuations of text. It is trained on user data and wants to make its users happy (e.g. by providing short and clear answers), but it’s not optimised for truthfulness. For example, it can generate sources when the user asks for references because that will make the user happy, but those references can be completely made up.
But how reliable is the content in that case? How do you distinguish confabulations from facts? This is still an active field of research. A possible solution can be to first search for factual content with classical techniques (e.g., in books or with reliable online sources) and then use ChatGPT to summarise all the information from those sources. Unfortunately, asking ChatGPT in your prompt to “be truthful” won’t solve this problem. It might lead to a more truthful answer, but then that’s purely a coincidence.
But if producing truthful content isn’t what ChatGPT does, how can we use it in education? Martijn Stegeman, lecturer and coordinator of the UvA minor Programming, identified two main areas for which ChatGPT can be used in education:
Remember, ChatGPT is programmed to provide answers that make the reader happy. Vice versa, that means that asking the right questions – called prompts – is essential when using it. Therefore, Martijn provided guidance for writing good prompts. The most important thing is to not phrase the prompt too broadly or vaguely, but as precisely as possible. Your prompt should include all the criteria that you are looking for in the answer. For example, if you need questions that test whether your students can apply their knowledge, asking ChatGPT for 10 questions might result in questions of a quality you cannot use. However, if you ask ChatGPT to create 10 questions that require a relational response (as described in the SOLO taxonomy) it will adhere to those requirements (see images below).
Keep in mind, if there’s a lot of content available about the topic or if it is a common genre, ChatGPT’s output becomes better (e.g. ChatGPT is really good at creating a study guide entry based on a couple of your notes).
For more tips about writing good prompts for ChatGPT, you can read this Dutch article from Neuroflash.
The participants experimented with ChatGPT in small groups and discussed how they can use ChatGPT in their own field of expertise. These are their most important findings:
See the slides from the workshop here:
Hoe ga je als docent om met ChatGPT?
By Silvester Draaijer (via VU.nl)
Dutch article with practical suggestions on how to use AI writers in higher education.
ChatGPT – Verzameling bronnen
By Bertine van Deyzen (via surf.nl)
Collection of relevant Dutch and English sources and articles on ChatGPT in relation to education.
Dit is niet geschreven door AI. Het gaat over de impact van AI op ons onderwijs.
By Erdinç Saçan (via surf.nl)
Dutch article citing different opinions and articles about the impact of ChatGPT on education.
AI content generatie tools zoals ChatGPT leiden tot kansen en uitdagingen voor leren, opleiden en onderwijs.
By Wilfred Rubens (via te-learning.nl)
Dutch article about the potential impact of natural language models on education.
‘Full-on robot writing’: the artificial intelligence challenge facing universities.
By Jeff Sparrow (via The Guardian)
English article about the effect of AI on university teaching.
ChatGPT: Hoe je de perfecte prompts schrijft.
By Sophia Schmid (via neuroflash.com)
Dutch article with tips about writing good prompts for ChatGPT.