ChatGPT workshop for lecturers

On January 16th 2023, the Programming Teaching Lab and the Teaching and Learning Centre Science organised a 4-hour ChatGPT workshop for lecturers. ChatGPT is a language model with chat capabilities and it seems like it can do almost anything: Answer all of your questions, come up with “new” ideas, write your essay for you, and whatnot. Lecturers all over the world shudder in fear: Can we still trust the content of our students or is everything generated by AI?

However, there are also a lot of opportunities with this new technique. During the workshop, we wanted to focus on the positives: What can lecturers take away from ChatGPT? How can we use ChatGPT to our own advantage? Should we think about new ways of assessing? Can it generate new assignments, embed its use into homework? We would argue that the sky is the limit here.

What is ChatGPT?

Jelle Zuidema, associate professor in Natural Language Processing, Explainable AI and Cognitive Modelling at the Institute for Logic, Language and Computation, explained that ChatGPT didn’t come out of the blue, but there has been a continuous development of natural language processing technology in recent years. There is no explicit source where it gets all its information from, but it makes abstractions from all the available online information. However, that doesn’t mean that the information it provides is always correct. It produces a lot of content that sounds right because it has learned the grammar and different types of writing genres and can reproduce those structures. However, the content can be (and regularly is) actually nonsense. As ChatGPT itself explains it:

It is a pure language model and produces content based on what are plausible continuations of text. It is trained on user data and wants to make its users happy (e.g. by providing short and clear answers), but it’s not optimised for truthfulness. For example, it can generate sources when the user asks for references because that will make the user happy, but those references can be completely made up.

But how reliable is the content in that case? How do you distinguish confabulations from facts? This is still an active field of research. A possible solution can be to first search for factual content with classical techniques (e.g., in books or with reliable online sources) and then use ChatGPT to summarise all the information from those sources. Unfortunately, asking ChatGPT in your prompt to “be truthful” won’t solve this problem. It might lead to a more truthful answer, but then that’s purely a coincidence.

How can ChatGPT be used in education?

But if producing truthful content isn’t what ChatGPT does, how can we use it in education? Martijn Stegeman, lecturer and coordinator of the UvA minor Programming, identified two main areas for which ChatGPT can be used in education:

1. Student learning aid. Student enrolments are very high, leaving lecturers with too little time to properly supervise and guide their students. How can we use ChatGPT to help students study and to make education better? For example, ChatGPT can explain Python code, write flashcard questions that students can use for self-study, and execute a roleplay with ChatGPT as the lecturer.
2. Lecturer inspiration. As a lecturer, ChatGPT can save you a lot of time and give you inspiration for your teaching. For example, ChatGPT can serve as a test-student to check whether your exam questions are unambiguous, create exam questions, generate a creative programming assignment, write a study guide for a course, and give feedback on an assessment scale.

How to ask the right questions?

Remember, ChatGPT is programmed to provide answers that make the reader happy. Vice versa, that means that asking the right questions – called prompts – is essential when using it. Therefore, Martijn provided guidance for writing good prompts. The most important thing is to not phrase the prompt too broadly or vaguely, but as precisely as possible. Your prompt should include all the criteria that you are looking for in the answer. For example, if you need questions that test whether your students can apply their knowledge, asking ChatGPT for 10 questions might result in questions of a quality you cannot use. However, if you ask ChatGPT to create 10 questions that require a relational response (as described in the SOLO taxonomy) it will adhere to those requirements (see images below).

Keep in mind, if there’s a lot of content available about the topic or if it is a common genre, ChatGPT’s output becomes better (e.g. ChatGPT is really good at creating a study guide entry based on a couple of your notes).

For more tips about writing good prompts for ChatGPT, you can read this Dutch article from Neuroflash.