How can generative AI be used to innovate your course? Stefan Mol (Leadership and Management) and teaching assistant John Gatev share how they did this for the Business Research Methods course. They have introduced two key innovations: a chatbot for real-time feedback and a diversity-driven student group matching algorithm.

Can you share the inspiration behind your innovations? What specific challenges were you aiming to address? 
Stefan: This year, we have introduced two key innovations. The first is a chatbot designed to assist students in real-time by offering feedback on assignments, summarising course materials, generating practice questions, and answering queries about assessments. The second is a diversity-driven student group matching algorithm, aimed at promoting more inclusive student interactions. 

The main inspiration behind the innovations comes from the challenges students face early in their curriculum. Since this course is one of the first they encounter, they often struggle with complex research concepts and vocabulary. The chatbot was created to provide step-by-step guidance to help students better understand and engage with course content outside of class hours. 

Stefan Mol & John Gatev

John, could you tell more about your role and how you became involved with the UvA AI chatbot integration? 

John: I started as a teaching assistant in late 2022, and in 2023, Stefan introduced me to the idea of integrating the UVA AI chatbot. I had used a similar tool during my studies but saw the potential in the UVA system, which uses retrieval-augmented generation. My role involved ensuring the chatbot had access to all the relevant course materials like lecture slides and tutorials. I also configured the chatbot’s responses to ensure it could support students effectively.

What features did you specifically help set up for students? 
John: One key feature I worked on was the chatbot’s ability to provide instant summaries and practice questions. For example, students can type “practice” to get multiple-choice questions that align with exam styles. Additionally, I developed a feedback loop where the chatbot tracks students’ progress and tests their understanding. We also incorporated feedback from previous years’ assignments to provide personalised feedback, much like a tutor would. 

How have students responded to using the chatbot in terms of their engagement and understanding of course material? 
Stefan: Student engagement has been noticeably higher, and they appreciate the chatbot’s interactive nature. It offers an accessible way to revise course materials compared to traditional methods. Students have mentioned that the chatbot’s use of proper course terminology sets it apart from other tools like ChatGPT. It has become a valuable resource, especially when they are stuck and need immediate clarification. 

Have you seen any measurable improvements in student performance or engagement linked to using the chatbot? 
Stefan: Yes, we looked at midterm scores and found that students who used the chatbot scored an average of 26 points, compared to 23 for those who didn’t. While it’s hard to attribute this entirely to the chatbot, it suggests that the tool is helping students perform better. They seem to be more confident in their understanding, which likely contributes to the improved exam results. 

Could you elaborate on the second innovation: the diversity-driven student group matching algorithm? How does it work, and what outcomes have you observed? 
Stefan: The idea for the diversity-driven matching algorithm came from students who proposed it to the Diversity & Inclusion Committee. They observed that students often grouped with peers of similar backgrounds, limiting their exposure to diverse perspectives. The algorithm matches students based on shared extracurricular interests, such as hobbies, while ensuring diversity in nationality. We ran an experiment with a control group (students grouped by similar backgrounds) and a treatment group (diverse groupings). Post-course surveys revealed positive feedback on the sense of belonging among diverse groups, though some students did express frustration about not choosing their own group members.

Do you foresee scaling or expanding the chatbot’s capabilities to other courses or areas of student support? 
Stefan: Yes, we plan to expand its use. The chatbot currently helps guide students through general tasks, but the next step would be to integrate it with more specialised content—like course readings and lecture notes. We would need to ensure the content is legally permissible, but it’s an exciting opportunity to enhance its capabilities. We also see potential for the chatbot to support areas beyond Business Research Methods. 

Looking ahead, do you have any plans to further develop or refine these innovations? What lessons have you learned so far? 
Stefan: In the future, I’d like to see the course become more autonomous, with the chatbot assisting more in grading calibration among teaching staff to ensure consistency. We also considered having the chatbot grade assignments and compare results with human graders, but this might not be feasible due to time constraints. 

We also aim to improve the onboarding process for students using the chatbot, as some struggled with navigating the platform. These lessons will guide further developments to make the tools more effective. 

This interview was first published in the TLC-EB magazine 2025.