Assistance with AI literacy with the AI Maturity in Education Scan (AIMES)

We are in a period where teaching quality is under pressure. Many teachers do not have a good idea of how their students came to work out their assignments and whether they are using GenAI to do so, for example. That is not to say, by the way, that these students are not developing skills, but the skills may differ from the teacher’s intention. When it comes to teaching quality, we need to go back to the drawing board and take advantage of the opportunities presented by new technologies and minimize threats. A high level of AI literacy is essential in this regard.

Luuk says: “Regularly I notice that there is still a lot of confusion about what we mean by AI. To some, ChatGPT seems almost synonymous with AI. And far from everyone is aware that a chatbot can also provide incorrect information or that GenAI can reinforce prejudice. AIMES is therefore promoting awareness of the consequences of AI applications within education. Many educational institutions offer workshops aimed at increasing AI literacy. But more is needed. We are simply not there alone with all kinds of informal activities, however well intentioned.” Luuk points out that AI literacy must also have a place within qualification pathways – such as University Teaching Qualification (UTQ/ BKO) and Senior Teaching Qualification (STQ) – in order to ensure educational quality. He therefore works with the coordinators of these qualification programmes. He is also in contact with coordinators of the national programme Npuls where integration of AI within qualification programmes is also an important action point. Luuk continues: “The vision on teaching and learning, institution plans and accreditation processes also need to be updated. AI literacy must also start to have a place in these. We are in a transitional phase with regard to professionalization within education in many areas. Here AIMES serves as a starting point for not only the good but certainly necessary conversation.”

The initiative to develop the scan arose well before the November 2022 launch of ChatGPT 3.5. This launch accelerated everything. Luuk: “At that time, I was already working with colleagues on an ethical framework based on the seven requirements for trustworthy AI. An activating concept was needed to increase the impact of this ethical framework. This led to the development of AIMES.” The scan offers participants instant insight into the extent of effective, efficient and responsible applications of AI in their education.   

Luuk continues: “AIMES consists of three separate questionnaires, each aimed at a specific target group: lecturers, program directors and faculty administrators. A conscious choice; for each faculty and program, each target group builds the education being provided from its own role and function. Increasing AI literacy therefore requires an integrated approach.” 

The scan consists of a short questionnaire with 10 items that participants complete in about 5-10 minutes. For each item, you will see at which of the three possible maturity levels you are at when it comes to AI in education. You will also receive actionable feedback and tips on how to get to the next level. These include teaching tips, workshops, events or components of formal qualification programmes such as the University Teaching Qualification (UTQ/ BKO), Senior Teaching Qualification in Education (STQ) and the Educational Leadership programme (LOL) of UvA Teaching & Learning Centers (TLC). All feedback is based on the EU AI ACT, recent publications from UNESCO and available information at UvA or VU Amsterdam.

Connecting with students’ world of experiences is important 

AI is fully part of many students’ lives. Luuk emphasizes, “It cannot be ignored in education, just pointing out what is not allowed or worse, banning it. AI plays an important role in our society and in the labor market.  We need to prepare students for it. It is important that in their studies they learn how to use AI in an ethical and effective way. It is a tool in your toolbox of academic skills and therefore you have to learn how to work with this tool. The people who teach and shape education have an important role in this. Therefore, it is also important that AI literacy is not limited to students and faculty. But program directors and faculty administrators also have an important role in this.”   

Collaboration with an international expert panel  

Luuk is the initiator and has been working with an international expert panel of 46 people from 8 different countries since AIMES was launched in January 2024. These experts in the field of AI in education give their input on the items and action-oriented feedback included in the scan through two rounds of feedback (Delphi method).   

The development of the scan has now been in progress for almost six months. The second feedback round in which input from the various perspectives of the experts was collected was completed in summer. During the summer, input was also sought from experts within the UvA and the VU. These include members of AI in education working groups at both universities. Technical realization has now begun.   

First, an ‘Open Educational Resource’ variant will be delivered (expected availability at the end of September 2024). This version is suitable as a basis for any Higher Education institution worldwide. After that, AIMES will be further developed into a UvA and VU specific version in which participants will receive feedback and tips and be provided with information that is in line with the institutional plan, policies and information available within their own university. These variants are expected to be available in December 2024. Luuk: “Developments are obviously rapid. AIMES is therefore updated every six months.”