Feedback is crucial to so many professions as it enables us to understand what we did well and what we did badly. Research from the University of Cambridge suggests that AI might one day be able to provide us with robust feedback on the quality of our work.

The researchers worked with nearly 200 trainee teachers in Germany and used AI to assess the ability of the teachers to gauge whether pupils had learning difficulties.

Accurate assessments

The trainee teachers were given six fictional students to assess, each of whom had potential learning difficulties. The trainees were given various things to help them, such as examples of their schoolwork, their behavior record, and even transcripts of conversations with their parents. They then had to assess whether the pupil had a learning difficulty, such as dyslexia or Attention Deficit Hyperactivity Disorder (ADHD), or not, and what their reasoning for their assessment was.

After submitting their assessment, half of the trainees were given the assessment of a qualified professional, with the other half getting feedback from an AI system, which highlighted the areas of the solution that were correct and the areas that needed improvement. After a few of these assessments, the trainers then took a couple of follow-up tests without feedback but with their assessments marked by researchers according to accuracy and diagnostic reasoning.

The results show that trainees in the AI group scored around 10% higher for diagnostic reasoning than their peers who had received feedback from experts. The researchers believe this could be due to the adaptive nature of AI, which allows the system to respond to the unique assessments of each student rather than the more generic feedback provided by the experts.

Skills gap

As such, it’s not clear whether the AI approach would be better than one-to-one feedback from an expert, but the authors are nonetheless confident that their findings have merit, not least because such one-to-one support is not always available.

“Teachers play a critical role in recognizing the signs of disorders and learning difficulties in pupils and referring them to specialists. Unfortunately, many of them also feel that they have not had sufficient opportunity to practice these skills,” they explain. “The level of personalized guidance trainee teachers get on German courses is different to the UK, but in both cases, it is possible that AI could provide an extra level of individualized feedback to help them develop these essential competencies.”

Suffice to say, whether this will actually occur remains open to question, but the researchers believe that their work should at least prompt further studies to explore how and whether AI can be an effective part of teacher training and assessment.

“In large training programs, which are fairly common in fields such as teacher training or medical education, using AI to support simulation-based learning could have real value,” they conclude. “Developing and implementing complex natural language processing tools for this purpose takes time and effort, but if it helps to improve the reasoning skills of future cohorts of professionals, it may well prove worth the investment.”