Rapid Rundown : A summary of the week of AI in education and research

AI Education Podcast

This week's episode was our new format shortcast - a rapid rundown of some of the news about AI in Education. And it was a hectic week! Here's the links to the topics discussed in the podcast

Australian academics apologise for false AI-generated allegations against big four consultancy firms

https://www.theguardian.com/business/2023/nov/02/australian-academics-apologise-for-false-ai-generated-allegations-against-big-four-consultancy-firms?CMP=Share_iOSApp_Other

New UK DfE guidance on generative AI

The UK's Department for Education guidance on generative AI looks useful for teachers and schools.

It has good advice about making sure that you are aware of students' use of AI, and are also aware of the need to ensure that their data - and your data - is protected, including not letting it be used for training.

The easiest way to do this is use enterprise grade AI - education or business services - rather than consumer services (the difference between using Teams and Facebook)

You can read the DfE's guidelines here: https://lnkd.in/eqBU4fR5

You can check out the assessment guidelines here: https://lnkd.in/ehYYBktb

"Everyone Knows Claude Doesn't Show Up on AI Detectors"

Not a paper, but an article from an Academic

https://michellekassorla.substack.com/p/everyone-knows-claude-doesnt-show

The article discusses an experiment conducted to test AI detectors' ability to identify content generated by AI writing tools. The author used different AI writers, including ChatGPT, Bard, Bing, and Claude, to write essays which were then checked for plagiarism and AI content using Turnitin. The tests revealed that while other AIs were detected, Claude's submissions consistently bypassed the AI detectors.

New AI isn't like Old AI - you don't have to spend 80% of your project and budget up front gathering and cleaning data

Ethan Mollick on Twitter: The biggest confusion I see about AI from smart people and organizations is conflation between the key to success in pre-2023 machine learning/data science AI (having the best data) & current LLM/generative AI (using it a lot to see what it knows and does, worry about data later)

Ethan's tweet 4th November

His blog post:

https://www.oneusefulthing.org/p/on-holding-back-the-strange-ai-tide

Open AI's Dev Day

We talked about the Open AI announcements this week, including the new GPTs - which is a way to create and use assistants.

The Open AI blog post is here: https://openai.com/blog/new-models-and-developer-products-announced-at-devday

The blog post on GPT's is here: https://openai.com/blog/introducing-gpts

And the keynote video is here: OpenAI DevDay, Opening Keynote

Research Corner

Gender Bias

Quote: "Contrary to concerns, the results revealed no significant difference in gender bias between the writings of the AI-assisted groups and those without AI support. These findings are pivotal as they suggest that LLMs can be employed in educational settings to aid writing without necessarily transferring biases to student work"

Tutor Feedback tool

Summary of the Research: This paper presents two longitudinal studies assessing the impact of AI-generated feedback on English as a New Language (ENL) learners' writing. The first study compared the learning outcomes of students receiving feedback from ChatGPT with those receiving human tutor feedback, finding no significant difference in outcomes. The second study explored ENL students' preferences between AI and human feedback, revealing a nearly even split. The research suggests that AI-generated feedback can be incorporated into ENL writing assessment without detriment to learning outcomes, recommending a blended approach to capitalize on the strengths of both AI and human feedback.

Personalised feedback in medical learning

Summary of the Research: The study examined the efficacy of ChatGPT in delivering formative feedback within a collaborative learning workshop for health professionals. The AI was integrated into a professional development course to assist in formulating digital health evaluation plans. Feedback from ChatGPT was considered valuable by 84% of participants, enhancing the learning experience and group interaction. Despite some participants preferring human feedback, the study underscores the potential of AI in educational settings, especially where personalized attention is limited.

High Stakes answers

Your Mum was right all along - ask nicely if you want things! And, in the case of ChatGPT, tell it your boss/Mum/sister is relying on your for the right answer!

Summary of the Research: This paper explores the potential of Large Language Models (LLMs) to comprehend and be augmented by emotional stimuli. Through a series of automatic and human-involved experiments across 45 tasks, the study assesses the performance of various LLMs, including Flan-T5-Large, Vicuna, Llama 2, BLOOM, ChatGPT, and GPT-4. The concept of "EmotionPrompt," which integrates emotional cues into standard prompts, is introduced and shown to significantly improve LLM performance. For instance, the inclusion of emotional stimuli led to an 8.00% relative performance improvement in Instruction Induction and a 115% increase in BIG-Bench tasks. The human study further confirmed a 10.9% average enhancement in generative tasks, validating the efficacy of emotional prompts in improving the quality of LLM outputs.