GenAI and Safety in the Classroom

Teachers’ top 4 safety questions, answered.

With GenAI creeping into our classrooms in more ways than ever, teachers are asking: is it safe?

We’re seeing AI technology evolve faster than schools can keep up. There are a lot of unanswered questions about the impact of GenAI in the classroom, so it is perfect timing to take a closer look at the safety concerns and solutions. Thankfully, Jacaranda is spearheading GenAI safety by designing our classroom GenAI tool, jacTUTOR, with student safety at front of mind.

1. Privacy: how can we safeguard students’ personal information?

Key concerns: Schools are rightfully worried about how GenAI systems handle sensitive information about students. Could personal information be accessed? Could students share personal information without knowing it, putting their own safety at risk?

jacTUTOR safeguards: We’ve built a tool with strong privacy parameters – there’s no need for students to plug any personal information into jacTUTOR. If you’ve heard of ‘PII’ in tech terms, students’ ‘personally identifiable information’ is not passed onto the tutor. Students access jacTUTOR in the safe environment of learnON – no additional logins or web browsing needed to use the tutor function.

Our Head of Software Engineering, Claire Lord, feels very strongly about Jacaranda’s role in getting this right for schools. “We are acutely aware of our responsibility in privacy and security when working with GenAI – from the way we program the software through to how user data is handled. Our learnON and jacTUTOR applications are securely hosted here in Australia.”

2. Appropriateness: are ‘chats’ between students and GenAI safe?

Key concern: We’ve all used GenAI tools that let you ask a question, and a chatbot responds in conversational language. Two things are worrying teachers – will students be able to start conversations with chatbots in the classroom? And what could the chatbots say back to them?

jacTUTOR safeguards: jacTUTOR has guardrails put in place to make sure they are steering students in one direction only: the learning set by their teacher. In fact, one of the best safety measures we’ve developed is the boundaries we’ve put around students’ interactions with jacTUTOR. Teachers are loving the fact that we’ve limited the interaction to fixed prompts that the student can use to ask jacTUTOR a question. Using even more guardrails behind the scenes, we moderate and guide the responses from jacTUTOR to the student.

Claire Lord explains, “There’s currently no opportunity to ‘chat’ freely with jacTUTOR at this point – students can only choose from the set of guiding questions.” There’ll likely be an opportunity for schools to opt in to free-text fields in the future, giving teachers the choice as to how they want their students to interact with GenAI.

3. Accuracy: are we sure it’s a safe, reliable source?

Key concern: Teachers worry about the accuracy of AI-generated content. It’s true to say that public AI tools may not always give correct information based off the sources they are pulling from. Plus, they don’t always use a teaching style we’d use in our Australian classrooms. What if GenAI provides inaccurate or unsafe answers?

jacTUTOR safeguards: A major safety benefit of jacTUTOR is how it’s guided to use content from the curriculum. As the experts say, it’s highly ‘context-aware’. That means jacTUTOR is aware of where the student is in the curriculum that’s being taught by their teacher so that it is pitched to the right level. Lord further explains, “on the technical side, we’ve given jacTUTOR our Jacaranda resources to craft its answers, so the Jacaranda content is its primary source.”

4. Responsibility: should it weigh on teachers’ minds?

Key concern: Cyber safety has never been more important, and teachers are hardwired to look out for their students’ safety. Educators are concerned about the implications of using GenAI – from considering the risks to students, to worrying that students will get steered in the wrong direction.

jacTUTOR safeguards: We’ve been heavily guided by the Australian Government’s principles in the Australian Framework for Generative Artificial Intelligence in Schools while developing jacTUTOR. Just like the Framework supports schools, it also guides resource developers like Jacaranda to develop GenAI tools ethically and responsibly. Honouring these guidelines have been a huge part of bringing jacTUTOR to life.

Safety first.

So it’s clear there are safety matters to consider, but now is the time to help shape the role of GenAI in the classroom. We call it ‘safety by design’. The important thing is that we all put safety and security first, and remember that our mission is to use this new technology for good.

Continue the AI conversation with our webinar designed for teachers: GenAI in the Classroom presented by Leon Furze.

Watch this session where Leon gives insights into Generative Artificial Intelligence (GenAI), what it means for the future and the framework set out by the Australian Department of Education.

WATCH NOW