It can be, if the tool is built for education and the school sets safeguards before launch. The UK Department for Education’s generative AI product safety standards, updated January 2026, set the benchmark. They cover filtering, safeguarding alerts to the Designated Safeguarding Lead, data protection, and no final answers by default. Teachers must be able to see what students do.
What do regulators expect of an AI tutor for schools?
The most detailed benchmark is the DfE’s Generative AI: product safety standards. They were first published in January 2025 and updated on 19 January 2026 “to include new standards on cognitive development, emotional and social development, mental health, and manipulation”. They are aimed at suppliers, but the DfE notes schools “may also find these standards helpful in assessing which AI products are safe for use in education”. The standards explicitly cover “personal tutors and chatbots”.
| Standard | What an AI tutor should do |
|---|---|
| Filtering | ”Effectively and reliably prevent users from accessing harmful or inappropriate content”, on any device |
| Monitoring and reporting | Log activity; “identify and alert local supervisors of disclosures that indicate a possible safeguarding issue”; take the school’s DSL contact at setup |
| Security | Resist “jailbreaking”; let administrators set permission levels |
| Privacy and data protection | Comply with data protection law; no personal data used for “further model training and fine-tuning” without a lawful basis |
| Intellectual property | Not use students’ or teachers’ work for training without consent (from a parent or guardian for under-18s) |
| Design and testing | Prioritise “transparency and children’s safety”, including testing with children |
| Governance | A risk assessment for every product and a formal complaints route |
| Cognitive development | ”Not to provide final answers, full solutions, or complete worked examples by default”; start “with hints or partial steps” |
| Emotional and social development | Not imply “emotions, consciousness or personhood”; remind users AI cannot replace human relationships; default time limits |
| Mental health | ”Detect signs of learner distress”, such as mentions of self-harm, and raise “a safeguarding flag to the institution’s safeguarding lead” |
| Manipulation | No “sycophancy and flattery”; no design “to prolong use, for increased engagement or revenue” |
Do other bodies set expectations too?
Yes, and they point the same way. UNICEF’s Guidance on AI and Children 3.0 (December 2025) sets ten requirements. They include ensuring safety for children, protecting their data and privacy, and supporting their “best interests, development and well-being”. Its checklist asks systems to “clearly warn children and caregivers upfront that they are interacting with an AI, not a human”. In Abu Dhabi, ADEK requires schools “to use AI learning tools that meet ADEK requirements”. Its Technology and AI Policy covers data protection, human oversight of AI-generated decisions, academic integrity and content safety.
In the EU, the AI Act already prohibits AI that infers students’ emotions in education institutions, except for medical or safety reasons. From 2 December 2027, AI systems that “evaluate learning outcomes” or monitor students during tests are high-risk, with extra obligations (Regulation (EU) 2024/1689; Regulation (EU) 2026/1744).
Is it safe to let students use general chatbots instead?
It is riskier. The DfE’s policy paper says pupils “should only be using generative AI in education settings with appropriate safeguards in place, such as close supervision and the use of tools with safety and filtering and monitoring features”. Schools should also “comply with age restrictions set by AI tools” (DfE). The OECD’s Digital Education Outlook 2026 warns that general-purpose tools “may enhance the apparent quality of student work … without improving their actual learning”. It argues they “must be used with pedagogical purpose or redesigned as specific educational GenAI tools”.
What about student data and consent?
Keep personal data out of AI tools unless the school has a lawful basis and families understand it. The DfE recommends “that personal data is not used in generative AI tools”. Where it is used, schools should make sure pupils and parents “understand that their personal data is being processed using AI tools”. Under EU GDPR Article 8, where consent is the lawful basis for an online service offered to a child, the child must be at least 16 unless a parent consents. Member states may lower the age, but not below 13. Schools often rely on other lawful bases, so check local law rather than applying one age rule. Our AI policy template includes a data and age section.
How should a school deploy an AI study assistant?
Treat it as a safeguarding project, not a software rollout. A deployment plan:
- Choose a school-licensed tool and check it against the standards above. Our 12-point AI tool checklist turns them into questions.
- Complete a data protection impact assessment before any student logs in.
- Connect safeguarding. Enter the DSL’s details, test that alerts arrive, and brief the safeguarding team on what the tool will flag.
- Set use rules by task. Practice and revision, yes. Coursework, only as the boards allow (AI in IGCSE coursework).
- Tell families what the tool does, what data it holds and how to raise concerns.
- Keep teachers in the loop. Teachers should see student activity and act on it; the tool supports teaching, not the reverse.
- Review each term: usage patterns, safeguarding flags, complaints and learning impact.
How schools do this with AI Buddy
AI Buddy is a school-licensed AI study assistant built for Cambridge and Pearson Edexcel IGCSE and A Level practice. Students get instant AI feedback on past-paper-style questions, and teachers see every student’s activity on their dashboards. Student data sits in GDPR-compliant, encrypted hosting, and white-labelling keeps the tool under the school’s name. Schools should still check AI Buddy, like any tool, against the standards and local rules above as part of their own due diligence.
Frequently asked questions
Is it safe for schools to give students an AI tutor?
It can be, with an education-specific tool that filters content, alerts the DSL to safeguarding concerns, protects data, avoids giving final answers by default, and lets teachers see activity.
What should an AI tutor do about safeguarding?
The DfE’s standards expect activity logging, alerts to the Designated Safeguarding Lead when a disclosure suggests a safeguarding issue, and detection of signs of distress such as mentions of self-harm.
Can students use a general school AI chatbot?
The DfE says pupils should use generative AI only with safeguards such as close supervision and tools with filtering and monitoring, and within the tools’ age restrictions.
Should an AI study assistant give students the answers?
Not by default. The DfE’s 2026 standards expect tools to start with hints or partial steps rather than final answers or full worked solutions.
Discover how AI Buddy helps schools strengthen teaching, learning and evidence-informed school improvement. Or start a short consultation with our schools team using the form below — we will get back to you directly.
Sources
- Department for Education, Generative AI: product safety standards (updated 19 January 2026)
- Department for Education, Generative artificial intelligence (AI) in education (updated 12 August 2025)
- UNICEF Innocenti, Guidance on AI and Children 3.0 (December 2025)
- OECD, Digital Education Outlook 2026
- European Union, Regulation (EU) 2024/1689, Regulation (EU) 2026/1744 and GDPR Article 8
- Abu Dhabi Media Office, ADEK AI Literacy initiative (17 September 2026)