The lack of transparency, explainability and interpretability in AI systems (“black box problem”) presents risks in the context of education. If an AI system makes recommendations on a child’s learning pathway or provides recommendations, which may have long-term consequences for the child’s development, then teachers and parents must be able to understand the reasoning behind its decisions, including the parameters used, and have the ability to evaluate and override them if necessary. Likewise, AI systems used in admissions or examinations could have significant implications for rights holders’ educational opportunities and future prospects. The opacity of AI can also make it difficult to provide genuinely informed consent or to contest its decisions and outcomes.[1] Consent must unambiguously be freely given and able to be refused without detriment.[2] Sufficient levels of transparency should be ensured. The same apply with respect to adult learners.

Member States should also ensure the effective implementation of their obligations under Article 13 ECHR to fulfil the right of children and adult learners to an effective remedy when their human rights and fundamental freedoms have been infringed using AI systems in the educational context.

In particular for children, this entails the provision of available, known, accessible, affordable, and child-friendly avenues through which children, as well as their parents or legal representatives, may submit complaints and seek remedies. Effective remedies can include, depending on the violation in question, inquiry, explanation, reply, correction, proceedings, immediate removal of unlawful content, apology, reinstatement, reconnection and compensation.[3] States should also ensure that in all cases, access to courts or judicial review of administrative remedies and other procedures are available, in line with the principles set out in the Guidelines of the Committee of Ministers of the Council of Europe on child-friendly justice (2010).