The use of AI in social services involves processing sensitive personal data, raising serious privacy concerns under Article 8 ECHR. The aggregation of sensitive data, such as health records, financial and employment history, and other personal details, that enables the State to acquire a detailed profile of the most intimate aspects of citizens’ lives, may result in particularly invasive interference with private life.[1]

For example, concerns related to compliance with Article 8 ECHR were raised in the “SyRi” case, where the Hague District Court found that an algorithm used for the purpose of identifying potential social welfare fraud (the “Systeem Risico Indicatie” or “SyRi”) and the relevant legislation did not meet the requirements for necessity and proportionality as required by Article 8(2) ECHR.[2]

An additional risk is the misuse of personal data collected in social services, including unauthorised surveillance, profiling without consent, or accidental breaches. Concerns also arise from businesses involvement in developing or maintaining AI systems or outsourcing social services to private companies. Considering that AI systems store vast amounts of sensitive data, particular importance should also be placed on data security, including when a particular AI system is developed and maintained by third-party (private) vendors.

 

[1] Szabó and Vissy v. Hungary, No. 37138/146, 12 January 2016, § 70.

[2] The Hague District Court, NCJM et al. and FNV v The State of the Netherlands, 6 March 2020, available in English (ECLI:NL:RBDHA:2020:865). The system concerned was the “Systeem Risico Indicatie” or “SyRi”. See also the amicus curiae brief submitted by the United Nations Special Rapporteur on Extreme Poverty and Human Rights stressing in particular the discriminatory and stigmatizing effect of SyRi”.

Brief by the United Nations Special Rapporteur on extreme poverty and human rights as Amicus Curiae in the case of NJCM c.s./De Staat des Nerdelanden (SyRI) before the District Court of The Hague