As already observed, AI decision-making processes can be opaque, making it difficult to understand how and why a decision was made. This lack of transparency can undermine accountability in the delivery of social services, especially when individuals are denied benefits or services based on AI decisions.

If a person is disadvantaged by an AI decision (e.g., being wrongly denied welfare benefits), it may be challenging for them to appeal or challenge the decision due to the "black-box" nature of many AI systems, whether it is intentional (i.e., for intellectual property considerations) or intrinsic (i.e., too complicated for anyone without particularly advanced digital skills).

The lack of transparency and accountability around the use of AI systems can lead to depriving the subjects of AI decision-making from an explanation or the opportunity to appeal against decisions that in some cases may be of vital importance to them and thus interferes with their right to an effective remedy. In cases where the events in issue lie wholly, or in large part, within the exclusive knowledge of the authorities, as would arguably be the case when AI systems are involved, or when it would be extremely difficult in practice for the applicant to prove discrimination, the Court/ESCR has shifted the burden of proof on the authorities.[1]

 

[1] Salman v. Turkey [GC], No. 21986/93, 27 June 2000, § 100; Anguelova v. Bulgaria, no 38361/97, 13 June 2002, § 111; Cînţa v. Romania, No. 3891/19, 18 February 2020, 3891/19, §79; Mental Disability Advocacy Centre (MDAC) v. Bulgaria, Complaint No. 41/2007, decision on the merits of 3 June 2008, § 52.