Artificial intelligence system AI systems lifecycle Machine-based system Autonomy Adaptiveness AI systems objectives Environment or context Input Inference Output Transparency Explainability Interpretability

Artificial intelligence system

"Artificial intelligence system” means a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations or decisions that may influence physical or virtual environments. Different artificial intelligence systems vary in their levels of autonomy and adaptiveness after deployment.[1]

This definition reflects a broad understanding of what artificial intelligence systems (AI systems) are, specifically as opposed to other types of simpler traditional software systems based on the rules defined solely by natural persons to automatically execute operations.[2] It was drafted for the purposes of the Framework Convention, drawing upon the 2023 OECD definition,[3] and aims at ensuring legal precision and certainty, while also remaining sufficiently abstract and flexible to stay valid despite future technological developments. However, it is not meant to give universal meaning to the relevant term.[4] 
Back to top

AI systems lifecycle

The lifecycle of an AI system may encompass a range of activities, depending on the type of technology and other contextual elements and change over time. The following are non-exhaustive relevant examples of activities: (1) planning and design, (2) data collection and processing, (3) development of artificial intelligence systems, including model building and/or fine-tuning existing models for specific tasks, (4) testing, verification and validation, (5) supply/making the systems available for use, (6) deployment, (7) operation and monitoring, and (8) retirement.[5] These activities often take place in an iterative manner and are not necessarily sequential. They may also start all over again when there are substantial changes in the system or its intended use. The decision to retire an AI system from operation may occur at any point during the operation and monitoring phase.[6]
Back to top

Machine-based system

The term ‘machine-based’ refers to the fact that AI systems are developed with and run on machines. The term ‘machine’ can be understood to include both the hardware and software components that enable the AI system to function. The hardware components refer to the physical elements of the machine, such as processing units, memory, storage devices, networking units, and input/output interfaces, which provide the infrastructure for computation. The software components encompass computer code, instructions, programs, operating systems, and applications that handle how the hardware processes data and performs tasks.[7]
Back to top

Autonomy

AI system autonomy means the degree to which a system can learn or act without human involvement following the delegation of autonomy and process automation by humans. Human supervision can occur at any stage of the AI system lifecycle.[8] Some AI systems can generate outputs without these outputs being explicitly described in the AI system’s objective and without specific instructions from a human.[9]
Back to top

Adaptiveness

Adaptiveness refers to the capability of an AI system to evolve and modify its behaviour through direct interaction with input and data before or after deployment. Examples include a speech recognition system that adapts to an individual’s voice or a personalised music recommender system. AI systems can be trained once, periodically, or continually, and operate by inferring patterns and relationships in data. Through such training, some AI systems may develop the ability to perform new forms of inference not initially envisioned by their programmers.[10]
Back to top

AI system objectives

AI systems are designed to operate according to one or more objectives. The objectives of the system may be explicitly or implicitly defined. Explicit objectives refer to clearly stated goals that are directly encoded by the developer into the system. For example, they may be specified as the optimisation of some cost function, a probability, or a cumulative reward. Implicit objectives refer to goals that are not explicitly stated but may be deduced from the behaviour or underlying assumptions of the system. These objectives may arise from the training data or from the interaction of the AI system with its environment.[11]
Back to top

 Environment or Context

An environment or context in relation to an AI system is an observable or partially observable space perceived using data and sensor inputs and influenced through actions (through actuators). The environments influenced by AI systems can be physical or virtual and include environments describing aspects of human activity, such as biological signals or human behaviour. Sensors and actuators are either humans or components of machines or devices.[12]
Back to top

Input

Input is used both during development and after deployment. Input can take the form of knowledge, rules and code that humans put into the system during development or data. Humans and machines can provide input. During development, input is leveraged to build AI systems, e.g., with machine learning that produces a model from training data and/or human input. Input is also used by a system in operation, for instance, to infer how to generate outputs. Input can include data relevant to the task to be performed or take the form of, for example, a user prompt or a search query.[13
Back to top

Inference

The concept of “inference” generally refers to the step in which a system generates an output from its inputs, typically after deployment. “Infer how to generate outputs” should be understood as also referring to the build phase of the AI system, in which a model is derived from inputs/data.[14]
Back to top
 

Outputs

Outputs generally reflect different tasks or functions performed by AI systems. They can be broadly categorised as recommendations, predictions, content and decisions. These categories correspond to different levels of human involvement, with “decisions” being the most autonomous type of output (the AI system affects its environment directly or directs another entity to do so) and “predictions” the least autonomous. They include, but are not limited to, recognition (identifying and categorising data, e.g., image, video, audio and text, into specific classifications as well as image segmentation and object detection), event detection (connecting data points to detect patterns, as well as outliers or anomalies), forecasting (using past and existing behaviours to predict future outcomes), personalisation (developing a profile of an individual and learning and adapting its output to that individual over time), interaction support, interpreting and creating content to power conversational and other interactions between machines and humans, possibly involving multiple media such as voice text and images), content generation (including but not limited to goal-driven optimisation (finding the optimal solution to a problem for a cost function or predefined goal) and reasoning with knowledge structures (inferring new outcomes that are possible even if they are not present in existing data, through modelling and simulation).[15]
Back to top

Further technical concepts relevant for AI and human rights

Transparency

In the context of AI, transparency refers to openness and clarity in the governance of activities within the lifecycle of AI systems. It means that the decision-making processes and general operation of AI systems should be understandable and accessible to appropriate AI actors and, where necessary and appropriate, relevant stakeholders.[16] The means of ensuring transparency would depend on many different factors such as, for instance, the type of artificial intelligence system, the context of its use or its role, and the background of the relevant actor or affected stakeholder. Moreover, relevant measures include, as appropriate, recording key considerations such as data provenance, training methodologies, validity of data sources, documentation and transparency on training, testing and validation data used, risk mitigation efforts, and processes and decisions implemented, in order to aid a comprehensive understanding of how the artificial intelligence system’s outputs are derived and impact human rights, democracy and the rule of law.[17]

To this end, the use of open-source software and interoperable technical standards should be encouraged for AI systems, insofar as it contributes to the transparency and verifiability of the systems, and contestability of the outcomes.[18] AI systems used in contexts that may impact human rights, such as electoral processes, should maintain complete and tamper-proof audit logs, allowing the tracing of all decisions and actions carried out.[19] These logs should be preserved in accordance with the legal time limits applicable and remain accessible to the competent oversight authorities, subject to appropriate data protection safeguards.[20]
Back to top

Explainability

Explainability is a particularly important component of transparency. AI systems integrating machine learning (ML) rely on mathematical models derived from automatic processing of data, rather than by explicit programming by humans. This makes it difficult even for AI experts, including the developers of the systems, to understand how their outputs are subsequently produced.[21] The resulting opacity, or “black box” effect, not only makes decisions more difficult to understand, but it can also have direct impact on individuals since it can hide deficiencies in AI systems, such as the existence of bias, inaccuracies, so-called “hallucinations”.

“Explainability” therefore refers to the capacity to provide, subject to technical feasibility and taking into account the generally acknowledged state of the art, sufficiently understandable explanations about why an AI system provides information, produces predictions, content, recommendations or decisions as well as a general understanding of its capabilities and limitations.[22] It is the idea that the outcome of an automated system or algorithm can be explained in a way that “makes sense” to people, enabling those who have been affected by an output to understand and challenge it. This includes providing – in clear and simple terms, and as appropriate in the context – the main factors included in a decision, the determinant factors, and the data, logic or algorithm used to reach a decision.[23]
Back to top

Interpretability

Interpretability refers to the ability to understand how an AI system makes its predictions or decisions or, in other words, the extent to which the outputs of AI systems can be made accessible and understandable to experts and non-experts alike. It involves making the internal workings, logic, and decision-making processes of artificial intelligence systems understandable and accessible to human users, including developers, stakeholders, and end-users, and persons affected.[24]
Back to top