Article 5: Prohibited AI Practices

Official Statutory Text

Article 5 Prohibited AI practices 1. The following AI practices shall be prohibite d: (a) the placing on the market, the putting into service or the use of an AI system that deploys subliminal techniques beyond a person’s consciousness or purposefully manipulative or deceptive t echniques, with the objective, or the effectof materially distorting the behaviour of a person or a group of persons byappreciably impairing their ability to make an informed decision, thereby causing them to take a decision that they would not have other wise takenina manner that causes or is reasonably likely to cause that person, another person or group of persons significant harm; (b) the placing on the market, the putting into service or the use of an AI system that exploits any of the vulnerabilities of a natural person or a specific group of persons due to their age, disability or a specific social or economic situation, with the objective, or the effe ct, of materially distorting the behaviour of that person or a person belonging to that group in a manner that causes or is reasonably likely to cause that person or another person significant harm; (c) the placing on the market, the putting into service or the use of AI systems for the evaluation or classification of natural persons or groups of persons over acertain period of time based on their social behaviour or known, inferredor predicted personal or personality characteristics , with the social score leading to either or both of the following: (i) detrimental or unfavourable treatmentof certain natural persons or groups of persons in social contexts that are unrelated to the contexts in which the data was originally generatedor collect ed; (ii) detrimental or unfavourable treatmentof certain natural persons or groups of persons that is unjustifiedor disproportionate to their social behaviour or its gravity ; (d) the placing on the market, the putting into service for this specific purpose, or the use of an AI system for making risk assessments of natural persons in order to assess or predict the risk of a natural person committing a criminal offe nce, based solely on the profiling of a natural person or on assessing their personality traits and character istics; this prohibition shall not apply to AI systems used to support the human assessmentof the in volvementof a personina criminal activity , which is already based on objective and verifiable facts directly linked to a criminal activity ; (e) the placing on the market, the putting into service for this specific purpose, or the use of AI systems that create or expand facial recognition databases through the untarget ed scraping offacial images from the inter netor CCT V f ootage; (f) the placing on the market, the putting into service for this specific purpose, or the use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions, except where the use of the AI system is intended to be put in place or into the market for medical or safety reasons; (g) the placing on the market, the putting into service for this specific purpose, or the use of biometric categorisation systems that categorise individually natural persons based on their biometric data to deduce or infer their race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual or ientation; this prohibition does not cover any labelling or filter ing of law fullyacquired biometric datasets, suchas imag es, based on biometric data or categorizing of biometric data in the area of law enforcement ; (h) the use of ‘real-time’ remote biometric identification systems in publiclyaccessible spaces for the purposes of law enforcement, unless and insofar as suchuse is strictly necessary for one of the following objectives: (i) the targeted search for specific victims of abduction, trafficking in human beings or sexual exploitation of human beings, as well as the search for missing persons; (ii) the prevention of a specific, substantial and imminent threat to the life or physical safety of natural persons or a genuine and presentor genuine and foreseeable threatof aterror ist attack; (iii) the localisation or identification of a person suspected of having committed a criminal offence, for the purpose of conducting a criminal investigation or prosecution or executing a criminal penalty for offences referred to in Annex II and punishable in the Member State concerned bya custodial sentence or adetention order for a maximum period of at least four y ears. Point (h) of the first subparagraph is without prejudice to Article 9 of Regulation (EU) 2016/679 for the processing of biometric data for purposes other than law enforcement. 2. The use of ‘real-time’ remote biometric identification systems in publiclyaccessible spaces for the purposes of law enforcement for any of the objectives refer red to in paragraph 1, first subparagraph, point (h), shall be deployed for the purposes setout in that pointonly to confirm the identity of the specifically targeted individual, and it shall take into account the following elements: (a) the nature of the situation giving rise to the possible use, in particular the ser iousness, probabilityand scale of the harm that would be caused if the system were not used; (b) the consequences of the use of the system for the rights and freedoms of all persons concerned, in particular the ser iousness, probabilityand scale of those consequences. In addition, the use of ‘real-time’ remote biometric identification systems in publiclyaccessible spaces for the purposes of law enforcement for any of the objectives referred to in paragraph 1, first subparagraph, point (h), of this Article shall comply with necessaryand proportionate safeguards and conditions in relation to the use in accordance with the national law authorising the use thereof, in particular as regards thete mporal, geographic and personal limitations. The use of the ‘real-time’ remote biometric identification system in publiclyaccessible spaces shall be authorised only if the law enforcement authority has completed a fundamental rights impact assessment as provided for in Article 27 and has registered the system in the EU database according to Article 49. However , in duly justified cases of urgency , the use of such systems maybe commenced without the registration in the EU database, provided that such registration is completed without undue delay . 3. For the purposes of paragraph 1, first subparagraph, point (h) and paragraph 2, eachuse for the purposes of law enforcementof a ‘real-time’ remote biometric identification system in publiclyaccessible spaces shall be subject to aprior authorisation granted bya judicial authority or an independent administrative authority whose decision is binding of the Member State in which the use is to take place, issued upon a reasoned request and in accordance with the detailed rules of national law referred to in paragraph 5. How ever , in a duly justified situation of urgency , the use of such system maybe commenced without an authorisation provided that such authorisation is requested without undue delay , at the lat est within 24 hours. If such authorisation is rejected, the use shall be stopped with immediate effect and all the data, as well as the results and outputs of that use shall be immediately discarded and deleted. The competent judicial authority or an independent administrative authority whose decision is binding shall grant the authorisation only where it is satisfied, on the basis of objective evidence or clear indications presented to it, that the use of the ‘real-time’ remote biometric identification system concerned is necessary for , and proportionate to , achieving one of the objectives specified in paragraph 1, first subparagraph, point (h), as identified in the request and, in particular , remains limited to what is strictly necessary concerning the period of time as well as the geographic and personal scope. In deciding on the request, that authority shall take into account the elements referred to in paragraph 2. No decision that produces an adverse legal effe cton a person maybetaken based solely on the outputof the ‘real-time’ remote biometric identification system. 4. Without prejudice to paragraph 3, eachuse of a ‘real-time’ remote biometric identification system in publiclyaccessible spaces for law enforcement purposes shall be notified to the relevant market surveillance authorityand the national dataprotection authority in accordance with the national rules refer red to in paragraph 5. The notification shall, as a minimum, contain the information specified under paragraph 6 and shall not include sensitive operational data. 5. A Member State may decide to provide for the possibility tofully or partiallyauthorise the use of ‘real-time’ remote biometric identification systems in publiclyaccessible spaces for the purposes of law enforcement within the limits and under the conditions listed in paragraph 1, first subparagraph, point (h), and paragraphs 2 and 3. Member States concerned shall lay down in their national law the necessary detailed rules for the request, issuance and exercise of, as well as super vision and reporting relating to, the authorisations referred to in paragraph 3. Those rules shall also specify in respectof which of the objectives listed in paragraph 1, first subparagraph, point (h), including which of the criminal offences referred to in point (h)(iii) thereof, the competent authorities maybe authorised to use those systems for the purposes of law enforcement. Member States shall notify those rules to the Commission at the latest 30 days following the adoption thereof. Member States may introduce, in accordance with Uni on law , more restrictive laws on the use of remote biometric identification systems. 6. National market surveillance authorities and the national dataprotection authorities of Member States that have been notified of the use of ‘real-time’ remote biometric identification systems in publiclyaccessible spaces for law enforcement purposes pursuant to paragraph 4 shall submit to the Commission annual reports on suchuse. For that purpose, the Commission shall provide Member States and national market surveillance and dataprotection authorities with atemplate , including information on the number of the decisions taken by competent judicial authorities or an independent administrative authority whose decision is binding upon requests for authorisations in accordance with paragraph 3 and their result. 7. The Commission shall publish annual reports on the use of real-time remote biometric identification systems in publiclyaccessible spaces for law enforcement purposes, based on aggregated data in Member States on the basis of the annual reports refer red to in paragraph 6. Those annual reports shall not include sensitive operational data of the related law enforcement activities. 8. This Article shall not affect the prohibitions that apply where an AI practice infringes other Union law . CHAPTER III HIGH-RISK AI SYSTEMS SECTION 1 Classification of AI systems as high-risk