SkarpChapter 3 of 25
The Boundary of Unacceptable AI
The Act draws its hardest line around practices considered incompatible with Union values. This module examines the rationales for banning manipulation, exploitation, social scoring and other practices while permitting only tightly controlled law-enforcement biometric exceptions.
1. Reading the Boundary: What These Recitals Do
Recitals set the rationale
This lesson follows Recitals (28)-(45). They explain why some AI practices cross an unacceptable boundary, but they usually say should, not the operative Article 5 wording shall.
The values at stake
Recital (28) grounds the boundary in dignity, freedom, equality, democracy, the rule of law, and Charter rights including non-discrimination, data protection, privacy, and children's rights.
Currency note
A July 8, 2026 Digital Omnibus amendment is signed pending publication, not in force. It adds other Article 5 prohibitions, but does not change the source text taught here today.
2. Manipulation and Exploitation: Harmful Distortion
Not every influence is manipulation
Recital (29) focuses on an objective or effect of materially distorting behaviour, paired with significant harm or a reasonable likelihood of it. Mere persuasion is not enough on this account.
How distortion can occur
The recital names subliminal stimuli beyond perception and manipulative or deceptive techniques that people cannot recognise, control, or resist. VR and machine-brain interfaces may intensify control of stimuli.
Vulnerability and accumulated harm
Exploitation may concern age, disability, or a specific social or economic situation. The harm can affect the targeted person, another person, or groups, and may accumulate over time.
Intent is not always required
External, unforeseeable, unmitigable factors may prevent assuming intent to distort. But intent to cause significant harm is unnecessary where harm results from manipulative or exploitative AI-enabled practices.
3. Worked Examples: Distortion, Consent, and Legitimate Practice
Scenario A: harmful VR pressure
Imperceptible cues and constrained choices can fit Recital (29) where they appreciably undermine autonomous decision-making and cause, or are reasonably likely to cause, significant financial harm.
Scenario B: exploitation is conditional
A vulnerable group alone is not the complete test. Connect vulnerability, exploitation, material behavioural distortion, and significant harm or a reasonable likelihood of harm.
Scenario C: medical treatment
Lawful psychological treatment or physical rehabilitation, carried out under applicable law and medical standards, such as with explicit consent, should not be affected by this recital's prohibition rationale.
Scenario D: advertising
Compliant ordinary advertising should not, in itself, count as harmful manipulation. The AI Act rationale also complements consumer-law rules against unfair practices, whether AI is used or not.
4. Biometric Categorisation and Social Scoring
Sensitive inference from biometrics
Recital (30) concerns using faces, fingerprints, or other biometric data to deduce or infer listed political, religious, union, racial, sex-life, or sexual-orientation characteristics.
What remains outside that account
Lawful sorting of legally acquired biometric datasets according to biometric attributes themselves, such as hair or eye colour, should not be covered. The distinction is between sorting and inferring sensitive traits.
The structure of social scoring
Social scoring can combine behaviour across contexts with known, inferred, or predicted personal or personality characteristics over time, producing a score about people or groups.
The objectionable outcome
Recital (31) focuses on detrimental treatment in an unrelated social context, or treatment disproportionate or unjustified to the gravity of behaviour. Specific lawful evaluations remain distinct.
5. Real-Time Remote Biometric Identification: A Narrow Exception Framework
Why real-time use is especially intrusive
In public spaces, real-time remote identification may affect much of the population, normalise surveillance, deter assembly, and amplify bias. Its immediacy leaves limited time to verify or correct errors.
Not a general policing tool
The recital describes prohibition except for exhaustively listed, narrowly defined situations. Use must be strictly necessary for a substantial public interest whose importance outweighs the risks.
The four-year seriousness threshold
For annex-listed offences, the law of the Member State must provide "a custodial sentence or a detention order for a maximum period of at least four years".
Targeting and preparation
Use should confirm a specifically targeted individual's identity and be limited by time, geography, and persons. A fundamental-rights impact assessment and database registration are generally required first.
6. Authorisation, Urgency, National Rules, and Data Limits
Prior authorisation is the principle
Each use should have express, specific authorisation from a judicial authority or an independent administrative authority whose decision is binding. The normal sequence is authorisation before use.
Urgency has strict conditions
Urgency requires a duly justified case where prior authorisation is effectively and objectively impossible. Use should be the absolute minimum necessary, with safeguards set by national law and the case.
The 24-hour requirement
In urgency, the authority should request authorisation with reasons "without undue delay and at the latest within 24 hours." Rejection means immediate cessation and deletion of linked data.
National choice and notification
Member States may decline to create any authorisation route. Where they adopt detailed rules allowing it, "Such national rules should be notified to the Commission within 30 days of their adoption."
7. Predictive Policing, Facial Scraping, and Emotion Detection
Prediction is not proof
Recital (42) rejects criminal-risk assessment based solely on profiling or personality traits. It does not address analytics that are not based on individual profiling, such as suspicious-transaction analysis of undertakings.
Untargeted facial scraping
"AI systems that create or expand facial recognition databases through the untargeted scraping of facial images from the internet or CCTV footage, should be prohibited".
Emotion detection in unequal settings
Limited reliability and cultural variation combine with workplace and education power imbalances. The recital distinguishes systems strictly placed on the market for medical or safety reasons, such as therapeutic systems.
Other Union law still matters
Recital (45) says this Regulation should not affect practices prohibited under Union data-protection, non-discrimination, consumer-protection, or competition law.
8. Boundary Lab: Classify Four Cases
Decide, then check your reasoning
For each case, identify the recital most directly relevant and the condition that matters most.
- A university installs a camera system that infers whether students are bored, anxious, or engaged during lectures, then sends reports to instructors. Is the key issue Recital (44), and why does the educational setting matter?
- A police authority uses a live public-square camera to locate a named missing child. It has a suitable reference database, a completed fundamental-rights impact assessment, and an express specific authorisation. Which elements point toward the narrow framework in Recitals (33)-(35)? What additional question remains about national law under Recital (37)?
- A government ranks residents using benefit claims, transport fines, and online activity, then denies housing access on the resulting score. Identify the two outcome features in Recital (31): is the housing decision in an unrelated social context, and is it disproportionate or unjustified to the gravity of the behaviour?
- Customs analyses shipping routes and declared cargo patterns to predict where illicit goods may be located. Does Recital (42) describe this as the prohibited type of individual risk assessment? Explain using its express distinction.
Self-check
- Workplace or education emotion detection is the focal issue; the power imbalance and intrusive nature matter.
- The case resembles an exhaustively listed purpose, but the Member State must also have expressly provided an authorisation possibility in detailed national rules.
- It is the harmful cross-context or disproportionate/unjustified treatment that makes the scoring practice unacceptable.
- No: the recital gives comparable non-individual profiling risk analytics as an example outside that prohibition rationale.
9. Flashcards: Recall the Conditions
Flip each card, then state the condition from memory before reading the answer.
- What is the central concern in Recital (29)?
- Material distortion of behaviour through subliminal, manipulative, deceptive, or exploitative AI-enabled practices, causing or reasonably likely to cause significant harm.
- Does Recital (29) require intent to cause significant harm?
- No. It says intent to cause significant harm is not necessary, provided harm results from the manipulative or exploitative AI-enabled practice.
- What makes social scoring unacceptable in Recital (31)?
- A score leading to detrimental or unfavourable treatment in unrelated social contexts, or treatment disproportionate or unjustified to the gravity of social behaviour.
- What is the criminal-offence threshold for the Recital (33) biometric-identification exception?
- The offence must be punishable by "a custodial sentence or a detention order for a maximum period of at least four years" in the Member State concerned.
- What is the urgent authorisation deadline in Recital (35)?
- The authority should request authorisation "without undue delay and at the latest within 24 hours."
- What facial-database practice does Recital (43) identify?
- Creating or expanding facial recognition databases through untargeted scraping of facial images from the internet or CCTV footage.
10. Quiz: The Urgency Exception
Choose the answer that most precisely follows Recital (35).
In a duly justified urgent case, which sequence best reflects Recital (35)?
- The authority may use the system without any authorisation request if the objective is important.
- The authority may begin only where prior authorisation is effectively and objectively impossible, restrict use to the absolute minimum necessary, and request authorisation without undue delay and at the latest within 24 hours.
- The authority may use the system for 24 hours before deciding whether an authorisation is needed.
- The authority may rely solely on system output if it deletes the data later.
Show Answer
Answer: B) The authority may begin only where prior authorisation is effectively and objectively impossible, restrict use to the absolute minimum necessary, and request authorisation without undue delay and at the latest within 24 hours.
Recital (35) permits an urgency exception only under the stated impossibility condition. It requires minimum necessary use, safeguards, and a request for authorisation without undue delay and no later than 24 hours. It also says no adverse legal-effect decision should be based solely on the output.
11. Quiz: Reading Social Scoring Precisely
Choose the answer that keeps Recital (31)'s qualifiers intact.
Which practice most closely matches the unacceptable social-scoring outcome described in Recital (31)?
- A university grades a student's submitted examination under its published assessment rules.
- A bank assesses a loan application for that particular loan using information lawfully relevant to creditworthiness.
- A city combines residents' behaviour across multiple contexts into a score and uses it to impose housing restrictions unrelated to where the data was collected, without justification proportionate to the behaviour.
- A police officer verifies the identity of an unconscious accident victim during an in-person identity check.
Show Answer
Answer: C) A city combines residents' behaviour across multiple contexts into a score and uses it to impose housing restrictions unrelated to where the data was collected, without justification proportionate to the behaviour.
Recital (31) focuses on scoring based on multiple social-behaviour data points or personal characteristics over time that leads to detrimental treatment in an unrelated social context, or treatment disproportionate or unjustified to the gravity of behaviour. It preserves lawful evaluation for a specific purpose under Union and national law.
Key Terms
- Recital
- A preamble paragraph that explains context, objectives, and interpretation. In this source slice, recital language such as "should" must not be rewritten as an operative standalone "shall" duty.
- Lex specialis
- A more specific legal framework that governs a particular issue in preference to a more general rule. Recital (38) uses this concept for the Regulation's law-enforcement real-time biometric-identification framework.
- Social scoring
- Evaluation or classification of natural persons or groups using multiple data points about social behaviour in multiple contexts or known, inferred, or predicted personal or personality characteristics over time.
- Untargeted scraping
- Collecting facial images broadly from the internet or CCTV footage without targeting particular individuals, in order to create or expand facial-recognition databases.
- Biometric categorisation
- A system using biometric data, such as a face or fingerprint, to categorise people. Recital (30) focuses on deduction or inference of listed sensitive characteristics.
- Materially distorting behaviour
- Changing behaviour in the way described in Recital (29), through techniques that subvert or impair autonomy, decision-making, or free choice, together with the recital's significant-harm condition.
- Fundamental-rights impact assessment
- An assessment that Recital (34) says the relevant law-enforcement authority should complete before use of real-time remote biometric identification, subject to the recital's qualifications.
- Real-time remote biometric identification
- Remote identification of natural persons in publicly accessible spaces operating in real time; Recitals (32)-(39) describe a specific law-enforcement framework and narrow exceptions.