SkarpChapter 12 of 25
AI Literacy, Prohibitions and Binding High-Risk Classification
The first substantive operative duties move rapidly from workforce competence to categorical prohibitions and high-risk classification. Articles 4–10 show how the Act translates its risk hierarchy into literacy measures, bans, classification records and continuous controls over risks and data.
Article 4: AI Literacy Is a Contextual Duty
Who has the duty?
Article 4 applies to providers and deployers. They shall take measures; they cannot simply assume that users already understand an AI system.
The exact standard
The required aim is, to their best extent, a sufficient level of AI literacy among staff and other people operating or using systems on their behalf.
Literacy is not one-size-fits-all
Measures must reflect knowledge, experience, education, training, use context, and the persons or groups on whom the system is to be used.
Applying Article 4: Build Role-Specific Literacy
Scenario
A university uses an AI tool to flag students for possible support. Different people interact with the tool in different ways.
Different roles, different literacy
IT staff, advisers, managers, and contractors may need different literacy measures because their technical tasks and decisions differ.
Context and affected persons matter
For student-facing AI, literacy should address the educational setting and the people on whom the system is used, not only software operation.
Article 5(1)(a)-(d): Four Prohibited Practices
Manipulation has cumulative conditions
Article 5(1)(a) requires more than persuasive design: impaired informed decision-making, a decision otherwise not taken, and significant harm or a reasonable likelihood of it.
Vulnerability exploitation
Article 5(1)(b) specifically identifies vulnerabilities linked to age, disability, or a specific social or economic situation.
Social scoring
The prohibited outcome is detrimental treatment that is unrelated to the original context, unjustified, disproportionate to conduct or gravity, or both.
Criminal-risk assessment
Article 5(1)(d) targets prediction based solely on profiling or traits, while preserving a qualified exception for human assessment grounded in objective, verifiable facts.
Article 5(1)(e)-(g): Faces, Emotions and Biometric Categories
Facial databases
Article 5(1)(e) targets creation or expansion of facial-recognition databases through untargeted scraping from the internet or CCTV footage.
Emotion inference
Workplace and education emotion inference is prohibited, except where the intended placement or market purpose is medical or safety-related.
Biometric categorisation
The prohibition concerns individually categorising people from biometric data to infer the listed sensitive characteristics, subject to stated textual exclusions.
Article 5(1)(h)-(7): Real-Time Remote Biometric Identification
Default rule and limited objectives
Real-time remote biometric identification in publicly accessible spaces for law enforcement is prohibited unless strictly necessary for one of the three listed objectives.
Targeted confirmation only
Even where permitted, use is only to confirm the identity of the specifically targeted individual and must account for rights-and-freedoms consequences.
Safeguards before use
The text requires national-law safeguards, temporal/geographic/personal limits, a fundamental-rights impact assessment, and EU database registration, subject to urgent registration rules.
Urgent authorisation
Urgency does not erase scrutiny: authorisation must be requested without undue delay and no later than 24 hours after use begins.
Decision Exercise: Is the Biometric Use Within Article 5?
Decision Exercise: Trace Every Condition
A police authority proposes to use a real-time remote biometric identification system in a train station to identify a named suspect. The suspect is being investigated for an Annex II offence punishable in the Member State by a maximum prison sentence of five years.
Work through the source text in order:
- Is the purpose one of Article 5(1)(h)'s listed objectives?
- Is the proposed use strictly necessary, rather than merely convenient?
- Is use limited to confirming the identity of the specifically targeted individual?
- Have the authority considered both the harm if the system is not used and the consequences for all persons concerned?
- Has it completed the Article 27 fundamental-rights impact assessment (required before use — no urgency exception applies to the assessment) and registered the system in the EU database under Article 49? If registration is not yet complete, is this a duly justified urgency case allowing use to begin before registration, which must then follow without undue delay?
- Has a judicial authority or binding independent administrative authority granted prior authorisation? If urgent use began without it, was authorisation requested without undue delay, at the latest within 24 hours?
- Could an adverse legal decision be made solely from the output?
The final answer to question 7 is no. Article 5(3) states: "No decision that produces an adverse legal effect on a person may be taken based solely on the output of the ‘real-time’ remote biometric identification system."
If urgent authorisation is rejected, use must stop immediately. The data and all results and outputs of that use must be immediately discarded and deleted. Notifications, annual Member State reports, and a Commission annual aggregated report create further accountability layers.
Quiz: The Article 5 Urgency Rule
Check Your Understanding
Choose the most accurate statement about urgent law-enforcement use of a real-time remote biometric identification system in a publicly accessible space.
In a duly justified urgency situation, which statement follows Article 5(3)?
- Use may begin without prior authorisation only if authorisation is requested without undue delay, at the latest within 24 hours; if rejected, use stops immediately and data, results, and outputs are discarded and deleted.
- Use may begin without any authorisation request if the authority later files an annual report.
- The system output may itself be the sole basis for an adverse legal decision during the urgent period.
- Urgency removes the need for any temporal, geographic, or personal limitations.
Show Answer
Answer: A) Use may begin without prior authorisation only if authorisation is requested without undue delay, at the latest within 24 hours; if rejected, use stops immediately and data, results, and outputs are discarded and deleted.
Article 5(3) permits urgent commencement without authorisation only on the stated condition that authorisation is requested without undue delay and at the latest within 24 hours. Rejection triggers immediate stopping, discarding, and deletion. Article 5 also prohibits adverse legal decisions based solely on the system output.
Article 6: The Two Routes to High-Risk Classification
Route 1: Annex I products
Article 6(1) requires both Annex I product or safety-component coverage and a required third-party conformity assessment.
Route 2: Annex III
Article 6(2) begins with a binding rule: AI systems referred to in Annex III shall be considered to be high-risk.
A narrow derogation
An Annex III system may be non-high-risk only where it does not pose significant harm risk, including by not materially influencing decision-making outcomes.
Profiling overrides the derogation
An Annex III system shall always be high-risk where it performs profiling of natural persons.
Article 6 in Practice: Document the Classification
Non-high-risk is not automatic
A narrow procedural or preparatory task can fit a listed condition, but the provider must still assess whether significant risk of harm exists.
Documentation comes first
A provider considering an Annex III system non-high-risk must document its assessment before market placement or putting the system into service.
Profiling changes the result
For an Annex III system that performs profiling of natural persons, Article 6 says it shall always be considered high-risk.
Article 7: How Annex III Can Change
Adding a use case
Article 7 requires both Annex III-area relevance and risk or adverse impact equivalent to, or greater than, existing Annex III high-risk systems.
What the assessment examines
The criteria cover purpose, use scale, data, autonomy, override, documented harm, vulnerability, dependence, reversibility, benefits, and legal safeguards.
Removing a use case
Removal is possible only if significant risks no longer exist and the deletion does not decrease the overall protection level under Union law.
Article 8: The Compliance Baseline
The core compliance rule
High-risk systems shall comply with Section 2 requirements, considering their intended purpose and the generally acknowledged state of the art.
Risk management is integrated
Article 8 expressly requires the Article 9 risk-management system to be taken into account when ensuring compliance.
Avoiding duplicate processes
Where relevant product rules also apply, providers may integrate necessary AI testing, reporting, information, and documentation into existing procedures, as appropriate.
Article 9: Risk Management Throughout the Lifecycle
Not a one-time assessment
Article 9 requires a documented, maintained process across the entire lifecycle, with regular systematic review and updating.
Four process steps
Identify foreseeable risks; evaluate intended-use and foreseeable-misuse risks; use post-market data; and adopt appropriate targeted measures.
Residual risk
Each hazard's residual risk and overall residual risk must be judged acceptable after the required design, mitigation, information, and training approach.
Human context remains relevant
Risk reduction must consider the deployer's expected technical knowledge, experience, education, training, and the presumable use context.
Article 9: Testing Is Timed and Measured
Why test?
Testing identifies targeted risk-management measures and must show consistent intended-purpose performance and Section 2 compliance.
When must testing happen?
Testing occurs as appropriate throughout development and, in every case, before the system is placed on the market or put into service.
How must testing be framed?
Testing uses prior-defined metrics and probabilistic thresholds appropriate to intended purpose. The text does not set universal numerical values.
Who may be affected?
Implementation must consider likely adverse impact on people under 18 and, as appropriate, other vulnerable groups.
Article 10(1)-(4): Data Governance and Data Quality
When Article 10 applies
For high-risk systems trained with data, used training, validation, and testing datasets must meet the Article 10 quality criteria.
Governance is more than collection
Article 10 covers data origin, preparation, assumptions, suitability, bias examination and mitigation, and gaps that could prevent compliance.
The quality standard
Datasets shall be relevant, sufficiently representative, and, to the best extent possible, free of errors and complete for their intended purpose.
Fit the actual setting
Data must account, as required by intended purpose, for the specific geographical, contextual, behavioural, or functional setting of use.
Article 10(5)-(6): Special-Category Data for Bias Correction
An exceptional permission
Providers may exceptionally process special-category personal data only where strictly necessary for Article 10 bias detection and correction.
The necessity test
The purpose must not be effectively achievable with other data, including synthetic or anonymised data.
All safeguards are cumulative
Reuse limits, privacy measures, strict authorised access, no third-party access, deletion, and documented necessity all must be satisfied.
Systems without model training
For high-risk systems not using AI-model training techniques, Article 10(2)-(5) apply only to testing datasets.
Flashcards: Articles 4-10 Core Rules
Review Key Terms
Flip each card, state the rule aloud, then identify its Article.
- Article 4: Who must take AI-literacy measures?
- Providers and deployers of AI systems shall take measures for a sufficient level of AI literacy among relevant staff and other persons dealing with operation and use on their behalf.
- Article 5(1)(e): What facial-recognition practice is prohibited?
- Creating or expanding facial-recognition databases through the untargeted scraping of facial images from the internet or CCTV footage.
- Article 5(3): What is the urgent authorisation deadline?
- Authorisation must be requested without undue delay, at the latest within 24 hours.
- Article 5(3): Can adverse legal effect rest solely on real-time biometric output?
- No. No decision producing an adverse legal effect may be taken based solely on that output.
- Article 6(1): What two conditions create product-related high-risk status?
- Annex I product or safety-component coverage, and a required third-party conformity assessment under that Annex I legislation.
- Article 6(3): What defeats the Annex III non-high-risk derogation?
- An Annex III system shall always be high-risk where it performs profiling of natural persons.
- Article 9: What kind of process is risk management?
- A continuous iterative lifecycle process requiring regular systematic review and updating.
- Article 10(3): What is the data-quality rule?
- Datasets shall be relevant, sufficiently representative, and to the best extent possible, free of errors and complete in view of intended purpose.
Final Quiz: Classification and Data Governance
Final Check
Select the answer that most closely follows the source text.
Which statement is correct under Articles 6 and 10?
- Every Annex III system is automatically non-high-risk if a human reviews one output.
- An Annex III system performing profiling of natural persons shall always be considered high-risk; special-category data for bias correction may be processed only exceptionally, where the cumulative Article 10(5) conditions are met.
- A provider may treat a product-related system as high-risk whenever it falls under Annex I, even if no third-party conformity assessment is required.
- Training, validation, and testing data need only be representative if the provider processes special-category personal data.
Show Answer
Answer: B) An Annex III system performing profiling of natural persons shall always be considered high-risk; special-category data for bias correction may be processed only exceptionally, where the cumulative Article 10(5) conditions are met.
Article 6(3) makes profiling an absolute exception to the Annex III non-high-risk derogation. Article 10(5) permits exceptional processing of special-category data only where strictly necessary and where all stated safeguards and conditions are met. Article 6(1) requires both Annex I coverage and third-party conformity assessment, while Article 10(3) applies the dataset-quality rule generally where such datasets are used.
Key Terms
- Deployer
- An actor that uses or operates an AI system and, under Article 4, shall take literacy measures for relevant staff and other persons acting on its behalf.
- Provider
- A term used in Article 4 and throughout the source text for an actor with responsibilities relating to an AI system, including the Article 4 literacy duty and Article 6 classification documentation duty.
- AI literacy
- A sufficient level of understanding among relevant staff and other persons dealing with operation and use of AI systems, assessed in light of their knowledge, experience, education, training, use context, and affected persons or groups.
- Residual risk
- Risk remaining after risk-management measures; Article 9 requires each relevant hazard's residual risk and the overall residual risk to be judged acceptable.
- Safety component
- A component concept used in Article 6(1)(a), where an AI system is intended to be used as a safety component of an Annex I-covered product.
- High-risk AI system
- An AI system classified under Article 6 through the product-related route or the Annex III route, subject to the Article 6(3) derogation and profiling exception.
- Putting into service
- A legally significant act named in Article 5, Article 6, and Article 9; the provided text does not define the term.
- Placing on the market
- A legally significant act named in Article 5 prohibitions and Article 6 classification provisions; the provided text does not define the term.
- Profiling of natural persons
- A function that means an Annex III system shall always be considered high-risk under Article 6(3).
- Reasonably foreseeable misuse
- A condition of use whose risks must be estimated and evaluated under Article 9(2)(b).
- Third-party conformity assessment
- The second cumulative condition in Article 6(1) for product-related high-risk classification.
- Special categories of personal data
- Data that Article 10(5) permits providers to process exceptionally for bias detection and correction only where strictly necessary and subject to all listed conditions.
- Fundamental rights impact assessment
- An assessment referred to in Article 5(2), to be completed by the law-enforcement authority before use of a real-time remote biometric identification system, subject to the stated urgent registration rule.
- Training, validation and testing data sets
- Datasets governed by Article 10 quality and governance requirements where a high-risk AI system uses techniques involving training AI models with data.
- Real-time remote biometric identification system
- A system addressed in Article 5(1)(h)-(7) when used in publicly accessible spaces for law-enforcement purposes.