SkarpSkarp

Chapter 4 of 11

Are We High-Risk? Classifying Your AI Systems Step by Step

Before you can comply, you need to know which of your systems are high-risk; walk through a practical classification workflow that turns abstract annexes into a concrete inventory of AI systems and their risk levels.

15 min readen

Step 1 – What Counts As an AI System Under the EU AI Act?

Why Start With the Definition?

Before you can label anything high-risk, you must know what actually counts as an AI system under the EU AI Act. Non-AI software is out of scope for the Act’s risk rules.

Core Idea of the AI Definition

An AI system is software that uses learning or complex inference methods and generates outputs like predictions, recommendations, or decisions that influence real-world actions.

Typical In-Scope Systems

In-scope examples: credit scoring models, computer vision defect detectors, LLM-based chatbots, or expert systems that infer eligibility or risk from input data.

Typical Out-of-Scope Tools

Out-of-scope examples: simple if-else business logic, static reports, spreadsheets with formulas, or basic CRUD apps that do not infer new patterns from data.

Rule of Thumb for Your Inventory

If a component predicts, classifies, recommends, ranks, or decides using data-driven models or complex inference, treat it as an AI system for classification.

Step 2 – Quick Sorting: AI or Not?

Apply the Act’s definition to some borderline cases. For each, decide: AI system or not an AI system under the AI Act.

  1. Email spam filter that uses a trained model to label messages as spam or not.
  2. Warehouse routing script: a programmer hard-codes a fixed picking route for each order type.
  3. Customer churn predictor: a model trained on past subscriptions to predict who will cancel.
  4. Excel sheet with formulas that always compute the same tax rate from income.

Your task:

  • Write down your classification for each (AI / Not AI).
  • Then check the reasoning below.

Suggested reasoning (self-check):

  1. Spam filter: AI system. It uses a trained model to classify messages.
  2. Routing script: Not AI (as described). It is fixed, deterministic logic with no learning or complex inference.
  3. Churn predictor: AI system. It is a predictive model trained on data.
  4. Excel tax sheet: Not AI. It has no learning or inference; it just applies fixed formulas.

If you hesitated on any, note why. These edge cases are where misclassification often starts.

Step 3 – Map Your AI Systems Into an Inventory

Why an AI Inventory?

You cannot classify risk if you do not know what AI you have. A basic AI inventory lists all AI systems, who owns them, and what they do.

Core Fields to Capture

Track for each system: name, description, owner, purpose, delivery type (embedded vs stand-alone), users, affected persons, and input/output types.

Embedded vs Stand-Alone

Note whether the AI is inside a physical product (like a medical device) or is stand-alone software (like a SaaS credit scoring API). This matters for Annex II vs III.

Simple Implementation

A spreadsheet with 8–10 well-chosen columns is enough at first. Focus on consistency so you can later add risk level and legal basis per system.

Step 4 – Worked Example: Building a Mini AI Inventory

Meet DataBright’s AI Systems

Consider a fictional company, DataBright, with several AI tools. We will build a mini inventory for three of them and later classify their risk.

System 1: FactoryVision QA

FactoryVision QA is a computer vision model embedded on a production line camera. It labels parts as defective or OK and highlights defect locations for operators.

System 2: UniHire Screen

UniHire Screen is an HR analytics model. It ranks job applicants based on CVs and forms, integrated as a stand-alone web service in the recruitment system.

System 3: SmartThera Pump AI

SmartThera Pump AI is an embedded control algorithm in an insulin pump. It adjusts insulin doses using glucose sensor data, directly affecting patient health.

Why This Matters

Each description includes purpose, delivery type, users, and affected persons. These details are exactly what you need to decide if a system is high-risk.

Step 5 – Annex II: High-Risk AI Embedded in Regulated Products

Annex II: The Product Route to High-Risk

Annex II covers AI that is embedded in products already regulated by EU product safety laws, like medical devices, machinery, lifts, toys, and some vehicles.

Key Question to Ask

Is the AI part of a product that must be CE-marked under EU safety rules? If yes, and the AI performs a safety function, it is usually high-risk under Annex II.

SmartThera Pump Example

SmartThera Pump AI sits inside an insulin pump, a medical device listed in Annex II. Because it controls insulin dosage, it is clearly safety-related and high-risk.

Inventory Tip

Add a column like "Annex II product?" and note the specific law, such as Medical Devices Regulation or Machinery Regulation, for each relevant AI system.

Step 6 – Annex III: Stand-Alone High-Risk AI Use Cases

Annex III: The Use-Case Route

Annex III lists stand-alone AI use cases that are high-risk because of their impact areas, like employment, credit, education, critical infrastructure, and law enforcement.

Typical Annex III Areas

Key areas include biometric ID, critical infrastructure, education, employment and worker management, essential services like credit, law enforcement, migration, and justice.

How to Check Annex III

For each system, read its purpose. If it clearly matches an Annex III area and is not an Annex II product, it is a stand-alone high-risk AI system.

UniHire Screen Example

UniHire Screen ranks job applicants. This falls under employment and worker management in Annex III, so it is high-risk via Annex III, even though it is stand-alone.

Inventory Tip

Add an "Annex III category" column in your inventory to note which high-risk use-case, if any, each AI system falls under.

Step 7 – Quiz: Annex II or Annex III or Neither?

Test your ability to distinguish Annex II from Annex III high-risk systems.

Your company has built an AI system that automatically scores and ranks university entrance exam answers and assigns applicants to programs. It is delivered as a cloud service to universities, not embedded in any physical device. How should you classify it under the AI Act?

  1. High-risk via Annex II (embedded product safety AI)
  2. High-risk via Annex III (stand-alone high-risk use case)
  3. Not high-risk under the AI Act
Show Answer

Answer: B) High-risk via Annex III (stand-alone high-risk use case)

This AI is not embedded in a regulated product like a medical device or machinery, so Annex II does not apply. It directly affects education access and student allocation, which is one of the Annex III stand-alone high-risk areas. Therefore it is high-risk via Annex III.

Step 8 – Design Your Own Classification Workflow

Now connect everything into a step-by-step workflow you could actually use in an organization.

Use this template and adapt it to your context:

  1. Create or update your AI inventory.
  • List all candidate systems that might be AI.
  1. AI or not?
  • For each system, apply the AI definition (predicts/classifies/recommends/decides using data-driven or inference methods?).
  • Mark non-AI systems and remove them from the AI inventory.
  1. Check Annex II (embedded product).
  • Is the system embedded in a product subject to EU product safety rules listed in Annex II?
  • If yes and it performs a safety function, mark: "High-risk – Annex II".
  1. Check Annex III (stand-alone use case).
  • If not Annex II, does the purpose match any Annex III category (employment, credit, education, etc.)?
  • If yes, mark: "High-risk – Annex III" and note the category.
  1. If neither Annex II nor Annex III applies:
  • Classify the system as limited-risk (if it triggers transparency obligations) or minimal-risk based on your previous module.
  1. Record decisions and reasoning.
  • For each system, store: risk level, Annex II/III reference (if any), and a short justification.

Your task:

  • Sketch how you would implement steps 1–6 in a small team (e.g., who fills the inventory, who signs off on the risk level, how often you review it). Write a short 5–6 line outline for yourself.

Step 9 – Flashcards: Key Terms for Classification

Use these flashcards to reinforce the core concepts you need for classifying AI systems under the EU AI Act.

AI system (EU AI Act – simplified)
Software using learning or complex inference techniques that, for defined objectives, generates outputs like predictions, recommendations, or decisions that influence environments.
Annex II high-risk AI
AI systems embedded in products regulated by existing EU product safety laws (e.g., medical devices, machinery) where the AI performs a safety-related function.
Annex III high-risk AI
Stand-alone AI systems used in specific high-impact areas (e.g., employment, credit scoring, education, critical infrastructure, law enforcement) listed in Annex III.
Embedded AI system
An AI component that is integrated into a physical or digital product (such as a medical device or machine) and operates as part of that product’s safety or functionality.
Stand-alone AI system
An AI system delivered as software or a service (e.g., a web API or internal tool) that is not tied to a specific regulated product but may still be high-risk via its use case.
AI inventory
An internal register listing all AI systems, including their purpose, owner, delivery type, users, affected persons, and whether they fall under Annex II or Annex III.

Key Terms

Annex II
Part of the EU AI Act listing existing EU product safety legislation. AI systems embedded in products regulated under these laws and performing safety functions are classified as high-risk.
AI system
Under the EU AI Act, software that uses learning or complex inference methods to generate outputs (predictions, recommendations, decisions, content) that influence environments for given objectives.
Annex III
Part of the EU AI Act listing specific stand-alone high-risk AI use cases, such as in employment, credit, education, critical infrastructure, law enforcement, migration, and justice.
Embedded AI
An AI system integrated into a physical or digital product (e.g., a medical device, machine, or toy), typically subject to product safety and CE-marking rules.
AI inventory
An organization’s structured list of AI systems, capturing key attributes like purpose, owner, inputs/outputs, delivery type, users, affected persons, and risk classification.
Stand-alone AI
An AI system provided as independent software or a service, not tied to a specific regulated product but potentially high-risk due to its use case under Annex III.
High-risk AI system
An AI system that falls under Annex II (embedded in certain regulated products with safety functions) or Annex III (specific stand-alone high-risk use cases) of the EU AI Act.
Conformity assessment
A process under EU law to demonstrate that a product or high-risk AI system meets applicable legal and technical requirements before it is placed on the market.
Limited-risk AI system
AI systems that are not high-risk but trigger specific obligations under the EU AI Act, such as transparency duties (for example, some chatbots or emotion recognition tools).
Minimal-risk AI system
AI systems with very low or no regulatory obligations under the EU AI Act, such as many recommender systems or spam filters, provided they do not fall under Annex II or III.

Finished reading?

Test your understanding with a custom practice exam on this chapter.

Test yourself