A comprehensive EU regulation adopted in 2024 that uses a risk-based approach to govern the design, development, and use of AI systems, focusing on ex ante obligations like risk management, data quality, transparency, and human oversight.

Under the EU AI Act, an AI system used in sensitive areas (e.g., safety components, employment, credit, law enforcement, medical devices) that must meet strict requirements for risk management, data, documentation, and oversight.

Product Liability (EU)

A legal regime under the Product Liability Directive that imposes strict liability on producers for damage caused by defective products, now explicitly including software and AI.

A product that does not provide the safety the public is entitled to expect, considering its presentation, foreseeable use, timing, and, for AI, behavior influenced by data, updates, and learning.

A 2024 EU regulation establishing a risk-based framework for AI systems, with strict obligations for high-risk AI related to risk management, data, transparency, and oversight.

What is Human Oversight?

Requirements that humans can meaningfully supervise, intervene in, or override AI systems, especially in high-risk contexts.

What is Strict Liability?

Liability that applies regardless of fault; the claimant need not prove negligence, only defect and causation.

What is Defective Product?

A product that fails to provide the level of safety that the public is entitled to expect, given all relevant circumstances.

Law Meets Philosophy: Responsibility and Liability for Autonomous Systems — Thinking Machines: Philosophy, Consciousness, and the Ethics of AI

1. Setting the Scene: Autonomous Systems and Responsibility

Why This Matters

Autonomous systems like self-driving cars and medical AI can cause serious harm. When they do, we must ask: who is legally liable, and who is morally responsible?

Old Law, New Tech

Traditional law assumed human decision-makers and static products. AI challenges this because it can learn after sale, involves many actors, and may be hard to explain.

Key EU Tools

Two major EU developments now shape this area: the EU AI Act (risk-based regulation of AI) and the revised Product Liability Directive (covering software and AI as products).

Our Aim

You will learn how these laws try to close responsibility gaps for AI, and where deep philosophical questions about moral responsibility remain unresolved.

2. The EU AI Act: A Risk-Based Approach

What the AI Act Does

The EU AI Act, adopted in 2024, sets before-the-fact rules for how AI must be built and used. It does not directly decide who pays damages but shapes duties for AI actors.

Risk Categories

The Act classifies AI into: unacceptable risk (banned), high-risk (strict duties), limited risk (transparency), and minimal risk (few specific rules).

High-Risk Examples

High-risk AI includes safety components in products, AI for hiring, credit scoring, law enforcement, migration control, and many medical and critical infrastructure systems.

Key Duties for High-Risk AI

Providers must manage risks, use high-quality data, keep documentation and logs, ensure transparency, build in human oversight, and guarantee robustness and cybersecurity.

Link to Liability

If an AI provider ignores these duties and harm occurs, that non-compliance can later count as evidence of defect or negligence under liability rules.

3. Example: AI Act and an Autonomous Vehicle

AutoMind's AI Car System

AutoMind builds an AI driving system for autonomous vehicles. Because it controls a safety function, it is a high-risk AI system under the EU AI Act.

Duties AutoMind Owes

AutoMind must manage risks, use high-quality and diverse training data, keep documentation and logs, and design meaningful human oversight into the driving system.

A Nighttime Accident

The system fails to detect a pedestrian at night, causing a fatal crash. Investigators learn AutoMind skipped nighttime testing and did not log key AI decisions.

Consequences of Non-Compliance

Regulators can fine AutoMind under the AI Act. Victims can also point to these rule breaches as evidence of defect or negligence in civil liability claims.

Takeaway

The AI Act does not itself award damages, but it sets concrete standards. Breaking those standards becomes powerful evidence in later responsibility and liability disputes.

4. Product Liability: AI and Software as Products

Why Update Product Liability?

The original 1985 EU product liability rules were built for physical goods. The 2024 revision explicitly includes software and AI as products.

AI as a Product

Under the revised PLD, software and AI models count as products, whether embedded in devices or supplied as standalone applications.

Strict Liability

Producers are strictly liable for damage caused by defective products. Victims do not need to prove fault, only that the product was defective and caused harm.

What Counts as Defect?

A product is defective if it lacks the safety the public is entitled to expect, considering marketing, foreseeable use, timing, and for AI, updates, data, and learning behavior.

Link to AI Act

If an AI system violates mandatory safety rules (like the AI Act), that non-compliance can be used as evidence that the product was defective.

5. Thought Exercise: Who Is Liable?

Consider this scenario and decide who you think is legally liable under EU-style rules.

Scenario:

MedAI develops a diagnostic AI that analyzes chest X-rays.
A hospital deploys the AI and relies heavily on its outputs.
A patient receives a false negative (the AI misses a clear sign of cancer), leading to delayed treatment and serious harm.
Investigations show:
MedAI did not properly test the system on images from older X-ray machines, common in smaller hospitals.
The hospital ignored the user manual warning that the AI was validated only for newer machines.

Reflect step by step:

Under the AI Act

MedAI may have violated risk management and data quality duties.
The hospital may have misused the system contrary to instructions.

Under the revised Product Liability Directive

The AI is a product. MedAI is a producer.
The system may be defective if it did not provide the expected safety, given how it was marketed and foreseeable uses.

National fault-based liability (tort law)

The hospital might be negligent for over-relying on the AI and ignoring warnings.

Your task:

Write down, in 2–3 bullet points, how you would allocate legal liability among MedAI, the hospital, and possibly others.
Then add 1–2 bullet points on who you think is morally responsible, and whether that matches the legal outcome.

Use this structure:

Legal liability:
...
...
Moral responsibility:
...
...

There is no single correct answer, but try to justify your allocation using ideas from the AI Act and product liability.

6. Responsibility Gaps and Multi-Actor Systems

What Is a Responsibility Gap?

A responsibility gap arises when harm occurs but it is unclear who is responsible, or everyone seems only partly responsible, especially in complex AI systems.

Many Hands Problem

AI involves data providers, developers, integrators, deployers, and users. Responsibility can become diffused because each actor contributes only a piece.

Legal Responses

The AI Act assigns roles and duties. Product liability uses strict and sometimes joint liability so at least one actor is clearly on the hook for defects.

Philosophical Questions

If AI learns in unforeseen ways, is the developer morally responsible? When an accurate system harms a few, does anyone owe special responsibility to those individuals?

Link to Ethics

Utilitarianism, deontology, and virtue ethics offer different views on how to distribute responsibility and what level of risk is morally acceptable.

7. Quick Check: AI Act vs Product Liability

Test your understanding of how the EU AI Act and the revised Product Liability Directive interact.

Which statement best captures the relationship between the EU AI Act and the revised Product Liability Directive (PLD) regarding AI systems?

The AI Act and PLD both directly award damages to victims, so they overlap and usually only one will apply.
The AI Act sets ex ante safety and governance duties for AI, while the PLD provides ex post strict liability for defective AI products, with AI Act non-compliance serving as evidence of defect.
The PLD replaces the AI Act for all high-risk AI systems, because product liability is stricter than regulatory compliance.

Show Answer

Answer: B) The AI Act sets ex ante safety and governance duties for AI, while the PLD provides ex post strict liability for defective AI products, with AI Act non-compliance serving as evidence of defect.

The AI Act focuses on ex ante obligations (risk management, data quality, human oversight). The revised PLD governs ex post strict liability for defective products, now including AI and software. Non-compliance with AI Act duties can be used as evidence that an AI product was defective, but the PLD does not replace the AI Act.

8. Moral vs Legal Responsibility: A Short Reflection

Take 2–3 minutes to reflect on how legal liability and moral responsibility can come apart.

Prompt:

Think of a case (real or imagined) where an autonomous system caused harm but no one was found legally liable, or liability was minimal.
Ask yourself:

Do you still feel that someone is morally responsible? Who?
Or do you think this is a case of moral bad luck, where no one could reasonably prevent the harm?

Connect to ethical theories:

A utilitarian might focus on overall risk–benefit and accept some unavoidable harms.
A deontologist might insist on strict duties not to use systems that could violate rights, even if the overall benefits are large.
A virtue ethicist might ask whether the designers and deployers acted with appropriate care, humility, and honesty about limitations.

Write a short paragraph (4–6 sentences) answering:

How should we treat cases where the law finds no liable party, but people still feel that something morally wrong happened?

You will not be graded on the "right" answer here; the goal is to practice connecting legal rules to philosophical ideas about responsibility.

9. Key Terms Review

Flip these cards (mentally or with a partner) to review the core concepts from this module.

EU AI Act: A comprehensive EU regulation adopted in 2024 that uses a risk-based approach to govern the design, development, and use of AI systems, focusing on ex ante obligations like risk management, data quality, transparency, and human oversight.
Risk-Based Regulation: A regulatory approach that tailors rules to the level of risk posed by a technology or activity, imposing stricter obligations on high-risk uses and lighter rules on lower-risk ones.
High-Risk AI System: Under the EU AI Act, an AI system used in sensitive areas (e.g., safety components, employment, credit, law enforcement, medical devices) that must meet strict requirements for risk management, data, documentation, and oversight.
Product Liability (EU): A legal regime under the Product Liability Directive that imposes strict liability on producers for damage caused by defective products, now explicitly including software and AI.
Defective Product: A product that does not provide the safety the public is entitled to expect, considering its presentation, foreseeable use, timing, and, for AI, behavior influenced by data, updates, and learning.
Responsibility Gap: A situation where harm occurs but it is unclear who is responsible, often because many actors contributed to the outcome or because the behavior of an autonomous system is hard to attribute.
Strict Liability: Liability that does not depend on proving fault or negligence; the producer is liable if a defective product causes damage, regardless of how careful they were.
Ex Ante vs Ex Post: Ex ante rules apply before harm occurs (e.g., design and safety requirements). Ex post rules apply after harm occurs (e.g., compensation through liability law).

10. Bringing It Together: Law Meets Philosophy

Legal Tools

The EU AI Act regulates AI risks before harm occurs, while the revised Product Liability Directive provides strict liability for defective AI products after harm occurs.

Interaction of Laws

Non-compliance with AI Act duties can be used as evidence that an AI product is defective under product liability, linking governance and compensation.

Persistent Responsibility Gaps

Even with these laws, multi-actor, learning systems create responsibility gaps where it is hard to assign clear moral responsibility for harms.

Beyond Compliance

Legal rules set minimum standards. Ethical design and deployment of AI often require going further, especially where human lives and rights are at stake.

Your Ongoing Questions

Keep asking: where do law and morality diverge in AI responsibility, and how should we design systems and institutions to narrow that gap?

Key Terms

EU AI Act: A 2024 EU regulation establishing a risk-based framework for AI systems, with strict obligations for high-risk AI related to risk management, data, transparency, and oversight.
Human Oversight: Requirements that humans can meaningfully supervise, intervene in, or override AI systems, especially in high-risk contexts.
Strict Liability: Liability that applies regardless of fault; the claimant need not prove negligence, only defect and causation.
Defective Product: A product that fails to provide the level of safety that the public is entitled to expect, given all relevant circumstances.
Ex Post Liability: Legal mechanisms that apply after harm occurs, such as claims for compensation in tort or product liability.
Ex Ante Regulation: Rules that apply before harm occurs, guiding design and deployment (e.g., safety standards).
Responsibility Gap: A situation where harm occurs but it is hard to attribute responsibility to any single agent, often due to complex, distributed systems.
High-Risk AI System: An AI system used in sensitive or safety-critical domains that is subject to enhanced requirements under the EU AI Act.
Risk-Based Regulation: A method of regulation where the strictness of rules depends on the level of risk posed by an activity or technology.
Product Liability Directive (PLD): EU legislation imposing strict liability on producers for damage caused by defective products; revised in 2024 to include software and AI.