When does high-risk AI apply to your SaaS?
The EU AI Act Annex III lists eight use case areas where AI is considered high-risk. These are not hypothetical future categories — they cover products that exist today and that many SaaS companies are building or deploying.
You need to read this carefully: Annex III is not based on the sophistication of the AI, the size of the company, or the number of users affected. It is based on the intended purpose of the AI system and the domain it operates in. A simple rules-based CV filtering algorithm used in recruitment is still subject to high-risk obligations. A highly sophisticated recommendation engine for a retail app probably is not.
The eight Annex III areas and the SaaS use cases they cover:
| Annex III Area | SaaS use cases that fall in | Common SaaS use cases that don't |
|---|---|---|
| §1 Biometrics | Facial recognition for identity verification, emotion recognition for security screening | Face unlock on consumer devices (different safeguards), liveness detection for login |
| §2 Critical infrastructure safety | AI managing energy/water/transport infrastructure safety components | Standard B2B SaaS for utility companies (no safety function) |
| §3 Education & vocational training | AI determining admission to schools, assessing student exams, detecting cheating | Personalised learning recommendations, content delivery, quizzes with human grading |
| §4 Employment & HR | CV screening, recruitment ranking, performance evaluation, task allocation, promotion decisions | HR admin tools, payroll, scheduling (no evaluation/ranking of individuals) |
| §5 Essential services access | Credit scoring, creditworthiness assessment, health risk scoring, public benefits eligibility | Fraud detection (detection only, not access decisions), B2B financial analytics |
| §6 Law enforcement | AI predicting criminal risk, lie detection in investigations, criminal pattern analysis by police | Fraud prevention for private companies, security analytics (no law enforcement function) |
| §7 Migration & border | Asylum case processing, border control risk assessment, visa applications | Not relevant to commercial SaaS |
| §8 Justice & democracy | AI in court proceedings, election targeting based on political opinions | Legal research tools, contract review (no decision-making in justice proceedings) |
The full set of Art. 9–15 obligations: what each requires
Art. 9 — Risk management system
The risk management system is not a one-time document — it's a continuous process throughout the AI system's lifecycle. It must:
- Identify and analyse all known and reasonably foreseeable risks to health, safety, and fundamental rights
- Estimate and evaluate risks from reasonably foreseeable misuse
- Adopt risk management measures and verify their effectiveness through testing
- Document all of the above
The key word is lifecycle. The risk management system must continue after the product is deployed — informed by post-market monitoring data, incident reports, and changes in the AI system's operating environment.
Art. 10 — Data and data governance
Training, validation, and testing datasets must be subject to data governance practices covering:
- Design choices about data collection methodologies
- Data preparation, annotation, and labelling processes
- Examination of datasets for biases that could lead to discriminatory outcomes (Art. 10(2)(f))
- Measures to address identified biases
Art. 10(5) allows the processing of special category personal data in datasets where "strictly necessary for the purpose of ensuring bias monitoring, detection, and correction" — but only with appropriate safeguards.
Art. 11 / Annex IV — Technical documentation
Technical documentation must be drawn up before the AI system is placed on the market and kept up to date throughout its lifecycle. It must be comprehensive enough for a conformity assessment body to evaluate compliance. Annex IV specifies what's required, including:
- General description: intended purpose, persons affected, categories of natural persons, hardware/software interaction
- Design specifications: development choices, training methodology, algorithmic approach
- Training dataset information: sources, scope, data collection processes, labelling, bias examination
- Validation and testing: evaluation methodology, results, metrics, known limitations
- Risk management documentation
- Post-market monitoring plan
Art. 12 — Logging
High-risk AI systems must have automatic logging capabilities that enable traceability of outputs — particularly to identify risks and to allow for monitoring. Logs must capture at minimum: the period of each use, the reference database used (for biometric systems), the input data, and the identity of any persons involved in verifying results.
Deployers must retain these logs for at least 6 months — or longer as required by applicable law (e.g. longer retention under financial regulation).
Art. 13 — Transparency toward deployers
Providers must supply deployers with instructions for use (Annex IV §9) that include:
- Identity and contact details of the provider
- Characteristics and capabilities of the AI system
- Level of accuracy and known limitations
- Performance on specific groups or categories of persons (bias and limitation disclosure)
- Specifications of input data
- Instructions for setting up and maintaining human oversight
- Expected lifetime of the AI system and any maintenance/update requirements
Art. 14 — Human oversight
The AI system must be designed to enable natural persons to:
- Fully understand the system's capabilities and limitations
- Detect and address anomalies, dysfunctions, and unexpected performance
- Contextualise and interpret outputs — not rely on them blindly
- Decide to not act on or override outputs in specific situations
- Intervene or interrupt the AI system through a stop button or similar mechanism
For deployers: the persons assigned to human oversight must have the competence, authority, and resources necessary to exercise this oversight. This means training, clear authority to override the AI, and time to actually review outputs — not just a nominal oversight function.
Art. 15 — Accuracy, robustness, and cybersecurity
The AI system must achieve the appropriate level of accuracy as declared in technical documentation. It must also be resilient to:
- Errors and faults in training, validation, and testing data
- Adversarial attacks — attempts to alter behaviour through manipulated inputs (adversarial examples, data poisoning, model evasion, model inversion)
- Cybersecurity threats that could compromise the integrity of model outputs
Conformity assessment (Art. 43)
For most Annex III high-risk AI systems, conformity assessment is self-assessment under Annex VI — the provider conducts and documents the assessment themselves and draws up the EU Declaration of Conformity. No third-party notified body is required for most categories.
The exception: AI systems that are safety components of products covered by EU harmonised legislation (Annex I), and biometric identification systems. These require a third-party conformity assessment by a notified body (Annex VII).
Registration in the EU AI database (Art. 49)
Before placing a high-risk AI system on the EU market, providers must register it in the EU AI database — maintained by the European Commission. The database is public for most information. Deployers of systems in Annex III §1 (biometrics), §6 (law enforcement), and §7 (migration) must also register their use.
Post-market monitoring and incident reporting (Art. 72-73)
Post-market monitoring is not just good practice for high-risk AI — it's mandatory. Providers must actively collect data on the AI system's performance in the real world throughout its lifecycle. The monitoring plan must be documented as part of technical documentation.
Serious incidents must be reported to the national market surveillance authority without undue delay. A serious incident is one that causes: death, serious injury to health, significant damage to property, significant disruption to basic services, or a serious infringement of fundamental rights.
Timeline for high-risk AI providers
| Date | Milestone |
|---|---|
| 1 August 2024 | AI Act enters into force; prohibited practices apply |
| 2 August 2025 | GPAI model obligations; governance rules; AI Office operational |
| 2 August 2026 | High-risk AI obligations (Annex III) fully apply — providers must comply before placing on market |
| 2 August 2027 | High-risk AI embedded in products covered by Annex I (machinery, medical devices, etc.) — obligations apply |
| Ongoing | Post-market monitoring, incident reporting, documentation updates required throughout product lifecycle |
Use the EU AI Act Compliance Checklist Generator to score your compliance posture against all obligations mapped to your role and risk tier. The AI Risk Register Generator provides the risk documentation required by Art. 9. For GDPR intersection, the AI Privacy Impact Assessment Generator covers Art. 35 DPIA requirements for AI systems processing personal data.