What is an AI Privacy Impact Assessment?
An AI Privacy Impact Assessment (AI-PIA) — also called an AI DPIA — is a Data Protection Impact Assessment conducted specifically for AI systems. It combines the GDPR Article 35 DPIA requirements with the additional AI-specific risk factors introduced by the EU AI Act: algorithmic bias, opacity, automated decision-making, and data governance for training datasets.
A general DPIA asks: "Does this processing create high privacy risks?" An AI-PIA asks the same question, but the answer is almost always yes — and the specific risks are different from traditional data processing.
When is a DPIA Mandatory for Your AI System?
GDPR Article 35(1) requires a DPIA when processing is "likely to result in a high risk" to individuals. For AI systems, this threshold is met in most cases. The EDPB Guidelines 09/2022 identify the following high-risk criteria, and AI systems commonly meet multiple:
| EDPB High-Risk Criterion | How AI Systems Typically Meet It |
|---|---|
| Evaluation or scoring | Almost all AI systems evaluate, classify, or rank individuals |
| Automated decision-making with legal/similar effects | Credit, hiring, insurance, pricing, moderation decisions |
| Systematic monitoring | AI that continuously monitors user behaviour or employees |
| Sensitive data or data of vulnerable nature | Health AI, mental health apps, HR analytics with inferred attributes |
| Large-scale processing | AI processing at the scale of a SaaS product with thousands of users |
| Innovative use of technology | New technology = always meets this criterion |
If your AI system meets 2 or more of these criteria, a DPIA is mandatory. In practice, most SaaS AI features involving personal data meet at least 3. When in doubt, conduct the DPIA — the cost of doing it unnecessarily is low; the cost of not doing it when required (GDPR Art. 83(4): up to €10M or 2% global turnover) is high.
The EU AI Act Layer: What's Different in 2026
The EU AI Act does not replace the GDPR DPIA requirement — it adds obligations on top. The intersection matters:
| Obligation | Source | What It Requires |
|---|---|---|
| DPIA | GDPR Art. 35 | Privacy risk assessment before deployment |
| Fundamental Rights Impact Assessment | EU AI Act Art. 26(9) | Deployers of high-risk systems (public sector / certain deployers) |
| Risk Management System | EU AI Act Art. 9 | Continuous lifecycle risk management for high-risk AI |
| Technical Documentation | EU AI Act Art. 11 + Annex IV | Detailed technical docs for high-risk systems |
| Data Governance | EU AI Act Art. 10 | Training/validation/test data quality and bias evaluation |
| Transparency Notice | EU AI Act Art. 50 + GDPR Art. 13/14 | Disclose AI use, chatbot identification, content marking |
For most SaaS founders using AI APIs (OpenAI, Anthropic, Google Gemini), you are a deployer under the EU AI Act (Art. 3(4)) — you use a third-party AI system rather than developing your own. Your DPIA should reflect this: you cannot fully assess the model's training data, but you can document your data governance for inference inputs and outputs.
AI-Specific Risks That a Standard DPIA Misses
A standard DPIA checklist was designed for traditional data processing (databases, forms, analytics). AI systems introduce qualitatively different risks:
1. Algorithmic Bias
AI systems can encode, amplify, or introduce discriminatory outcomes against protected characteristics (gender, race, age, disability) even when those attributes are not explicit inputs. An AI trained on historical data inherits historical biases. Your AI-PIA must assess this — which demographic groups are at risk, how you evaluate fairness, and what you do when bias is detected.
Under GDPR, algorithmic discrimination engages Article 21 (right to object) and Article 22 (automated decisions). Under the EU AI Act, Article 9(7) requires bias testing as part of the risk management system for high-risk AI.
2. Opacity and Explainability
LLMs and deep learning models are inherently opaque. When an AI system affects an individual — denies credit, rejects a job application, flags content for removal — GDPR Article 22(3) requires that the individual be able to obtain "meaningful information about the logic involved, as well as the significance and the envisaged consequences."
Your AI-PIA should document what explainability you can provide and, critically, how you implement the Art. 22(3) right to human review and contest.
3. Training Data Privacy
If the AI model was trained on personal data — including data scraped from the internet — there are potential GDPR violations in the training pipeline. Even if you use a third-party model (OpenAI, etc.) and don't control training, your DPA with the model provider should address whether your inference inputs are used to train future models.
Critically: a user submitting a support ticket or a document to your AI feature may not realise that content could be used to improve the model. If it is, this requires a lawful basis and disclosure in your privacy notice.
4. Inference Attacks and Model Inversion
AI models can memorise and reproduce training data. An adversary may be able to extract personal data from a model through carefully crafted queries (model inversion, membership inference). Your AI-PIA should address this risk, particularly if the model was trained on customer data.
5. Prompt Injection
For AI features that process user-supplied inputs, prompt injection attacks can manipulate the model to expose other users' data, generate harmful content, or bypass safety controls. This is a data security risk that intersects with GDPR Article 32 security obligations.
Automated Decision-Making Under GDPR Article 22
Article 22 is the most consequential GDPR provision for AI systems. It prohibits decisions "based solely on automated processing" that produce "legal or similarly significant effects" on individuals — unless one of three exceptions applies:
- Necessary for entering into or performing a contract (Art. 22(2)(a))
- Authorised by EU or Member State law (Art. 22(2)(b))
- Based on explicit consent (Art. 22(2)(c))
"Similarly significant effects" is broader than it sounds. EDPB guidelines include: automatic refusal of an online credit application; e-recruitment practices without human intervention; insurance pricing; creditworthiness assessment. For most SaaS AI features that make recommendations rather than final decisions, Art. 22 doesn't apply — but if the recommendation is followed automatically without human review, that changes the analysis.
The 9 Steps of an AI-PIA
- Identify the need — Does GDPR Art. 35 apply? (Apply the EDPB 9-criteria test above)
- Describe the AI system — What it does, how it works, training approach, deployment context
- Classify under the EU AI Act — Provider or deployer? Risk category? Annex III use case?
- Map data flows — Input data → model → output → downstream use. Who are the processors?
- Assess necessity and proportionality — Is AI necessary? Less intrusive alternatives? Minimum data processed?
- Assess AI-specific risks — Bias, opacity, training data, inference attacks, prompt injection, human oversight gaps
- Identify and implement safeguards — Technical, organisational, and contractual measures
- Implement data subject rights — How to handle access, erasure, Art. 22 objection, explanation requests for AI
- Consult DPA if high residual risk remains — GDPR Art. 36 prior consultation
Special Category Data and AI
AI systems frequently process or infer special category data under GDPR Article 9, even when that's not the intent:
- NLP models analysing customer sentiment may infer emotional or mental health states
- HR analytics platforms may infer race, disability, or trade union membership from employment patterns
- Computer vision systems may process biometric data (facial recognition falls under Art. 9)
- AI trained on general internet data has likely ingested health information, political opinions, and sexual orientation data
Special category processing requires both a GDPR Art. 6 lawful basis AND an Art. 9(2) condition (typically explicit consent or substantial public interest). A DPIA is mandatory when processing special category data at large scale.
DPA Prior Consultation (GDPR Art. 36)
If your AI-PIA concludes that high residual risk remains even after implementing safeguards, GDPR Article 36 requires you to consult your supervisory authority before proceeding. "High residual risk" means risks that cannot be sufficiently mitigated. For high-risk AI systems under the EU AI Act (Annex III), this threshold is often met.
The consultation process: submit your DPIA to the DPA, who has 8 weeks to provide written advice (extendable by 6 weeks). Processing may not begin until advice is received or the consultation period expires.
Document Control and Review
An AI-PIA is not a one-time document. You must review it when:
- The AI model is updated (new version, fine-tuning, different base model)
- New personal data categories are added to inference inputs
- The deployment context changes (new use cases, new user segments)
- A new regulation comes into force that affects the system
- An incident occurs that reveals new risks
- Bias evaluation reveals discriminatory outcomes
Tools & Further Reading
Use the ComplyKit AI Privacy Impact Assessment Generator to produce a complete GDPR Art. 35 + EU AI Act aligned document in minutes. Also relevant:
- General DPIA Template Generator — for non-AI high-risk processing
- EU AI Act Transparency Declaration Generator — Art. 50 user-facing notice
- AI/ML Model Card Generator — technical documentation for AI systems
- Privacy Policy Generator — must disclose AI use under Art. 13/14
- EU AI Act for SaaS Founders — full framework overview
- DPIA Guide for SaaS Founders — general DPIA approach
⚠️ This article is for informational purposes only and does not constitute legal advice. GDPR Article 35 DPIA requirements and EU AI Act obligations vary by system type, deployment context, and jurisdiction. Consult your DPO or qualified legal counsel before deploying AI systems that process personal data at scale.