What compliance frameworks actually require for penetration testing
"Do we need a pen test for our audit?" is one of the most common questions SaaS founders ask. The answer depends on which framework, but the short version is: yes, if you're pursuing SOC 2 Type II, ISO 27001, PCI DSS, or DORA compliance — and you should be doing one anyway as a security hygiene minimum.
Here's what each framework actually requires:
| Framework | Reference | Requirement | Frequency | Who Conducts |
|---|---|---|---|---|
| SOC 2 | CC4.1 / CC7.2 | Risk assessment includes evaluation of threats; monitoring activities should include penetration testing | Annual (strongly expected by auditors; not explicitly mandated) | Internal or qualified third party; third party preferred for Type II |
| ISO 27001:2022 | A.8.8 / A.8.29 | Technical vulnerability management; security testing in development and acceptance including penetration testing | Annual (minimum); after significant changes | Internal or external; external preferred for Annex A A.8.8 |
| PCI DSS v4.0 | Req 11.4 | External and internal penetration testing at least once a year and after significant infrastructure or application upgrades | Annual (mandatory); after significant change | Qualified security assessor (QSA-accredited tester) for external test |
| DORA | Art. 24–25 | Digital operational resilience testing includes penetration testing; significant entities must conduct TLPT every 3 years | Annual (basic); every 3 years for TLPT (significant entities) | Qualified external tester; TIBER-EU certified for TLPT |
| NIS2 | Art. 21(2)(f) | Policies and procedures to assess effectiveness of cybersecurity risk management — penetration testing is a primary assessment method | Annual (expected) | Qualified internal or external |
| HIPAA | §164.308(a)(8) | Evaluation standard — periodic technical and non-technical evaluation of security controls; penetration testing is a primary tool | Annual; after major system changes | Internal or external; external for comprehensive evaluation |
SOC 2 and penetration testing: what auditors actually expect
SOC 2 does not use the word "penetration test" explicitly in the Trust Service Criteria. But auditors universally expect to see evidence of penetration testing for CC4.1 (monitoring activities) and CC7.2 (detection and monitoring of threats).
What SOC 2 auditors look for:
- A qualified third party conducted the test (in-house doesn't satisfy most SOC 2 auditors for the external attack perspective)
- Scope included external attack surface (public-facing applications, APIs, infrastructure)
- Test was conducted during the audit period (Type II) or recently before the report date (Type I)
- Findings were reviewed and remediated (or risk-accepted with justification)
- Evidence: pen test report (redacted OK) + evidence of remediation tracking
If you don't have a pen test: auditors will flag it as a gap, it will appear in the gap report, and it will make Type II harder. Some auditors will qualify the report. Budget for an external pen test before starting your SOC 2 audit preparation — it typically takes 4–8 weeks to procure, conduct, remediate, and document.
Penetration test scoping for SaaS
Scope determines cost, duration, and the value of the test. Too narrow and you miss critical attack vectors; too broad for your budget and you get shallow coverage.
Recommended scope for a standard SaaS compliance pen test:
| Scope Element | Include | Why |
|---|---|---|
| External web application | ✅ Yes — all production web apps | Highest-risk attack surface; tested by every compliance framework |
| APIs (REST/GraphQL) | ✅ Yes — all authenticated and unauthenticated API endpoints | Frequently where authorisation bypasses are found; OWASP API Security Top 10 |
| Authentication and session management | ✅ Yes | Business logic testing; account takeover paths; MFA bypass |
| Cloud infrastructure (AWS/GCP/Azure) | ✅ Yes — at minimum IAM review and public exposure audit | Misconfigured S3 buckets, public AMIs, over-permissive IAM are common critical findings |
| Internal network / VPN | ⚠️ Optional — if you have an office network | Relevant for ISO 27001; less so for pure-remote teams |
| Mobile applications | ⚠️ Optional — if you have a mobile app | Required for PCI DSS if mobile app handles CHD; ISO 27001 A.8.1 |
| Social engineering (phishing) | ⚠️ Optional — aligned with ISO 27001 A.6.3 | Tests human controls; provides security awareness training evidence |
| Physical security | ❌ Usually exclude — cloud-native SaaS | Not relevant unless you have on-premises infrastructure or offices with sensitive data |
Types of penetration testing
Not all pen tests are the same. The main types for SaaS compliance:
- Black box: Tester has no prior knowledge of the system — simulates an external attacker. Most realistic external threat model. Most time-consuming, so often least coverage for a given budget.
- Grey box: Tester has limited information (user credentials, architecture overview, API documentation). Most common for SaaS compliance testing. Balances realism with efficient coverage.
- White box: Full access to source code, architecture diagrams, network diagrams. Most thorough — auditors like this for ISO 27001 and HIPAA. Combined with static code review for maximum coverage.
- Red team: Longer-duration simulation of persistent threat actor; includes social engineering and physical elements. Required for DORA TLPT; generally overkill for early-stage SaaS compliance.
Recommended for most SaaS companies: grey-box external web application + API penetration test + cloud configuration review. This covers the primary attack surface efficiently and satisfies SOC 2, ISO 27001, and HIPAA requirements in a single engagement.
Penetration testing vs vulnerability scanning
These are not the same, and many companies conflate them. Auditors know the difference.
| Activity | What It Is | What It Finds | Frequency |
|---|---|---|---|
| Vulnerability scanning | Automated tool runs against systems to identify known CVEs and misconfigurations | Known CVEs in software/OS, common misconfigurations, missing patches | Weekly/monthly automated; quarterly authenticated scan |
| Penetration testing | Human expert attempts to exploit vulnerabilities using manual techniques and tooling | Business logic flaws, chained vulnerabilities, application-specific issues, authorisation bypasses — issues automated scanners miss | Annual (minimum); after significant changes |
| Red team assessment | Adversarial simulation of a persistent, targeted threat actor over weeks/months | Full kill-chain attack paths; social engineering weaknesses; detection gaps | Every 2-3 years; DORA TLPT requirement for significant entities |
For SOC 2 and ISO 27001, you need both: continuous/quarterly vulnerability scanning as an operational control, AND annual penetration testing as an assessment activity. They serve different purposes and provide different evidence.
Selecting a penetration testing provider
Key selection criteria:
- Certifications: CREST accreditation (UK/international standard); OSCP-certified testers; OSCE for advanced engagements; TIBER-EU qualification for DORA TLPT
- Insurance: Professional indemnity insurance; cyber liability — ask for certificates
- Report quality: Ask for a sample report — you want: executive summary (business impact), technical findings with CVSS scores, proof-of-concept evidence, reproduction steps, remediation recommendations, remediation verification re-test
- Scoping calls: A reputable tester will conduct a detailed scoping call before quoting — this is how they price accurately and you should be wary of providers who quote without scoping
- Rules of engagement: Written rules of engagement and authorisation letter before testing begins — this protects both parties legally
- Data handling: How are screenshots, credentials, and sensitive data from the test handled? What's their data retention and destruction policy?
Budget: for a standard SaaS grey-box web application + API pen test, expect £4,000–£12,000 / $5,000–$15,000 depending on scope and provider. Cloud infrastructure reviews add another £2,000–£5,000. Annual budget of £8,000–£15,000 is realistic for most Series A SaaS companies.
Managing pen test findings
A pen test report without remediation tracking is not a compliance artefact — it's a liability. Auditors don't want to see the full report (which may contain sensitive exploit details); they want to see your remediation process.
Remediation process that satisfies auditors:
- Import all findings into issue tracker (Jira, Linear, GitHub Issues) with severity labels
- Assign owners to each finding
- Remediation SLAs: Critical (24h), High (7 days), Medium (30 days), Low (next sprint)
- Re-test critical and high findings with the pen testing provider to confirm remediation
- Risk acceptance required for any findings not remediated within SLA — document justification and approver
- Final report summary showing: [X] Critical — 0 open, [X] High — 0 open (or risk accepted), [X] Medium — [X] remediated, [X] open
Evidence for auditors: pen test attestation letter or executive summary (redacted); remediation tracking in issue tracker; re-test confirmation; risk acceptance decisions for outstanding items.
DORA Threat-Led Penetration Testing (TLPT)
DORA Art. 25 requires significant financial entities to conduct Threat-Led Penetration Testing (TLPT) at least every three years. TLPT differs from standard pen testing:
- Based on real threat intelligence gathered about your specific organisation's adversaries
- Follows the TIBER-EU framework (or equivalent national framework, e.g. TIBER-NL, iCAST)
- Scope covers all critical and important functions
- Must be conducted by TIBER-EU qualified testers — not just any CREST-accredited firm
- Results and remediation plans shared with the competent authority
- Pooled TLPT: multiple financial entities using the same ICT provider can conduct TLPT jointly, with the provider's participation
For most SaaS companies: TLPT is only required if you are a financial entity or a CTPP designated by the ESAs. Standard annual penetration testing satisfies DORA's basic digital operational resilience testing requirement (Art. 24).
Minimum viable pen testing programme for SaaS
- ☐ Annual external web application + API penetration test (grey box, qualified third party)
- ☐ Cloud infrastructure security review included in scope (or separate engagement)
- ☐ Written rules of engagement and authorisation letter before testing
- ☐ All findings tracked in issue tracker with owners and remediation SLAs
- ☐ Critical and high findings re-tested by provider after remediation
- ☐ Risk acceptance documented for any findings not remediated within SLA
- ☐ Pen test executive summary / attestation retained as audit evidence (minimum 3 years)
- ☐ Vulnerability scanning (automated) running continuously or monthly — separate from pen test
- ☐ Pen test scope reviewed annually — includes any significant new features or architectural changes
Related generators
Related generators: Information Security Policy, Vulnerability Management Policy, Secure SDLC Policy, SOC 2 Gap Assessment, ISO 27001 Gap Assessment, DORA ICT Risk Management Policy.
Related reading: SOC 2 Evidence Collection Guide, Vulnerability Management Policy Guide, Secure SDLC Policy Guide, DORA ICT Risk Management Policy Guide.
⚠️ This guide is for informational purposes only and does not constitute legal or security advice. Penetration testing engagements should be conducted under written rules of engagement by qualified professionals. Compliance requirements vary by framework and assessor interpretation.