Measuring Security Control Effectiveness

The compliance loophole is closing

For decades, compliance meant documenting that you did something. Regulations required action — not proof that the action worked. That era is ending.

Three EU frameworks — GDPR, NIS2 and DORA — now require organisations to demonstrate that their security controls produce measurable effects. The shift is subtle in regulatory language but fundamental in practice: having controls is no longer sufficient. You must be able to show they work. For compliance officers and security leaders, the question an auditor asks is changing from “do you have a control?” to “can you show what effect it has?”

Being effective versus having an effect

There is a meaningful difference between being effective and having an effect. “Effective” implies a positive outcome. “Having an effect” simply means something measurably changed — and that change could be positive or negative.

This distinction matters. An organisation that can document what changed after implementing a control — even when the change was not what it expected — is in a far stronger position than one that can only show the control exists. Documenting that something moved in the wrong direction is still evidence: it shows you measured, you noticed, and you can make an informed decision about what to do next. That is the governance posture auditors want to see.

Where the requirement appears

The whitepaper maps the effectiveness requirement across all three frameworks:

• GDPR — Article 32(1)(d) requires “a process for regularly testing, assessing and evaluating the effectiveness of technical and organisational measures.” Enforcement is now citing it by name.
• NIS2 — Article 21(2)(f) requires policies and procedures to assess the effectiveness of risk-management measures, while Article 20 introduces personal liability for management bodies.
• DORA — Article 13(4) requires financial entities to monitor the effectiveness of their resilience strategy on an ongoing basis, with no size exemption.

This is not a European phenomenon only. Equivalent provisions exist in the US FTC Safeguards Rule, Australia’s APRA CPS 234, Canada’s OSFI Guideline B-13 and the UK’s post-Brexit GDPR. Each jurisdiction arrived at the same conclusion independently: attestation is not evidence.

Why continuous evidence beats a point-in-time audit

Weak evidence is a point-in-time audit — it captures what existed at a single moment and cannot tell you whether a control implemented six months ago actually changed anything. Strong evidence is continuous monitoring of signals over time, connected to specific interventions: a control is implemented, a training is run, a policy changes — what did the relevant indicators do before, during and after? That before-during-after comparison, tied to a dated event, is what makes evidence defensible.

The organisational control layer — the measures meant to shape human behaviour — is where measurement almost universally fails. Most organisations still document that policies and training exist without tracking whether any of it changes behaviour. In 2018, only 4.7% of organisations had reached the highest measurement-maturity stage of the SANS Security Awareness Maturity Model; by 2025 that figure had risen to just 11.8% (SANS Security Awareness Report, 2018 and 2025). Nearly nine in ten organisations still produce activity evidence, not effectiveness evidence.

Read the whitepaper

Measuring Security Control Effectiveness: From Attestation to Evidence examines what the effectiveness requirement means, where it appears across GDPR, NIS2 and DORA, and what kind of evidence satisfies it — including the growing enforcement record and why the organisational layer is now measurable through Microsoft 365 behavioural signals. Download the full paper below.