Why training completion rates are not a security metric

Ninety per cent training completion. That number appears in board reports, compliance dashboards, and annual security reviews across thousands of organisations. It answers the question “did employees do the thing?” It does not answer the question “did anything change?”

The distinction between those two questions is the difference between process evidence and effectiveness evidence. Under current EU regulation, only the second one counts.

The measurement gap

Training completion rates document that an activity occurred. A module was assigned, employees opened it, they clicked through the slides or answered the quiz questions, and the system recorded their completion. That is what the number tells you.

It does not tell you what the employee learned. It does not tell you whether their behaviour changed — whether they handle suspicious emails differently, whether they lock their screens, whether they stopped sharing credentials. It does not tell you whether the organisation is now more secure or less secure than it was before the training ran.

A completion rate is evidence of process. It is not evidence of effect. Most organisations treat these as the same thing. They are not.

What the research says

The peer-reviewed evidence on whether security awareness training produces lasting behavioural change is not encouraging.

A study of more than 14,000 employees over 15 months (Lain, Kostiainen and Capkun, published at IEEE S&P 2022) found that click-time training — the embedded training that activates when an employee clicks a simulated phishing link — did not make employees more resilient to phishing. The study also observed side effects that may actually increase susceptibility. The researchers did not find that training made things slightly better. They found it did not work, and may have made things worse.

A separate study of 5,000 employees at an Israeli financial institution (Hillman, Harel and Toch, published in Computers & Security, 2023) examined whether the timing of training relative to a phishing simulation affected click rates. It did not. Training timing had no significant effect on whether employees clicked.

More recent research (Hydari et al., arXiv 2026, currently under peer review) goes further. It finds that observed improvement in phishing simulation click rates over time is largely explained by stable individual traits — characteristics of who someone is — rather than by the training itself. The people who were going to click kept clicking. The people who were not going to click did not need the training.

These are peer-reviewed studies conducted in real enterprise environments with large sample sizes. They are not vendor benchmarks or marketing claims. They represent the current state of the evidence on whether the most common form of security awareness training produces measurable behavioural change.

What this means for compliance

The three major EU regulatory frameworks — GDPR Article 32(1)(d), NIS2 Article 21(2)(f), and DORA Article 13 — all require evidence of effectiveness. Not evidence that a process ran. Evidence that the process produced a measurable effect.

Training completion does not satisfy that requirement. It is a record that an activity occurred. It cannot answer the question an auditor is increasingly likely to ask: “Can you show that your training programme had an effect on employee behaviour?”

That question cannot be answered with a completion report. It requires before-and-after behavioural data — evidence that something measurable changed as a result of the training, and that the change persisted. A completion report tells you the training was delivered. It says nothing about what the delivery accomplished.

The maturity gap

The SANS Security Awareness Maturity Model provides useful context here. In the 2018 SANS Security Awareness Report (n=1,718 respondents across 65 countries), only 4.7 per cent of organisations had reached Stage 5 — the “Robust Metrics Framework” stage, where organisations use data-driven metrics to measure and improve their awareness programme.

By the 2025 report (n=2,763, 70+ countries), that figure had grown to 11.8 per cent, now labelled “Optimization and Resilience.” Progress, certainly. But the majority of organisations — roughly nine in ten — remain below the level where their awareness programme is driven by measurement rather than activity.

This is the gap that matters. Most organisations can demonstrate they ran a programme. A small minority can demonstrate the programme worked.

The distinction regulators are drawing

Completion rates tell you something. They tell you whether your process ran. That is worth knowing for operational purposes — it confirms that the training was delivered, that the LMS functioned, that employees engaged with the assigned material.

But the question regulators are now asking is different. It is not “did the process run?” It is “did the process produce an effect?” Those are two different questions, and only one of them has a compliance answer.

An organisation that reports 90 per cent completion to its board has answered the first question. An organisation that can show what employee behaviour looked like before the training, what it looked like after, and whether the change held over time has answered the second. The regulatory direction is clear about which answer matters.

For the full regulatory argument and enforcement precedents, read the white paper: Measuring Security Control Effectiveness: From Attestation to Evidence