“The team successfully completed the exercise” says little about whether escalation was timely, authority was clear or recovery decisions matched the organisation's priorities.
Begin with an explicit expectation
An expectation may come from a plan, policy, regulatory obligation, recovery objective or approved role description. It should be specific enough to observe. “Escalate serious incidents promptly” is weak; “activate the crisis team within fifteen minutes of a material-impact determination” is testable.
Capture observed behaviour
Record what participants did, who owned it, when it happened and what information supported the decision. Observation should include delays, hand-offs, competing interpretations and unresolved questions—not only completed actions.
Describe the gap without overclaiming
If expected escalation was fifteen minutes and observed escalation was thirty-four, the finding is not automatically “failure.” The useful work is identifying why: unclear authority, unavailable contacts, uncertainty about materiality or competing operational priorities.
Measure dimensions that drive improvement
- Decision and escalation timing
- Ownership and authority
- Cross-functional coordination
- Regulatory and communications workflows
- Recovery prioritisation
- Evidence handling and uncertainty
- Plan adherence and justified deviation
NIST notes that qualitative and quantitative information can help determine the effectiveness of incident-response processes. Its current control guidance recognises walkthroughs, tabletop exercises and simulations as forms of incident-response testing. See NIST SP 800-171 Rev. 3.
Connect findings to sources
A source-linked finding is easier to review and act on. Show the expected process, the observed event and the practical consequence. Distinguish confirmed evidence from facilitator inference.
Retest the behaviour
Closure should require more than updating a document. If authority was unclear, the organisation should rehearse the revised authority under pressure. Improvement becomes credible when a later simulation shows that the behaviour changed.
COMMON QUESTIONS
Frequently asked questions
What are useful cyber exercise metrics?
Useful measures include decision timing, escalation, ownership clarity, coordination, recovery progress, communications quality and adherence to approved obligations.
Should an exercise produce a single score?
A score can simplify reporting but may hide uncertainty and context. Evidence linked to specific expectations usually supports better remediation.
How do you avoid subjective findings?
Define expected behaviours and evidence criteria before the exercise, capture decisions during the event and link findings to approved sources where possible.