Improve: From Finding to Change in Detections, Controls, Playbooks and Mandates

This chapter treats P4 Improve, the stage of the purple loop in which a finding becomes a change. P3 Observe (Chapter 9) ends with a finding: a statement that a layer of the defence did not do what it was expected to do against a catalogued scenario, in a stated condition, on a stated date. P4 Improve decides what kind of change the finding calls for, who owns it, who reviews it, what it costs and how long it may take, and it ends by handing a regression case to P5 Verify (Chapter 11). The chapter concerns P4 Improve above all, and it touches P3 Observe for what it receives and P5 Verify for what it hands on.

The stage is where most loops lose their value quietly. Exercises find things, and a year later the same techniques are found again, because the finding went into a backlog that no one owned. AI sharpens the problem from both ends. Continuous emulation (Chapter 8) raises the rate at which findings arrive, and AI drafting of detections and fixes raises the rate at which changes can be proposed. Neither raises the rate at which a person can review a change for soundness. The chapter argues that P4 Improve is bounded by review, not by drafting, and that the design task is to make review real: a named owner, a decision a second person can reproduce, a counted cost and a visible age.

A finding is routed by the layer that failed to a change (detection, control, playbook or mandate) or to a recorded risk acceptance, reviewed by a named person, and a regression case goes to P5 Verify.

Figure 10.1: A finding routed to a change or a recorded risk acceptance, reviewed by a named person, with a regression case for P5 Verify.

The chapter builds on four conclusions of the earlier volumes. A finding ends as a verified control, under the closure rule that Chapter 4 adopts and Chapter 11 operates (AI Red Team, Chapter 26; AI Blue, Chapter 11) (90, Nordqvist, 2026), (94, Nordqvist, 2026). A detection is a record with logic, data sources, ATT&CK mapping, tests, owner, alert budget and review date, and it counts only when a recent test shows that it fires (AI Blue, Chapter 6). Autonomy in response is matched to reversibility, and the lowest ceiling governs (AI Blue, Chapters 10 and 13). And verifying what AI proposes, when the reviewer depends on the AI, needs soundness defined in advance, review that does not start from the AI's answer and independent reproduction (The Transformed Security Architect, Chapter 14) (92, Nordqvist, 2026).

What P4 Improve receives and hands on

A finding arrives from P3 Observe as a record, FND-*, with a scenario (SCN-*), a technique, the date and the environment of the run, the outcome observed and the layer at which the defence failed to meet the expectation. P4 Improve adds a decision to that record and does not rewrite it: the route chosen, the owner, the change itself (a pull request, a contract change request or a mandate revision), the reviewer, the expected cost and an expiry. It hands on the regression case that P5 Verify will run, so that the stage ends not with a merged change but with a change and the test that will show whether it works.

Three rules follow from the closure rule of Chapter 11. A merged change is not a closed finding: the coverage record shows the technique as implemented but unverified and the register shows the finding as open until P5 Verify reports. The regression case is written when the change is decided, because a test written after the fix tends to describe what the fix does. And a finding that will not be fixed is also a decision with an owner, a reason and an expiry (the last section), not a quiet absence.

The stage has a legal and supervisory anchor for financial entities. DORA requires financial entities to establish procedures to prioritise, classify and remedy the issues revealed by their tests, and internal validation methods to ascertain that identified weaknesses are fully addressed (DORA Article 24(5)) (2, European Union, 2022). The architectural implication is the one this chapter draws for every organisation: remediation and the validation of remediation are two steps, and an auditor can be expected to ask for both (an architectural implication, not a prediction of supervisory practice). The TIBER-EU materials likewise treat a remediation plan and a test summary report as parts of a test, not as afterthoughts (20, ECB, 2025). NIS 2 Article 21 requires measures to assess the effectiveness of cybersecurity risk-management measures, and Article 20 places approval and oversight of those measures on the management body (3, European Union, 2022). Whichever instrument applies, responsibility for a finding that is not fixed stays with people.

The finding record and the change record

Two records carry the work, and this section defines them once; Chapter 9 shows a finding as P3 raises it, and Chapters 12 and 13 extend both for AI systems and for the defenders' own AI without replacing a key. P4 never rewrites the finding. It adds a change record, CHG-*, so that the observation and the decision about it remain separately reviewable.

Record Keys Written by
Finding FND-* kind: finding, id, scenario, technique, step, session, opened, expected, outcome (one of the seven of Chapter 9), layer_failed, evidence, times, proposed_owner, proposed_kind, status; optional handling_class (Chapter 16), children and change (the CHG-* identifier) P3 Observe; status moves with the change and the re-test
Change CHG-* kind: change, id, finding, route, decision (the AIDR record, routes considered, urgency, target), owner, artefact, drafted_by, scenario_author, reviewer, critique, cost, rollout, merged, handed_to_p5 (regression case, variants, cadence), review_date, status P4 Improve

The finding's status takes the five values of Chapter 4: open, implemented-unverified, re-test-failed, closed-verified and risk-accepted. The change's own status is narrower: proposed, merged, rolled-back or declined. A risk acceptance is a change record with the route risk-acceptance, no artefact and an expiry. A change record takes the number of its finding (CHG-HM-2028-007 for FND-HM-2028-007); a change raised outside any finding keeps the number of the change system, as CHG-HM-5521 does in Chapter 15.

Choosing the kind of change

A finding does not name its own fix. The first decision in P4 Improve is the layer at which the change belongs, and it is made from the layer at which the defence failed, not from the artefact that is easiest to edit. The series' architecture gives four routes and a fifth that is a decision rather than a change.

Route Use it when the finding shows Artefact changed Reference
Detection The activity was visible in telemetry and no detection fired, fired late, or fired and was routed wrongly Detection record DET-*, its logic and tests AI Blue, Chapter 6
Control The activity was possible because a preventive or architectural control was absent, misconfigured or bypassable Control catalogue entry, policy as code, configuration AI Red Team, Chapters 14 to 17
Playbook The activity was detected and the response was slow, wrong, too wide or too narrow Playbook manifest PBK-*, runbook, decision right AI Blue, Chapter 10
Mandate An agent, a human role or a provider acted outside, or short of, what it was authorised to do Mandate manifest MND-*, delegation profile DLG-*, contract annex AI Blue, Chapter 13
Recorded risk acceptance The change is not worth its cost, cannot be made now, or cannot be made at all A decision record with owner and expiry Last section of this chapter

Two cases need care. The first is the telemetry gap: if the log source was never collected, the cause is below the detection, the route is a change to collection, and the detection is held in shadow until the data arrives. The second is the finding that fits several routes. A credential added to a cloud identity outside a change window can be prevented (a control), detected (a detection) and contained (a playbook). The book's preference is to prevent where prevention is cheap and does not break service, to detect where it is not, and to keep a detection after a prevention is added, because a weakened control should leave a trace. Where a finding takes more than one route, it is split into one child finding per route, each with its own owner and re-test, so that no route hides behind another.

A route recorded without a reason cannot be reviewed, so the decision record states the layer that failed, the routes considered and why the others were rejected. A later reader can then see that a control was considered and set aside for cost, not forgotten, and the same record answers the supervisory question about how issues were prioritised, classified and remedied.

Ownership

A finding without an owner is an observation. The rules for ownership in this book follow the conclusion of AI Blue, Chapter 11, that one named role is accountable for the age of the register, and that the red team or the external tester is accountable for the quality of findings and not for their closure (94, Nordqvist, 2026). The purple coordinator (AI Blue, Chapter 17) schedules, routes and tracks; the owner of the artefact changes it.

Route Default owner Who must not own it
Detection The detection owner (ROLE-*, for Nordvik Region Karin Lund; for Hansa Mutual the detection-engineering lead) The tester who wrote the scenario
Control The owner of the control: platform, identity or network team The detection team, unless it owns the platform
Playbook The playbook owner named in the manifest The agent that ran it
Mandate The mandate's delegator, a named person The agent whose mandate it is
Risk acceptance A role with authority over the residual risk, not the owner of the work The person who would otherwise do the work

Two exclusions repeat across the table and carry the independence principle (Chapter 15) into ownership. The tester who authored a scenario does not tune the detection that answers it. The Region's call-off agreement already says the firm is not the Region's detection owner and does not tune the Region's rules, and the same rule applies to the Hansa red team and the engineers who write the detections for its scenarios. A tester who also fixes grades their own work.

Ownership extends to suppliers. Detection content that an organisation owns under its contract can be changed by the organisation; closure behaviour that a provider's AI applies cannot, and the route is then an evidence request and a test under the contract (the Nordvik example below).

The management body of an essential or important entity approves and oversees risk-management measures (NIS 2 Article 20), and the management body of a financial entity bears responsibility for ICT risk (DORA Article 5) (3, European Union, 2022), (2, European Union, 2022). An agent that drafts a change does not own it.

AI-drafted changes

AI can draft each kind of change. A drafting assistant can propose the logic of a detection from the behaviour description in a finding, a Sigma rule (30, SigmaHQ, n.d.) with positive and negative tests; it can propose a policy-as-code change for a control, a revision of a playbook step, or a mandate amendment that narrows an action class. These are tasks for Modes 1 to 3 under the engineer's identity, or for a Mode 4 or Mode 5 drafting agent at Level 1 Assisted or Level 2 Supervised. Drafting is practised with open rule formats and assistants today. The illustrative Hansa year of AI Blue, a constructed case offered for scale and not as evidence, had the drafting assistant produce 131 detections in a half-year, of which 52 were merged, under a cap of six open drafts per reviewer.

The risk is the one that AI Blue, Chapters 6 and 14, and The Transformed Security Architect, Chapter 14, describe from the other side. Drafts are fluent, and fluent drafts invite acceptance. The research on human use of automation documents complacency and automation bias, in which people who rely on an automated aid check it less (101, Parasuraman & Manzey, 2010), and Bainbridge's analysis points to the structure behind it: when automation takes the routine part of the work, the person is left with the part that is hardest to do well (100, Bainbridge, 1983). A loop that produces findings at machine speed and drafts changes at machine speed, reviewed by a fixed team, drives that structure towards its worst case.

Three further risks are specific to the loop. The first is overfitting. A detection drafted from the exact telemetry of the exercise matches that exercise. It may key on a feature of the scenario that a real adversary will not reproduce, and it will pass the re-test of the same scenario for that reason alone. The defences are to draft from the behaviour description at technique level and not from the emulation runner's script, to require a negative test and a replay over production telemetry, and to let P5 Verify run variants (Chapter 11), not the identical scenario.

The second is shared blind spots. If the model that proposed the scenario also drafts the detection, or if the model that drafts the fix also reviews it, both carry the same gaps. The independence principle (Chapter 15) applies here as it does elsewhere in the loop. The default is that the drafter and the reviewer differ in kind: the drafter is an assistant at Level 1 Assisted, the reviewer is a named person, and a critique agent, where one is used, runs on a different model family and is asked questions, not for a verdict. Nordvik Region applies the rule by role: the detection-drafting assistant is not used to write the detection for a scenario that it also helped to choose.

The third is the map. The draft, the finding and the scenario together describe a weakness, so they are held in the findings store with its access limits, not in a general-purpose assistant's history. For Nordvik, whose alert and finding data contains personal and health data, drafting stays on self-hosted capacity.

Observable, expected, speculation. The drafting of detections and of control and playbook changes by assistants, reviewed by a person, is observable practice in the tooling around open rule formats such as Sigma (30, SigmaHQ, n.d.); the same changes reviewed by a second AI under independence rules (different model family, no sight of the drafter's reasoning, seeded errors) are expected, on the trend of the review practice described in The Transformed Security Architect, Chapter 14. An agent that drafts, approves and deploys changes to production controls with no named reviewer is speculation, and the ceiling table at the end of this chapter exists to keep it out of the design.

Review

Review is the stage's bottleneck by design. A change to a production detection, control or playbook does not merge without a named reviewer who is not the drafter and who did not choose the scenario. The review has five questions, defined in advance so that they do not depend on the draft.

  1. Soundness. Does the change address the layer that failed? Does it fire on the behaviour of the finding and on variants of it, and stay silent on the negative case? The reviewer reproduces the key fact independently: they run the unit tests themselves, and replay the logic against the stored telemetry of the finding, before reading the drafter's explanation.
  2. Side effects. What else does the change touch? An exclusion added to quieten a noisy rule can silence a different detection. The suite of regression cases (REG-*) for every technique that shares the changed logic, data source or exclusion list is re-run, and a change that makes an earlier case fail is rejected.
  3. Cost. What alert volume does the change add, and does the rule's budget absorb it? The next section sets the numbers.
  4. Reversibility. Can the change be rolled back, how fast, and by whom? Detections ship through a shadow stage, a pilot and a full stage with a rollback rule. A control change that cannot be rolled back, or that touches production access, is a decision for the control owner, not a merge.
  5. Provenance. Which parts were drafted by an AI system, under which mandate, and who verified them? The record states it, in the form used for reviews in AI Blue, Chapter 11.

The measures that keep review real are those of AI Blue, Chapter 14: a cap on open proposals per reviewer (twelve, at Hansa), seeded errors and seeded incidents so that the catch rate is known, and an oversight-quality metric reported for the team and never for a named reviewer. Because they concern people, they are agreed with the employee representatives first: at Nordvik results are reported in aggregate and never by name; at Hansa the exercise clause KBV-IT-2024-SOC-05 applies and no exercise result enters an appraisal.

When the finding concerns an agent, the review has an asymmetry that speeds the safe direction. Lowering an agent's level, narrowing its mandate or adding a stop condition takes effect at once on the decision of the mandate's delegator, and is recorded; raising a level or widening a mandate is a decision by a named authority after a re-test. A finding that shows an agent acting beyond its mandate is therefore handled in two steps: an immediate reduction to Level 0 Advisory or Level 1 Assisted for the affected action class, and the proper change, which comes through review.

False-positive cost

Every new detection costs analyst attention for as long as it exists. A loop that turns every finding into a rule without counting the cost ends with an alert queue that no one trusts. The qualitative research on SOC analysts' views of alarms, and the survey of the literature on alert fatigue, give the reason to count: analysts experience a large share of alarms as false and describe the effects on how they work (96, Alahmadi et al., 2022), (98, Tariq et al., 2025). The evidence is largely qualitative and gives no number to import. The design rule is the book's own conclusion: the cost of a change is a number in the change record, reviewed with the change.

The detection record of AI Blue, Chapter 6, already carries the instrument, an alert budget per week and the observed alerts and precision over 30 days. P4 Improve adds three defaults.

  • Replay before merge. Run the changed logic over at least 30 days of production telemetry and record alerts per week. A change expected to exceed its budget at replay does not merge; it is tuned, or the budget is raised by a recorded decision that names who accepts the added load.
  • Shadow first. A new or changed detection runs in shadow for 14 days before it pages or opens a case. The shadow period is the first real measure of cost, and it is longer for rules that touch high-volume sources.
  • A tune-or-retire rule. If a rule's alerts per week exceed the budget for two consecutive weeks, it is reviewed; if they exceed twice the budget for two weeks, it rolls back. The thresholds are defaults that the SOC adapts to its capacity and writes down.

The total budget of the detections the loop adds in a quarter is visible to the SOC lead, who decides whether the team can carry it. The budget is not a reason to refuse a needed detection; it is a reason to retire an older rule with no owner or no source in the same change, which is what the detection debt register of AI Blue, Chapter 11, is for.

The backlog and the half-life

Findings accumulate faster than they are fixed unless something limits them. The backlog is the register of open findings with age, route, owner and expiry. Two measures describe it, and both are measures of the system and not of anyone.

The finding-to-control time is the time from a finding being opened to a control or detection being in place; AI Blue records a median of 19 days for Hansa Mutual (BTP-HM-2027-Q3, AIDR-0778). The finding half-life is the book's own measure: the number of days after which half of the findings opened in a quarter have a passing re-test. The first is easy to improve by merging faster. The second counts only verified closure, and the gap between them is the stock of changes implemented and not verified, which the register reports as open.

Prioritisation sets the order. Three questions, answered at the time of the decision and written into it, give a defensible order without false precision.

  1. Which technique and how important? A finding on a priority technique (Chapter 5) outranks one on a technique outside the list, and a priority technique whose evidence has expired (90 days for priority techniques at Hansa, 180 for others) counts as older than its last finding.
  2. What did the defence do? A technique that went undetected outranks one that was detected late, which outranks one detected and routed wrongly.
  3. What is at stake? The data, the service or the person affected, including the crown-jewel assets and the findings store itself.

The three answers sort findings into urgent (default target of 14 days to a change in production, and a re-test in the next cycle), standard (30 days) and deferred (90 days or the next planned release). These are defaults; each organisation adapts them to its contract, capacity and recorded risk appetite, and writes the adapted values down. Age is shown beside the target in the register, and a finding past twice its target is escalated to the role accountable for the register's age.

A tester's severity is an input and not a verdict; the record keeps the tester's rating, the owner's and the reason they differ.

AI helps with the sort and not with the decision. A Mode 3 or Mode 4 assistant at Level 1 Assisted can propose routes and order, group findings that share a root cause, and flag findings whose evidence is about to expire. A person confirms the route and the order. Grouping by root cause is the largest saving, because findings opened or closed are not a measure of learning (AI Blue, Chapter 11).

Findings no one can fix

Some findings cannot be fixed within the target, or at all. The legacy application cannot log the event, the vendor will not change the control, or the fix would break a service with no maintenance window this year. A loop with no place for these findings hides them or lets them age without a decision.

A finding that is not fixed has three options, each a decision with a record.

  • Compensating control. A different control or detection reduces the risk, and the compensating measure itself is tested by a scenario. A compensating control that has never been re-tested is declared, not verified.
  • Recorded risk acceptance. A role with authority over the residual risk accepts it, for a stated period, with a reason and a review date. The record is a versioned risk entry with likelihood, impact, owner and an expiring exception, as Architecture as Code (Chapter 9C) sets out for risk as code (87, Nordqvist, 2025). The default expiry is 90 days for a priority technique and 180 days otherwise; an exception that has no expiry is rejected by the pipeline.
  • Transfer or exit. The risk is moved by contract, or the system is retired or isolated. This too is recorded, and the supplier's consent is part of the record if the finding concerns their platform.

Three guards apply. The person who would do the work does not accept the risk, because the economy of effort pushes towards acceptance. The coverage record keeps showing the technique as failed or declared, with the acceptance reference beside it, so that a board reading the coverage page cannot mistake an accepted gap for a closed one (Chapter 17). And an accepted finding keeps its regression case: the scenario is re-run at its normal cadence, because the condition may change, and a compensating control may fail without anyone noticing.

Where the management body or its delegate accepts the risk, the record is the evidence that it approved and oversaw it (NIS 2 Article 20; DORA Article 5), which is the reason to keep it in the repository and not on a slide.

Practical guide: Writing a finding-to-change decision that a second person can reproduce

Purpose. Record, for one finding, the route, owner, change, review, cost, expiry and regression case in a form from which a second person, given the same finding, would reach the same decision or can say exactly where they differ. The standard is reproducibility, not agreement.

Roles. The finding owner (usually the purple team coordinator); the artefact owner who takes the change (detection owner, control owner, playbook owner or mandate delegator); the reviewer, who is not the drafter and not the scenario's author; the tester, who is consulted on the behaviour description and does not own the fix; and, where an AI system drafts, the person who runs it under a named mandate.

Steps.

  1. Restate the finding in behaviour terms. Write what the defence was expected to do, what it did, at which layer, with the technique identifier and the evidence reference. No scenario script, no tool strings.
  2. Name the layer that failed and list routes considered. For each route not chosen, give a one-line reason. If the telemetry was absent, the route is collection first.
  3. Choose the route and split by route. One child finding per route, each with an owner from the ownership table. Check that the owner is not the scenario's author.
  4. Set the urgency from the three questions (technique importance and evidence age, what the defence did, what is at stake), and write the target date from the default (14, 30 or 90 days), adapted and stated.
  5. State the expected cost. Alerts per week at replay, the budget, the shadow period, and the older rule retired to make room, if any.
  6. Name the reviewer and record provenance. Who drafted, with what mandate and at which level; who reviews; whether a critique agent was used and on which model family.
  7. Write the regression case now. Identify the REG-* case that P5 Verify will run, with at least one variant that does not reuse the scenario's own artefacts, and its cadence.
  8. Set the expiry. A review date for the change (default six months) and, for an accepted or compensated finding, an expiry (default 90 or 180 days).
  9. Test reproducibility. Hand steps 1 to 8 to a second person who has not seen the decision. If they choose a different route, either the record lacked a reason or the decision was not sound; resolve it before the change is built. For urgent findings, sample one in five; for others, one in ten.

Definition of done. The finding record holds the route, the owner, the reviewer, the cost, the regression case and the expiry; the change exists as a pull request, a change request or a mandate revision with a reference; the coverage record shows the technique as implemented but unverified until P5 Verify reports; and no field is blank. A blank is itself a finding.

Worked example: A finding and its change record at Hansa Mutual

Illustrative. On 2028-02-06 the continuous pipeline HM-SEC-004 ran the safe-class scenario SCN-HM-0031, which emulates the addition of an extra credential to a cloud service identity (ATT&CK T1098.001), against the group cloud estate. The scenario is S1: benign, reversible and tagged. The audit log carried the event; no detection fired. The finding was opened on 2028-02-09 and routed to a detection change by the detection-engineering lead. A preventive control, a policy that denies credential creation outside a change window, was considered and recorded as a separate child finding with the platform team, because it would block a legitimate rotation job until the job was changed. The detection was drafted by HM-SEC-002 at Level 1 Assisted from the behaviour description, not from the runner's script. Mareike Hartmann's red team wrote the scenario; the reviewer is the second detection engineer; the critique agent MND-HM-EMU-04, on a second model family, was asked for missing cases and gave no verdict.

# Illustrative. The finding, as P3 raised it (Chapter 9), with its status moved by P4.
kind: finding
id: FND-HM-2028-007
scenario: SCN-HM-0031            # S1, run by HM-SEC-004 on 2028-02-06
technique: T1098.001
step: 1
session: none                   # raised from a pipeline run, outside a purple session
opened: 2028-02-09
expected: "Alert within 15 minutes when a credential is added to a service identity outside a change window"
outcome: missed
layer_failed: detection          # telemetry present, no alert
evidence: {query: saved_query_H14, event: audit-log-0206-0417}
proposed_owner: detection-engineering-lead
proposed_kind: [detection, control]
children: [FND-HM-2028-008]      # control route, platform team
change: CHG-HM-2028-007
status: implemented-unverified   # until REG-HM-0031 passes with evidence
---
# Illustrative. The change record that P4 added.
kind: change
id: CHG-HM-2028-007
finding: FND-HM-2028-007
route: detection
decision:
  record: AIDR-0855
  routes_considered:
    - {route: detection, chosen: true}
    - {route: control, chosen: false, reason: "Deny rule would block rotation job ROT-12; split to child finding FND-HM-2028-008"}
  urgency: urgent                # priority technique, undetected, identity tier-0 adjacent
  target: 2028-02-23             # 14 days
owner: detection-engineering-lead
artefact: DET-HM-0612
version: 1.0.0
drafted_by: HM-SEC-002           # Level 1 Assisted, from behaviour description only
scenario_author: red-team-lead   # not the owner of the fix
reviewer: detection-engineer-2
critique: MND-HM-EMU-04          # second model family, questions only
cost:
  replay_days: 30
  alerts_per_week_at_replay: 3
  budget_per_week: 5
  shadow_days: 14
  retired_to_make_room: DET-HM-0201   # no owner, no live source
rollout: [shadow-14d, pilot-shift, all-shifts]
merged: 2028-02-20
handed_to_p5:
  regression_case: REG-HM-0031
  variants: [REG-HM-0031-A, REG-HM-0031-B]   # different identity type and path
  cadence_days: 30
coverage_cell: failed            # stays failed until REG-HM-0031 passes with evidence
review_date: 2028-08-20
status: merged

Three features matter. The finding's status is implemented-unverified and the coverage cell stays failed until P5 Verify has run the case and its variants and attached the evidence; the merge on 2028-02-20 is day 11 of a target of 14, which says nothing yet about closure. The scenario's author, the reviewer and the owner of the fix are different roles. And the cost is in the record: three alerts a week at replay against a budget of five, and an old rule retired in the same change. The group median of 19 days (BTP-HM-2027-Q3) is the number to compare with.

Worked example: A finding and its change record at Nordvik Region

Illustrative. The firm engaged under the call-off agreement NR-2026-071 ran the spring catalogue (EMU-NR-2028-01, Chapter 8) between 2028-03-20 and 2028-03-24. Scenario SCN-NR-0006 emulates the creation of a scheduled task for persistence (ATT&CK T1053.005) on a Region-owned test host that mirrors the server image of the benefits-triage service, with Elin Berglund's team in the lead. The endpoint console recorded the process event and the SIEM held it. No rule matched, as the Region had declared before the run, so the finding is that a known gap is still open. The finding came out of the purple session PRP-NR-2028-01 on 2028-03-28. The Region owns detection content under annex MS-1, so the primary change is the Region's own, a new rule. The rule protects nothing out of hours unless the provider's closure behaviour treats its alerts as not closable by AI, and that behaviour is the provider's, so the second change is a request under the contract.

# Illustrative. The finding, as P3 raised it (Chapter 9).
kind: finding
id: FND-NR-2028-003
scenario: SCN-NR-0006
technique: T1053.005
step: 1
session: PRP-NR-2028-01
opened: 2028-03-28
expected: "Alert, routed to a person within 15 minutes, when a scheduled task is created on a server in the benefits-triage segment outside a change window"
outcome: missed
layer_failed: detection           # telemetry present, no rule (gap declared before the run)
evidence: {query: saved_query_N3, telemetry: edr_process_event}
proposed_owner: ROLE-NR-SEC-DET
proposed_kind: [detection, mandate]
change: CHG-NR-2028-003
status: implemented-unverified
---
# Illustrative. The change record that P4 added.
kind: change
id: CHG-NR-2028-003
finding: FND-NR-2028-003
decision:
  record: AIDR-0805
  urgency: urgent
routes:
  - route: detection            # Region-owned content under annex MS-1
    owner: senior-security-analyst        # K. Lund, ROLE-NR-SEC-DET
    artefact: DET-NR-0012 v1.0.0 (new rule; severity high on benefits-triage servers)
    drafted_by: owner with detection-drafting assistant   # Level 1 Assisted; the assistant did not choose the scenario
    reviewer: second-analyst
    submitted_to_provider: 2028-04-04
  - route: mandate               # provider closure class, via annex MS-2 evidence request
    owner: ciso-office
    request: "State whether alerts from DET-NR-0012 can be closed by the provider's AI under any closure class; run an injected test for this pattern"
    response_due: 2028-04-18     # 10 working days, default adapted to the contract
    injected_test_due: 2028-05-02
tester_role: quality of finding only; does not tune DET-NR-0012
cost:
  replay_days: 30
  alerts_per_week_at_replay: 2
  budget_per_week: 8
handed_to_p5:
  regression_case: REG-NR-0006
  closure_requires: provider passes its injected test, reported by the Region's analyst
report_note: "Open at the first report to the audit committee, 2028-04-30, unless the injected test has passed"
review_date: 2028-09-30          # after the autumn catalogue run
status: merged

The two records show what a buyer of the loop can do. The Region cannot edit the provider's closure behaviour; it can own the content, state the requirement, ask for the class and the rationale, and make closure depend on a test the provider passes. The finding is expected to be open when the audit committee receives its first report (AIDR-0800), and the record says so. That is an honest answer to the committee's question: a spring run confirmed a declared gap in detection, the Region fixed what it owns, and the rest waits for a test with a date.

Across the degrees of delegation

The mode describes how the AI is deployed and whose identity it uses; the level describes who decides each action. The table sets default ceilings for P4 Improve. None of these actions touches a named person's access to work, and a change to a production control or a mandate is never merged without a named reviewer.

Activity Usual mode Default ceiling Condition
Group findings by root cause and propose routes and order Mode 3 or Mode 4 Level 1 Assisted A person confirms route and order; no priority is set by the assistant
Draft a detection from a behaviour description Mode 1 to 3 under the engineer's identity; Mode 4 or 5 drafting agent Level 1 Assisted Behaviour description only, not the runner's script; negative test required; cap of six open drafts per reviewer
Open a pull request for a detection, control or playbook change Mode 3 or Mode 5 agent with a mandate Level 2 Supervised A named reviewer who is not the drafter or the scenario's author approves
Run replay and unit tests on a draft Pipeline Level 3 Bounded Catalogued checks; results attached to the record
Merge a detection change to the shadow stage Pipeline Level 2 Supervised A person approves the merge; promotion past shadow is a person's decision
Roll back a detection that exceeds twice its budget for two weeks Pipeline Level 3 Bounded Rule written as numbers in the record; reversible by revert
Change a production control or a playbook step Person with an assistant Level 1 Assisted Owner decides; reversibility checked; no autonomous change
Lower an agent's level or narrow a mandate Delegator Immediate, recorded Takes effect at once; raising it is a decision by a named authority after a re-test
Accept a risk or set an expiry Person Level 0 Advisory for any AI A role with authority over the risk; never the person who would do the work

Level 4 Autonomous does not appear in the table: a change to production controls drafted, approved and deployed by agents with no named reviewer is outside the design.

Nordvik Region and Hansa Mutual

Nordvik Region buys the loop, and P4 Improve is where it finds out what it bought. A small library, a contracted tester twice a year and a team of six leave little spare capacity, so the discipline is to make every change small, owned and re-testable. Karin Lund owns the detection content and the finding's route, and the call-off agreement keeps the tester from tuning the Region's rules. The Region sends a finding to its managed provider through annex MS-1 and tracks the response against a date; it uses the quarterly injected tests of annex MS-2 as the re-test for provider-side behaviour. Its principal risk is a loop that stops after the first report, with findings in a register that no one has time to age-check. Its protection is the register's single accountable role, the review date on every change and the findings store with limited access.

Hansa Mutual runs the loop, and its detection-engineering team of four drafts changes with an AI assistant. The group's finding-to-control median of 19 days (BTP-HM-2027-Q3) is the starting point, and the nine priority techniques not verified at that date (four with expired evidence, three that failed a re-test, two never tested) are the work the continuous pipeline generates. Its risk is volume: findings from the pipeline, the simulation tool, the quarterly sessions and the annual exercise arrive faster than four engineers can review. Its protection is the cap on open drafts, the retire-to-make-room rule, grouping by root cause and a reviewer who is never the drafter nor the scenario's author. Where Nordvik's independence is by contract, Hansa's is designed between agents, teams and model families that share a repository.

Summary

This chapter treated P4 Improve, and the points at which it meets P3 Observe and P5 Verify. A finding is turned into a change by a recorded decision: the layer that failed, the routes considered, the route chosen, the owner, the reviewer, the cost, the expiry and the regression case that P5 Verify will run. The routes are a detection, a control, a playbook, a mandate or a recorded risk acceptance, and the finding is split by route so that no route hides behind another. The tester who authored a scenario does not tune the detection that answers it.

AI drafts every kind of change, and the bottleneck is review. Overfitting to the exercise, shared blind spots between the drafter and the reviewer, and the findings store as a map of the estate are the risks specific to the loop. The remedies are drafting from a behaviour description, variants and a negative test, a reviewer who is a named person and not the drafter, caps on open drafts, seeded errors and a findings store with limited access. The cost of a change is counted: replay, shadow, an alert budget, a tune-or-retire rule and the retirement of an older rule to make room. Findings are ordered by technique importance and evidence age, by what the defence did and by what is at stake, and the register reports the gap between merged and verified, the finding half-life.

Findings that cannot be fixed end in a compensating control, a recorded risk acceptance with an owner and an expiry, or a transfer, never in silence, and the coverage record keeps showing them as failed or declared. The two worked examples show a Hansa detection merged at day 11 of a 14-day target and still unverified, and a Nordvik change that waits on a provider's injected test and is expected to be open at the first report to the audit committee. Chapter 11 turns to P5 Verify: the re-test, the variants, the regression case and the evidence manifest that decide whether any of these changes closes a finding.

Sources

  1. European Parliament and Council of the European Union (2022). Regulation (EU) 2022/2554 on digital operational resilience for the financial sector (DORA). Official Journal of the European Union. (2, European Union, 2022)
  2. European Parliament and Council of the European Union (2022). Directive (EU) 2022/2555 on measures for a high common level of cybersecurity across the Union (NIS 2 Directive). Official Journal of the European Union. (3, European Union, 2022)
  3. European Central Bank (2025). TIBER-EU: Threat Intelligence-Based Ethical Red Teaming framework. ECB. (20, ECB, 2025)
  4. SigmaHQ. Sigma: Generic Signature Format for SIEM Systems. (30, SigmaHQ, n.d.)
  5. Nordqvist, G. (2025). Architecture as Code. The Transformed Architecture, Book 1. (87, Nordqvist, 2025)
  6. Nordqvist, G. (2026). AI Red Team: AI as an Active Part of Cyber Defence in European Enterprises and Public Authorities. The Transformed Architecture, Book 4. (90, Nordqvist, 2026)
  7. Nordqvist, G. (2026). The Transformed Security Architect: Role, Craft and Career in the Age of AI in European Enterprises and Public Authorities. The Transformed Architecture, Book 6. (92, Nordqvist, 2026)
  8. Alahmadi, B. A., Axon, L. & Martinovic, I. (2022). 99% False Positives: A Qualitative Study of SOC Analysts' Perspectives on Security Alarms. 31st USENIX Security Symposium. (96, Alahmadi et al., 2022)
  9. Tariq, S., Baruwal Chhetri, M., Nepal, S. & Paris, C. (2025). Alert Fatigue in Security Operations Centres: Research Challenges and Opportunities. ACM Computing Surveys 57(9). (98, Tariq et al., 2025)
  10. Bainbridge, L. (1983). Ironies of Automation. Automatica 19(6). (100, Bainbridge, 1983)
  11. Parasuraman, R. & Manzey, D. H. (2010). Complacency and Bias in Human Use of Automation: An Attentional Integration. Human Factors 52(3). (101, Parasuraman & Manzey, 2010)
  12. Nordqvist, G. (2026). AI Blue: How Artificial Intelligence Transforms the Blue Team in European Enterprises and Public Authorities. The Transformed Architecture, Book 8. (94, Nordqvist, 2026)