Emulate Continuously: Breach-and-Attack Simulation, Atomic Tests and Automated Red Teaming

This chapter treats adversary emulation as a pipeline that runs without a campaign around it: breach-and-attack simulation, atomic tests and continuous automated red teaming, triggered by change, by schedule and by intelligence. It concerns P2 Emulate above all. It touches P1 Target, because the pipeline can only run what the scenario library holds, P3 Observe, because every execution is worthless without a telemetry tag that lets the result be compared with the expectation, and P5 Verify, because a pipeline that re-runs scenarios is the cheapest regression machinery the loop has. The chapter answers four questions: what runs in the pipeline, where and when; how variation is introduced so that detections are not overfitted to their own tests; what realism costs; and whether continuous automated emulation makes the quarterly human-led session, or most of the red team's human hours, unnecessary. The last question is staged as a debate in four steps, followed by a verdict.

Figure 8.1: A continuous emulation pipeline: trigger, safety class gate, variation, execution, tagged result, and an abort path.

The chapter builds on earlier conclusions. AI Platform and Security, Chapter 17, kept the external exercise for discovery and converted every finding into an automated test that also runs on a schedule (89, Nordqvist, 2026). AI Red Team, Chapter 25, concluded that continuous verification needs catalogued, reversible experiments, a bounded blast radius and a stop path outside the system being stopped, with AI planning and narrow deterministic actuators acting (90, Nordqvist, 2026). AI Threat, Chapter 30, supplied the classification of controls as verified, declared or failed (91, Nordqvist, 2026), and AI Blue, Chapter 11, the rule that every detection built from an exercise carries the exercise's scenario as a regression test (94, Nordqvist, 2026). This chapter asks what changes when the re-running is automatic and frequent.

The pipeline

Three kinds of tool sit behind the phrase continuous emulation, and they differ in what they are good for.

Breach-and-attack simulation is the procured kind: a commercial product, usually deployed as agents or virtual appliances in the estate, that executes a vendor-maintained library of simulated attack behaviours against controls and reports whether each was prevented or detected. Its strength is breadth and repeatability at low operating cost. Its weakness is that the library is the vendor's and the logic is often opaque; the product is a Mode 4 built application that needs the same mandate, logging and stop-path discipline as any other agent that holds credentials. A tool whose scenarios the organisation cannot read is a declared control in the sense of AI Threat, Chapter 30, until the contract grants the right to inspect and to test it.

Atomic tests are the open kind: small, single-technique test procedures, each mapped to a MITRE ATT&CK technique, kept in a public library that the community maintains (32, Red Canary, n.d.). An atomic test answers one narrow question, "does this telemetry source record this behaviour, and does this detection fire", and it does so reproducibly. An organisation can read every test before it runs and pin the catalogue revision. The same openness is the weakness: defenders and adversaries read the same catalogue, and a detection tuned only to the public form of a test is the first example of overfitting that this chapter treats.

Automated red teaming is the planned kind: a platform that chains techniques into an operation, chooses among catalogued abilities according to what it finds, and reports what it did. Open-source platforms such as MITRE Caldera (Chapter 2) are built on ATT&CK (33, MITRE, n.d.); an AI-assisted variant adds a planning component that proposes a sequence from a plan, which a deterministic executor runs. AI Red Team, Chapter 12, gave the technical design. What matters here is where the model sits: a planner that proposes and a runner that executes catalogued abilities inside a signed scope carry different risks, and the design keeps the planner away from estate credentials.

At Hansa Mutual the three come together as HM-SEC-004, recorded as AIDR-0314: catalogued, benign emulations across the group estate, run from the red-team range, with no route into blue tooling except through findings. The emulation runner is a Mode 5 agent under MND-HM-EMU-01. It runs at Level 3 Bounded for scenarios in safety class S1, which are benign, reversible and tagged, inside the signed scope, and at Level 2 Supervised for anything else. The simulation tool runs a weekly atomic suite beside it. Chapter 14 fixes the full set of loop agents; this chapter needs only the runner.

A pipeline is not a campaign. A campaign (Chapter 7) is a plan built from intelligence, executed by people, with a beginning and an end and a report. A pipeline has no end and no author in the room. Its output is a stream of tagged executions, each compared with an expectation, and its value is that the stream never stops. A campaign shows whether the defence handled a coherent adversary this time; a pipeline shows whether a catalogued behaviour is still seen today, and again tomorrow after the next change.

It also differs from security chaos engineering (76, Rinehart & Shortridge, 2023) (77, Basiri et al., 2016). A chaos experiment checks that a steady state holds when a condition is injected, for example that a log source keeps flowing when a connector fails; an emulation execution checks that the defence reacts to adversary behaviour. They share infrastructure and safety discipline and test the machinery around detection and detection itself.

What runs where

Where an execution runs decides how much it can prove and how much it can harm. Three environments matter, and a fourth is a way of using telemetry and not a place.

Environment	What it proves	What it risks	Default
Red-team range	That the scenario executes as designed and produces the expected raw events	Nothing in the estate	Every new scenario and every new variant runs here first
Pre-production mirror	That detections fire on telemetry shaped like production, and that a change did not break them	Little, if the mirror is isolated and its identities are separate	Scenarios above S1 and every variant before promotion
Production, safety class S1 on the pipeline's own authority; S2 only with white-team approval per run	That the real log pipeline, the real connectors and the real detections work today	Operational noise, analyst attention and, if the scope is wrong, a real incident	Benign, reversible, tagged scenarios on a schedule
Shadow replay	That a new or changed detection finds a recorded execution in stored telemetry	Nothing, because nothing runs	Detection changes, before they are merged

The rule that follows is the one AI Red Team, Chapter 7, and Chapter 6 of this book set for rules of engagement: production carries only what is safe by construction. A scenario in safety class S1 creates a marker object, writes a benign file, attempts a sign-in against a canary account below any lockout threshold or issues a query whose only effect is a log line; it can be undone by deleting what it made, and it is tagged so that every event it produces carries the run identifier. Anything that could damage data, leave the signed scope, touch personal data beyond what the exercise is authorised to touch, or affect a named person is above S1, is not run by the pipeline in production on its own authority. Class S2 runs in production only with white-team approval per run (Chapter 6); classes S3 and S4 are never run in production by the pipeline, and are run by people in the pre-production mirror, on the range or in a campaign under a signed manifest.

Three questions are asked of every scenario before it enters the production rotation, and the answers are part of its record. What can be undone, and by what? Who can stop it, and how fast? What does it do to a third party who has not consented? The first answer is a cleanup step that the runner verifies, not one it merely attempts. The second is a stop path outside the runner: the policy decision point that authorises each execution can revoke the runner's credentials, and the right to do so is held by named people, the white-team lead and the SOC shift lead, who need no approval and owe no justification. A stop that has not been drilled recently is a declared control, not a verified one (Chapter 6). The third answer is a consent register. A pipeline that runs without it can turn a test into an incident for someone else.

The caution is not abstract: the incidents of Chapter 6 show that an agent which can act in an environment can leave the intended scope, and that the control which holds is enforced outside the agent (56, UK AISI, 2026).

Scheduling around change

A pipeline has three kinds of trigger, and each answers a different failure.

Change triggers answer the failure that a detection that worked yesterday does not work today. Detections break when a log source changes format, when a connector is upgraded, when a rule is edited, when an agent's mandate changes and when a provider alters its model. A change to any of these re-runs the scenarios for the techniques it can affect, within a stated delay. This is the trigger that most directly produces the regression cases of Chapter 11, and it is the one whose absence is most often discovered by an incident.

Schedule triggers answer the failure that nothing changed and the defence decayed anyway. Evidence expires: attackers' procedures move, the estate drifts, and silent failures in telemetry go unnoticed. The cadence is set from the age at which evidence stops being trusted. In this book's illustrative Hansa Mutual profile, evidence for a priority technique expires after 90 days and for other techniques after 180 days, and the pipeline's own target is tighter than the expiry so that evidence is renewed before it lapses: every priority technique is exercised at least every 30 days.

Intelligence triggers answer the failure that the world moved. A new technique in a threat report, a vulnerability added to a catalogue of known exploited vulnerabilities such as the one CISA maintains (28, CISA, n.d.), or a request from the threat-intelligence cell, asks whether a scenario exists for the behaviour and, if it does, runs it. Google's threat-intelligence analysis reports that exploitation in 2026 grew mainly through the rapid weaponisation of known vulnerabilities (n-days), which it suggests, without having shown it, may be eased by AI tools (65, Grunewald et al., 2026), and that is the reason the intelligence trigger has a deadline of days and not of the next quarter. A new scenario created by an intelligence trigger is a P1 Target decision, not an automatic one: the pipeline runs it only after a person has approved the scenario record and its safety class.

The pipeline holds back during a freeze window or a major incident, and records each hold-back so that coverage reports show a technique's evidence age honestly.

Practical guide: Choosing triggers and cadence for a continuous emulation pipeline

Purpose. Set triggers and cadence that keep evidence for the priority techniques inside its expiry, without exceeding what the blue team can read.

Roles. Emulation engineer (owns the pipeline), detection engineer (owns the detections the pipeline tests), SOC lead (owns analyst capacity), white-team lead (owns the stop path and the consent register), CISO (accepts the residual risk), employee representative (agrees the reporting rule where the exercise touches staff).

Steps.

1. List the priority techniques and, for each, the evidence expiry in days. Defaults: 90 days for a priority technique and 180 for others. Adapt them to the risk appetite and write the adapted values down.
2. Set the schedule trigger to one third of the expiry for priority techniques (30 days for a 90-day expiry) and to one half for the rest. The margin absorbs held-back runs and failed executions.
3. For each detection, connector and mandate, list the techniques it can affect and attach a change trigger with a maximum delay of 24 hours after the change is merged or deployed. Treat a provider's notice of a model change as a change.
4. Define the intelligence trigger as a named intake with a deadline: scenario record approved within 5 working days, S1 execution within 2 working days of approval, or a recorded decision not to run.
5. Cap the volume. Estimate the alerts per week that the pipeline will generate and compare them with the alert budget of the affected detections; if the pipeline would exceed a quarter of the budget, reduce the frequency for non-priority techniques first.
6. Add the hold-back rules (change freeze, active major incident, third-party maintenance window) and require that each hold-back is logged with the technique's evidence age.
7. Drill the stop path once per quarter and record the time from the stop request to the last execution halting. A default target is 2 minutes (Hansa holds the runner of HM-SEC-004 to 60 seconds, Chapter 6), the figure used in the stop drill of the illustrative Hansa year in AI Blue.
8. Review the cadence every six months against the evidence ages that the coverage records of Chapter 17 show, and against the analysts' view of the noise.

Definition of done. Every priority technique has an evidence age under its expiry, the three trigger types are in the pipeline definition, hold-backs and stops are logged, and the cadence and the cap were reviewed with the SOC lead in the last six months.

Variation

A scenario that is run unchanged teaches the defence only that scenario. The aim of variation is to test the technique and not the test: the same ATT&CK technique expressed through different parameters, sequences, timing, tools or hosts, so that a detection which fires only on the form it was written against is revealed.

Variation takes three forms, in increasing order of difficulty. Parameter variation changes values that should not matter: the account set, the host, the time of day, the order of two steps, the volume and the pacing of a repeated action. Procedure variation reaches the same technique by a different procedure, for example a different binary or a different interface to the same capability; ATT&CK itself records many procedures for a technique (17, MITRE, n.d.). Chain variation changes the sequence of techniques, which tests the correlation logic and the investigation, and not only the individual rule.

AI is useful here. A model can propose variants of a scenario faster than an emulation engineer can write them, and can check that a proposed variant stays within the technique's definition and the scenario's safety class. Open tooling already makes repetition cheap without any model; what AI adds is cheaper variation, planning and drafting, and no source in the bibliography measures that yet. It is expected: AI-generated variants at scale are a natural extension of the planning assistance that AI Red Team, Chapter 12, already describes, and they raise a question of independence that Chapter 15 treats in full, because a model that proposes the variants and a model that drafts the detections may share blind spots. Three controls apply meanwhile. The model proposes at Level 1 Assisted and a person approves a variant before it enters the catalogue. A variant inherits the safety class of its parent and can never exceed it, and the policy decision point enforces that, not the model. And each variant is a versioned scenario record with its parent, so that a regression case in Chapter 11 can cite both.

Overfitting and Goodhart

A detection engineer who reads the failing scenarios and edits the rule until they pass is doing the right thing, and doing it produces the conditions for overfitting. The pipeline's pass rate then measures how well the detections fit the scenarios, and not how well they detect the technique. This is Goodhart's law in the formulation of Strathern (80, Strathern, 1997), which Chapter 18 explains once; in a continuous loop it appears in four specific ways.

1. Marker leakage. The telemetry tag that gives ground truth is visible to the detection logic, so a rule fires on the tag and not on the behaviour. The test is mechanical: strip the tag from the detection's input and re-run.
2. Signature fitting. A rule matches the exact artefacts of the public form of an atomic test. It passes every run and catches nothing that differs in a single parameter.
3. Catalogue closure. The scenario library grows only from past findings and from the pipeline's own gaps, and the detection estate converges on the library. What the library does not describe is untested, and the dashboard cannot say so.
4. Dashboard comfort. A pass rate near 100 per cent over a stable library is read as safety. The research on human use of automation describes the human side: reliance on an automated aid brings complacency and reduced checking (101, Parasuraman & Manzey, 2010), and Bainbridge's analysis of automation points out that when the routine work is automated, the people left with the residual work lose the practice they need for it (100, Bainbridge, 1983).

The design response is the one used for machine-learning models and applied here as the book's own conclusion: keep a held-out set. A share of the scenarios and variants, the held-out share, is hidden from the detection engineers, is never used to tune a rule, and is run on the same schedule as the rest. The difference between the pass rate on known-form scenarios and the pass rate on held-out variants is the generalisation gap, and it is the pipeline's most honest number. A small gap says that detections generalise. A wide gap says that the green dashboard is a measure of fit. When a held-out variant fails, it becomes a finding and, once the detection is fixed, joins the known set, while a new held-out variant replaces it. The rotation costs scenario-writing effort, the price of a number that means something.

AI complicates the held-out set in one way. If the model that proposes variants also drafts detections, the set is not held out from it. The independence principle of Chapter 15 requires different model families and separate repositories for the two roles, and a human-authored share that neither model has seen.

Worked example: A pipeline definition for HM-SEC-004 (illustrative)

The definition below is the version-controlled file that the pipeline reads. The policy decision point enforces it outside the runner.

pipeline: HM-SEC-004
decision_record: AIDR-0314
mandate: MND-HM-EMU-01
rules_of_engagement: RoE-HM-2027-PURPLE
usage_mode: mode-5
delegation:
  safety_class_S1_inside_scope: level-3-bounded
  above_S1: level-2-supervised
  level_4: never
triggers:
  change:
    on: [detection-merged, connector-changed, mandate-changed, provider-model-notice]
    scope: affected-techniques
    max_delay_hours: 24
  schedule:
    priority_technique_max_age_days: 30
    other_technique_max_age_days: 90
    atomic_suite: weekly
  intelligence:
    intake: threat-intelligence-cell
    scenario_approval_days: 5
    first_run_after_approval_days: 2
gates:
  policy_decision_point: pdp-emulation
  consent_register_required: true
  hold_back_when: [change-freeze, major-incident, provider-maintenance]
variation:
  held_out_share: 0.25                 # pipeline default; the library default is 15 per cent and at least eight scenarios (Chapter 15)
  parameter_seed: per-run
  variant_safety_class: inherit-from-parent
  proposed_by: scenario-curator-level-1
telemetry:
  tag_field: exercise.run_id
  tag_visible_to_detections: false
  ground_truth_store: restricted
abort:
  holders: [white-team-lead, soc-shift-lead, group-ciso-deputy]
  mechanism: pdp-revokes-runner-credentials
  target_seconds: 60                   # runner of HM-SEC-004 (Chapter 6); the generic default is 120
  drill_interval_days: 90
evidence:
  expiry_days: {priority: 90, other: 180}
  report: aggregate-only

The file is short on purpose. Its important lines are the ones that say what the runner cannot do: it cannot exceed Level 3, it cannot run above S1 on its own authority, it cannot see its own abort mechanism and the detections it tests cannot see its tag.

Worked example: A Nordvik atomic-test subset as code (illustrative)

Nordvik Region has no pipeline. It has a contracted tester, a call-off agreement NR-2026-071 and two catalogue runs a year. The question it puts is which five atomic tests give the most for the tester's report. The selection rule is four lines: the technique is on the priority list for the benefits-triage service; the test is safe by construction on a Region-owned test host; the five tests cover five different tactics; and at least two have an existing detection, so that the result is a regression, while at least two are expected to find no detection, so that the result is a finding the Region can act on.

emulation_plan: EMU-NR-2028-01
rules_of_engagement: RoE-NR-2028-001
call_off: NR-2026-071
executor: contracted-tester-human-run
catalogue: atomic-red-team
catalogue_revision: pinned-at-signature
environment: region-owned-test-host
window: {start: 2028-03-20, end: 2028-03-24}
safety_class_max: S2                 # SCN-NR-0001 is S2 (Chapter 6); the other four are S1
tests:
  - {scenario: SCN-NR-0001, technique: T1110.003, tactic: credential-access, expected_detection: DET-NR-0001}
  - {scenario: SCN-NR-0004, technique: T1059.001, tactic: execution, expected_detection: none}
  - {scenario: SCN-NR-0006, technique: T1053.005, tactic: persistence, expected_detection: none}
  - {scenario: SCN-NR-0007, technique: T1087.002, tactic: discovery, expected_detection: none}   # declared gap (Chapter 7)
  - {scenario: SCN-NR-0010, technique: T1021.002, tactic: lateral-movement, expected_detection: unknown-to-be-recorded}
cleanup_verified_by: tester-and-detection-owner
report_to: [detection-owner, ciso-office]
report_fields: [outcome, telemetry-present, detection-id, evidence-date, finding]

The first test is the password-spray pattern for which Karin Lund's DET-NR-0001 (AI Blue, Chapter 6) is the detection, run against a canary account set agreed in the rules of engagement and below any lockout threshold. The other four are chosen because they are cheap to repeat and exercise different parts of the telemetry. The expected_detection field is written before the run: Chapter 9 argues that expectation comes first, and a report that records only what was found cannot show what the Region expected to find. The report uses the fields at the end, so that the next run, in autumn, compares like with like.

The cost of realism

Realism makes an emulation worth running, and it has three costs that decide the cadence.

Compute and licence cost. A procured tool has a licence, a planner a model cost per run, a range and mirror infrastructure. The cost is modest next to a red team's hours, and a runner with credentials is a privileged machine identity that must itself be secured (AI Red Team, Chapter 25).

Risk. Realism moves an execution towards a real incident. The more faithful the behaviour, the more likely it is to trip a real control, to disturb a production workload or to be mistaken for an intrusion by someone who was not told. This is the risk the safety classes and the stop path manage, and the reason the pipeline stays at S1 in production. It also moves a cost to third parties, whose consent may be needed; the cost cannot be removed by design and can be bounded by it.

Attention. This is the cost most often left out. Each execution produces telemetry, and some produce alerts that an analyst or an agent must close. A pipeline that adds 200 alerts a week to a team already over its alert budget gives it a queue, not evidence. The qualitative literature on alert fatigue reports tedium and desensitisation from false positives (96, Alahmadi et al., 2022) (98, Tariq et al., 2025); it does not measure simulated alerts, so the conclusion is a design rule and not a finding: tagged emulation alerts are routed to a separate lane, counted against the alert budget, closed by the same class policy as other alerts (AI Blue, Chapter 7) and measured at team level only. Where exercises involve staff, the representatives' agreements apply. At Hansa Mutual that means clause KBV-IT-2024-SOC-05, which requires advance information, aggregate reporting and no use of exercise results in appraisals.

The three costs lead to a rule of thumb that is the book's own conclusion and not a measured result: realism is bought where it changes a decision. A scenario that is cheap, safe and answers a question about a priority technique runs continuously. A scenario that is expensive, risky or needs judgement runs as a campaign, with people, a plan and a white team.

The debate: does continuous automated emulation make the quarterly human-led session, or most of the red team's human hours, unnecessary?

The question is asked in both organisations in the same month. Hansa Mutual's executive board asks, after the cost of the quarterly sessions, whether HM-SEC-004 makes the quarterly purple session redundant (AIDR-0850, 2028-01-18). Nordvik Region's audit committee asks what the first threat-led exercise has proved. The debate is staged so that each position is stated at its strongest, then the common ground is separated from the disagreement.

The case for concern

The concern is that a pipeline repeats the known and a loop that stops asking people to be creative loses the capacity to find what nobody has thought of.

The argument has a structural part and a human part. The structural part is that every automated scenario is a scenario somebody wrote, drawn from a library of behaviours that were observed, published or imagined; ATT&CK is a knowledge base built from observed adversary behaviour (17, MITRE, n.d.). An intrusion that combines legitimate business processes in an unlisted way is not in any catalogue. Variation widens the neighbourhood of what is tested and does not leave it. A dashboard of green results over such a library reports that the known is still detected, and an organisation that reads it as "we are secure" has mistaken the library for the world. The overfitting section above gives the mechanism by which the library and the detections converge.

The human part is the research on automation. Bainbridge argued in 1983 that automating the routine part of a task leaves people the residue that is hardest to do and removes the practice that would let them do it (100, Bainbridge, 1983). Parasuraman and Manzey's review reports complacency and automation bias in people who rely on automated aids (101, Parasuraman & Manzey, 2010). AI Threat, Chapter 14, drew the consequence for the route by which experts are made: automating junior work removes the apprenticeship (91, Nordqvist, 2026). If the pipeline takes the red team's routine work, the next red-team operators lose the work on which they learn. A red team reduced to maintaining a pipeline is still a team, and it is a team that does not practise the one skill the pipeline cannot supply.

A further concern is independence. AI Threat, Chapters 23 and 30, concluded that assurance cannot rest on one evaluation where models may behave differently under test (91, Nordqvist, 2026); a pipeline in which one set of models plans scenarios and another drafts detections can agree with itself.

The case for calm

The calm position is that automation delivers what no red team can afford, and that, given what it delivers, the board is right to ask how many human hours the same assurance still needs.

Its first argument is arithmetic. A human red team runs a handful of campaigns a year. The regulatory frame is periodic by design: DORA Article 24(6) requires that appropriate tests are conducted at least yearly on ICT systems and applications that support critical or important functions, and Article 26(1) requires advanced testing by threat-led penetration testing at least every three years for the entities in scope (2, European Union, 2022). NIS 2 Article 21(2)(f) requires policies and procedures to assess the effectiveness of cybersecurity risk-management measures (3, European Union, 2022). Between the dates of those tests, detections change daily and evidence decays. A regression of known techniques every day is a better answer to "does it still work" than a yearly test, and the cost per execution is small.

Its second argument is that the known is not trivial. The book's reading of the sources of Chapter 3 is that most of the techniques adversaries use in practice are old ones, and that the failures organisations find are more often a broken log source, a disabled rule or a silent connector than a novel technique. A pipeline finds those within a day, where an annual test finds them after a year. This is the same reasoning that supports continuous verification in security chaos engineering (76, Rinehart & Shortridge, 2023). It is an argument from the shape of common failures, and the evidence for it in any one organisation is its own history of findings.

Its third argument is about people. A red-team operator's hours are scarce, and spending them on re-running last year's techniques is a poor use of them; automation takes the repetition and leaves the work that needs judgement about consequence and social context.

Its fourth argument is cost, and it is the one the board is entitled to press. The human-led session raised five findings in the quarter and three of them only a person could have found; if a year of sessions yields no more than that, the same hours might buy more as a smaller protected share (a default of 30 per cent is the book's own figure, not a measured optimum) and a twice-yearly session, and the board may ask for the yield per hour before it keeps the cadence. On this view the human red team is not replaced, but a part of its hours is, and the question is how large a part.

Where the positions meet

Both positions accept more than the headline suggests. They agree that the pipeline repeats and varies what is known, and that it does so cheaply, continuously and at a scale no team of people can match. They agree that novelty, the invention of a path nobody catalogued, is human work. They agree that periodic threat-led testing is not replaced: the concern says it because novelty is human, and the calm position says it because the regulatory frame requires it, while AI Platform and Security, Chapter 17, likewise keeps the external exercise for discovery (89, Nordqvist, 2026). And they agree that evidence from automation is only as independent as the pipeline's design, so that a wide generalisation gap, a shared model family or a visible tag weakens it.

What remains in dispute is the size of the human share and the cadence of the session, which the novel-finding yield (action 2 below) is meant to settle. Whether a loop that automates the routine and keeps a smaller human-led share sustains the red team's capacity for novelty over several years has not been shown in either direction, and each organisation has to read its own history of findings to weigh novel intrusions against broken known detections.

What can be done

1. Keep a protected share of human-led, novel emulation. Set it as a share of red-team hours (a default of 30 per cent, the book's own figure and not a measured optimum, reviewed annually) and protect it from being consumed by pipeline maintenance. At Nordvik, which buys testing, the equivalent is a clause in the call-off agreement that a stated part of each tester visit is open-ended, with no pre-published scenarios.
2. Measure novel-finding yield separately. Count findings that no catalogued scenario could have produced, per 100 hours of human-led emulation, and report it beside the pipeline's regression results, with the hours spent, so that the yield per hour can answer the board's cost question and decide whether the session stays quarterly. A pipeline's findings and a campaign's findings answer different questions and are not summed.
3. Vary on purpose, and publish the gap. Keep the held-out share and report the generalisation gap. Treat a gap above ten percentage points as a finding on the detections.
4. Review what the pipeline does not cover. For each priority technique, state whether a scenario exists in S1, in the pre-production mirror only, or not at all, and review the list quarterly with the white-team lead. Untested is not covered.
5. Put independence in the design. Different model families for variants and detection drafts, separate repositories, and a human-authored share (Chapter 15).
6. Protect the apprenticeship. Give junior operators and detection engineers work on writing and reviewing scenarios and variants, so that the route by which experts are made survives the automation (Chapter 24).

Verdict

The evidence supports a verdict on three levels.

Observable. Breach-and-attack simulation, open atomic-test libraries and automated adversary-emulation platforms are documented and in use (32, Red Canary, n.d.) (33, MITRE, n.d.). Overfitting to one's own tests is a known property of any measure that becomes a target.

Expected. AI-generated variants at scale, and planners that chain catalogued abilities under a mandate, are expected to become routine in large organisations as the planning assistance of AI Red Team, Chapter 12, extends. The automated evidence is then expected to outrun what a person can review, and the control is the held-out share and the independence design, not more dashboards.

Speculation. That continuous automated emulation replaces the human red team, or that most of its human hours can safely be cut, is speculation. No evidence shows that a loop with no human-authored novelty and no independent verification detects novel intrusions better over several years, as judged by independent threat-led tests, and the thesis of this book would be shown wrong by such evidence. The book's own conclusion from the arguments above is that continuous automated emulation changes the human red team's work and does not replace it, while the size of the human share remains an open question for the yield measure to settle: the routine passes to the pipeline, and the humans' time goes to novelty, to the review of what the pipeline does not cover, to the held-out set and to the stop path.

What people and organisations can influence is the design: the safety class gate, the share of hours protected for novel work, the held-out set, the tag that detections cannot see and the people who hold the stop right.

Across the degrees of delegation

Activity in the pipeline	Mode	Ceiling	Who decides	Note
Running an S1 scenario inside scope	Mode 5 (runner) or Mode 4 (procured tool)	Level 3 Bounded	The policy decision point enforces the manifest; the delegator is the red-team lead under MND-HM-EMU-01	Catalogued, reversible, tagged; stop path held outside the runner
Running a scenario above S1	Mode 5	Level 2 Supervised	An exercise controller approves each run	Pre-production mirror by default
Proposing a variant	Mode 3 or Mode 5 planner	Level 1 Assisted	An emulation engineer approves	Inherits the parent's safety class
Approving a new scenario from intelligence	Mode 1 to Mode 3	Level 0 Advisory	The red-team lead	A P1 Target decision, not automatic
Tuning a detection from a failed run	Mode 3	Level 1 Assisted	The detection owner	Held-out scenarios never used for tuning
Any emulation action that is irreversible, leaves the signed scope, touches personal data beyond the authorisation or targets a named person	None	Not delegated; Level 4 is never used	A person under a signed manifest	Outside the pipeline altogether

Nordvik Region and Hansa Mutual

Hansa Mutual runs the loop, and its lever is the pipeline. In the first quarter of 2028 (illustrative figures), HM-SEC-004 ran 41 scenarios in S1 on the schedule above, and 31 of the 34 priority techniques were exercised within 30 days; the other three have only scenarios above S1 and are run by people. Of the 41 known-form scenarios, detections fired on 37 (90 per cent). Of 14 held-out variants, detections fired on 10 (71 per cent), a generalisation gap of 19 points, above the ten-point finding threshold that AIDR-0889 later fixed as a trigger (2028-04-14). The 8 failed executions were merged into 6 findings. The human-led quarterly session of 2028-03-09 (PRP-HM-2028-01, led by Mareike Hartmann and chaired by the white-team lead) raised five findings, of which three could not have been produced by the pipeline: three telemetry gaps in places its scenarios cannot reach, one of them waiting on a data-transfer decision for the Swiss entity (Chapter 9). The first report on AIDR-0850 is due by 2028-04-30; the design reasoning above points to keeping the session and changing its agenda: pipeline results and the gap in the first hour, and the remaining time on novelty.

Nordvik Region buys the loop, and its lever is the contract. It has no pipeline and no spare people, so the spring catalogue run of 2028 under NR-2026-071 is the nearest it comes to continuous emulation. The five atomic tests of EMU-NR-2028-01, run by Elin Berglund's firm under RoE-NR-2028-001, give the audit committee a dated, repeatable record per technique, which is what AIDR-0800 asked for, and the MSSP's quarterly injected tests under annex MS-2 supply the evidence between visits. The Region's risk is a loop that stops after the first report; Hansa's is a loop that runs without anyone questioning it. Both need a protected share of human-led work, and both need the stop path outside the system that is stopped.

Summary

This chapter treated P2 Emulate as a pipeline. Breach-and-attack simulation supplies breadth, atomic tests supply small readable single-technique checks, and automated red teaming chains catalogued abilities under a mandate. Production carries only scenarios in safety class S1, which are safe by construction, tagged and reversible, and a stop path held by named people outside the runner governs everything. Triggers by change, schedule and intelligence keep evidence for the priority techniques inside its expiry, and the cadence is bounded by what the blue team can read.

The chapter's main design lesson is that a pipeline is exposed to Goodhart's law: detections converge on the scenarios that test them. The responses are variation on purpose, a held-out share hidden from detection engineers, a telemetry tag that detections cannot see, and the generalisation gap as the pipeline's honest number. It touched P1 Target in the approval of new scenarios, P3 Observe in the tag and the expectation, and P5 Verify in the regression value of every re-run.

The debate on whether continuous automated emulation makes the quarterly session or most of the human red team's hours unnecessary ended with a verdict. Continuous regression is observable practice; AI-generated variants at scale are expected; replacement of the human red team is speculation, and the book's conclusion is that the work changes and a protected share of human-led, novel emulation remains. Chapter 9 takes the next stage, P3 Observe, and asks how what happened is compared with what was expected.

Sources

1. European Parliament and Council of the European Union (2022). Regulation (EU) 2022/2554 on digital operational resilience for the financial sector (DORA). Official Journal of the European Union. (2, European Union, 2022)
2. European Parliament and Council of the European Union (2022). Directive (EU) 2022/2555 on measures for a high common level of cybersecurity across the Union (NIS 2 Directive). Official Journal of the European Union. (3, European Union, 2022)
3. MITRE Corporation. MITRE ATT&CK: Adversarial Tactics, Techniques, and Common Knowledge. (17, MITRE, n.d.)
4. European Central Bank. TIBER-EU: Threat Intelligence-Based Ethical Red Teaming framework. (20, ECB, 2025)
5. Cybersecurity and Infrastructure Security Agency. Known Exploited Vulnerabilities Catalog. (28, CISA, n.d.)
6. National Cyber Security Centre (2026). NCSC statement in response to recent incidents resulting from frontier AI evaluations. (55, NCSC, 2026)
7. AI Security Institute (2026). Incident Report: unsanctioned agent behaviour during cyber testing. (56, UK AISI, 2026)
8. Grunewald, R., Mazumdar, S. & Vanderlee, K. (2026). Vulnerability Discovery and Exploitation Trends in the AI Era. (65, Grunewald et al., 2026)
9. Rinehart, A. & Shortridge, K. (2023). Security Chaos Engineering. (76, Rinehart & Shortridge, 2023)
10. Basiri, A. et al. (2016). Chaos Engineering. (77, Basiri et al., 2016)
11. Nordqvist, G. (2026). AI Platform and Security. (89, Nordqvist, 2026)
12. Nordqvist, G. (2026). AI Red Team. (90, Nordqvist, 2026)
13. Nordqvist, G. (2026). AI Threat. (91, Nordqvist, 2026)
14. Alahmadi, B. A., Axon, L. & Martinovic, I. (2022). 99% False Positives: A Qualitative Study of SOC Analysts' Perspectives on Security Alarms. (96, Alahmadi et al., 2022)
15. Tariq, S. et al. (2025). Alert Fatigue in Security Operations Centres. (98, Tariq et al., 2025)
16. Bainbridge, L. (1983). Ironies of Automation. (100, Bainbridge, 1983)
17. Parasuraman, R. & Manzey, D. H. (2010). Complacency and Bias in Human Use of Automation. (101, Parasuraman & Manzey, 2010)
18. Nordqvist, G. (2026). AI Blue. (94, Nordqvist, 2026)
19. Red Canary. Atomic Red Team. (32, Red Canary, n.d.)
20. MITRE Corporation. MITRE Caldera. (33, MITRE, n.d.)
21. Strathern, M. (1997). 'Improving ratings': audit in the British University system. (80, Strathern, 1997)