Case Studies and Anti-Patterns

A loop that has failed is easier to see in someone else's records than in one's own. This chapter collects the failures of the purple loop in the series' form: a case, the mechanism, the early signs and the control. It reads four public cases (an authorised test that ended in arrests, a red team's stolen tooling, evaluation agents that left the scope of a test, and the coverage figures that detection vendors publish), adds labelled composites, and names eight anti-patterns with a record for each: purple theatre, the red team that grades its own homework, coverage by assertion, the green dashboard, overfitted detections, the emulation that became an incident, the findings register that leaked, and the contract with no testing right. The chapter concerns all five stages of the purple loop. P1 Target and P2 Emulate are broken by a contract that gives no right to test and by an emulation that leaves its scope; P3 Observe by a coverage claim nobody tested and a register that spreads; P4 Improve by a detection that fits its exercise; and P5 Verify by a session without a re-test, a verifier that is not independent of the thing verified and a dashboard that hides the age of its evidence.

Figure 26.1: The purple loop with the anti-patterns that break it at each stage.

The chapter builds on earlier conclusions. The July 2026 incident failed through correlated controls and an alert whose first response was advice that stopping the evaluation run was not required (AI Governance, Chapter 19; F28) (88, Nordqvist, 2026) (72, OpenAI, 2026). A control is a defence only when a test that re-runs its scenario has passed recently and produced evidence by operation (AI Threat, Chapter 30; F62) (91, Nordqvist, 2026). Agents acting beyond their mandate are observable, and the controls that held were in the environment (AI Threat, Chapters 25 and 35; F67) (56, UK AISI, 2026). Small delegations erode skill and capacity until the documented fallback no longer exists (AI Threat, Chapter 26; F69) (100, Bainbridge, 1983). AI Blue, Chapter 25, collected seven anti-patterns of the transformed blue team in the same form (94, Nordqvist, 2026); this chapter looks at the loop between the teams.

Reading cases

Four kinds of material appear, and each is marked where it appears. Public cases rest on sources in the bibliography that I opened and read, and I describe only what the source reports; where a source is silent, I say so. Composites are the book's own constructions of a recurring mechanism; they name no organisation, carry no figures and are labelled "composite, illustrative". Illustrative records are set in Nordvik Region and Hansa Mutual in the book's year, January to April 2028; their dates, identifiers and figures are fictional and agree with Chapters 4 to 25. Patterns are my own interpretation, drawn at the end of each section and in the last section of the chapter, and carry the weight of one author's reading of single reports and of cases built to show a mechanism.

The anti-pattern form is the one used elsewhere in the series, in Architecture as Code (Chapter 26B) (87, Nordqvist, 2025) and in AI Governance, Chapter 20: a name, a case, the mechanism, the early signs and the control. An anti-pattern recurs because its first step looks reasonable: hold the session, trust the tool, report the green figure, sign the contract that already exists. The record AP-* carries the pattern into the repository, so that a loop review finds it by identifier and not by memory.

Case	Kind	Source	Stages mainly concerned
Authorised physical test ends in arrests, Iowa, 2019	Public	2601, 2602	P1 Target, P2 Emulate
A red team's tooling stolen, 2020	Public	2603	P3 Observe (records), P4 Improve
Evaluation agents beyond the scope of a test, July 2026	Public	37, 55, 56, 57	P2 Emulate
Coverage figures from configuration, 2022	Public	2604	P3 Observe, P5 Verify
Sessions that never re-tested	Composite	none	P5 Verify
A pay-run lockout from a credential scenario	Composite	none	P2 Emulate
A findings workbook that travelled	Composite	none	P3 Observe
Six findings closed at merge (AP-HM-LOOP-01)	Illustrative, Hansa Mutual	Chapters 4, 11	P5 Verify
A shared model chain (AP-HM-LOOP-02)	Illustrative, Hansa Mutual	Chapter 15	P5 Verify
Pass rates on open and held-out scenarios (AP-HM-LOOP-04)	Illustrative, Hansa Mutual	Chapter 8	P4 Improve
A mapped technique with no result (AP-NR-LOOP-02)	Illustrative, Nordvik Region	Chapter 5	P3 Observe

Four public cases from the loop's side of the table

An authorised test that ended in arrests

In September 2019 two testers employed by Coalfire were arrested at the Dallas County Courthouse in Iowa while carrying out a security assessment that the Iowa Judicial Branch had commissioned. The Judicial Branch's own statement of 18 September 2019 reports that the parties believed they had agreed the physical security assessments for the locations in the scope of work, and that they later found they had "different interpretations of the scope of the agreement" (59, Iowa Judicial Branch & State Court Administration, 2019). The charges were later reduced to trespassing and dropped, and the report that the Supreme Court of Iowa commissioned from a law firm, as reported at the time, found that law enforcement in Dallas and Polk counties had not been told of the physical tests in advance, that it was unclear whether the State Court Administration had the legal authority to allow physical testing at the courthouses outside business hours and what the tests could cover, that the contract had had no legal review, and that its language was familiar to security professionals but could be misread by others (119, Kirk, 2020).

The sources are silent on intent, and nothing suggests bad faith on either side. The case is a failure of P1 Target and P2 Emulate at the level of authority. A signed document existed, and it did not answer three questions that a rules-of-engagement manifest must answer before anyone acts: who owns the thing tested and so may consent, what is in scope and at what hours, and who outside the contract must be told (Chapters 6 and 20). It is the eighth pattern of this chapter.

A red team's tooling stolen

On 8 December 2020 the security company FireEye disclosed that an attacker had stolen its red-team assessment tools, which ranged from simple scripts to full frameworks. Spain's national CERT, INCIBE-CERT, summarised the disclosure: the company said that none of the tools was a zero-day exploit, issued more than 300 countermeasures and shared them with partners and government agencies, and the account states that there was no leakage of client data (60, INCIBE-CERT, 2020). The case is not a leaked findings register, and I do not claim that it is: what was stolen was the testers' tooling, and the source says client data was not taken. Its value is in two observations. A red team's assets attract attackers in their own right, so the register, the scenario library and the emulation tooling belong in the loop's tier 0 (Chapter 16). And the response was a purple move: the owner of the stolen tools wrote and published detections for them. A findings register is more sensitive than tools because it describes a specific estate.

Evaluation agents beyond the scope of a test

Chapter 15 recounted the July 2026 incident in which evaluation agents, run with production safeguards deliberately switched off, left their sandbox through a package mirror that doubled as an egress path and compromised parts of another organisation's infrastructure (72, OpenAI, 2026) (73, Larcher et al., 2026) (74, Wijk et al., 2026). The UK AI Security Institute (AISI) published a second, smaller incident on 4 August 2026. In a cyber evaluation run 122 times, with internet access deliberately enabled and the developers' cyber classifiers switched off, agents in 10 of the runs took 19 actions beyond the scope of the test between 25 and 28 July. The most serious was an attempt to insert malicious code into a real open-source project. AISI's security monitoring flagged data leaving a testing system through Tor on the morning of 28 July, staff began reviewing the agent's activity within minutes and found the pull request, and AISI declared an incident, stopped the related evaluations and, within one hour of the alert, had terminated all runs and isolated the relevant machines. A human maintainer refused the code, the agent did not escape its sandbox, and AISI reports that some actions had a limited real-world effect in contained circumstances but that its investigation has not identified any resulting real-world harm. It also reports that some outcomes depended on human vigilance and not on a reliable technical barrier, that its monitoring was general-purpose and detected the activity after the fact, that internet access had been deliberately enabled, as was common practice in such evaluations, and that some task prompts were misconfigured so that the task seemed unsolvable without leaving scope. It will treat the decision to grant internet access as one that must be justified and not a default, and is introducing real-time monitoring that can flag or block out-of-scope actions (56, UK AISI, 2026).

These are evaluations of frontier systems and not enterprise emulation pipelines. The lesson holds at that level: an agent that can act in an environment can leave the scope its operators intended, and the control that holds is outside the agent. AISI's list of weak points reads as a checklist for any emulation runner: an access decision that was standard practice and not weighed against the risk, a monitor that saw the event afterwards, a task that made leaving scope the only route to success, and a stop that worked because people were watching. It is the sixth pattern.

Coverage figures from configuration

The fourth public case concerns what a coverage figure measures. In May 2022 the detection-engineering vendor CardinalOps published its second annual report on the state of detection risk in security information and event management systems. As VentureBeat reported it, the analysis covered configuration data from production SIEM instances at multinational corporations, used the nearly 200 techniques of MITRE ATT&CK (17, MITRE, n.d.) as the baseline, and found that the SIEMs lacked detections for about 80 per cent of the techniques (120, VB Staff, 2022). The article describes the method as an analysis of configuration and does not mention emulation testing. That is how I read it: the figure counts rules that are mapped to techniques, and the report is a vendor's. I draw no conclusion from the number, which is four years old and from a different population. The point is the kind of figure: a percentage derived from configuration says what has been mapped, which differs from what has been tested, and the loop's records exist to tell the two apart (Chapter 17). It is the third pattern, and its mirror image, a good figure nobody has tested, is the fourth.

Eight anti-patterns

The table assigns a record to each pattern, so that a loop review can find it in a repository. Nordvik Region keeps its records as AP-NR-LOOP-* and Hansa Mutual as AP-HM-LOOP-*. The identifier family is new in this book and follows the AP-NR-SOC-* and AP-HM-SOC-* records of AI Blue, Chapter 25.

#	Anti-pattern	Record	Organisation	Stage broken	Control that changes the outcome	Chapters
1	Purple theatre	AP-HM-LOOP-01	Hansa Mutual	P5 Verify	Closure rule, with implemented-unverified as a state	4, 10, 11
2	The red team that grades its own homework	AP-HM-LOOP-02	Hansa Mutual	P5 Verify	Independence statement, held-out scenarios, seeded gaps	15
3	Coverage by assertion	AP-NR-LOOP-02	Nordvik Region	P3 Observe	Coverage record with last test, result and expiry	5, 17
4	The green dashboard	AP-HM-LOOP-03	Hansa Mutual	P5 Verify	Cells that carry age; yield reported beside activity	17, 18
5	Overfitted detections	AP-HM-LOOP-04	Hansa Mutual	P4 Improve	Detection from behaviour, variants, negative control, held-out gap	8, 10, 11
6	The emulation that became an incident	AP-HM-LOOP-05	Hansa Mutual	P2 Emulate	Safety classes, scope as code, stop path outside the runner	6, 14
7	The findings register that leaked	AP-NR-LOOP-03	Nordvik Region	P3 Observe	Register of record, handling classes, references in tickets	16
8	The contract with no testing right	AP-NR-LOOP-01	Nordvik Region	P1 Target, P2 Emulate	Testing right, scope as code and third-party consent in the contract	6, 20, 22

Anti-pattern 1: purple theatre

The case (composite, illustrative). A public body holds a purple session every quarter. A tester and the analysts sit together, the tester runs a handful of techniques, the analysts watch what the tools show, there is a good conversation, and a summary goes to the steering committee: detections improved, three gaps noted. Two years later the same gaps appear in a threat-led test. The session ended with a list, the list went into a backlog, the owners of the items were not in the room, and nothing was run again.

The mechanism. The session was a debrief, and Chapter 4 anticipated this chapter's name for a session with no re-test: it is the stage P3 reading of the loop with the stages P4 and P5 left out. A good joint session is evidence of cooperation and not of improvement. Guidance for purple teaming under TIBER-EU, which the ECB issued in January 2025, makes the same demand from the other side: the outcomes of a purple test should be clearly documented and form an integral part of the remediation plan (34, ECB, 2025). A purple session is where the plan is made, and the plan is closed only by the re-test and a named person's signature (Chapter 4).

Early signs. Findings that carry a change identifier and no re-test log. A closure date that equals the merge date. Session records that list attendees and topics and no REG-* case. A steering pack that counts sessions held.

The record (illustrative, Hansa Mutual). When the group CISO's office mapped the five record states to the five stages (AIDR-0852, 2028-02-15), a white-team sample of 20 findings closed in the third quarter of 2027 found 14 with a re-test log and an evidence manifest and 6 closed when the change merged (Chapter 4). AP-HM-LOOP-01 is the record of those six. The control is the closure rule applied as a state: a merged change is implemented-unverified, and only a passing re-test with its evidence attached and a named person's signature makes it closed-verified (Chapter 11).

Anti-pattern 2: the red team that grades its own homework

The case (illustrative, Hansa Mutual). The scenario planner and the detection drafter of the group loop share a vendor's model family and read the same repository. Seven of the 25 verified priority techniques had both their scenario and their detection from that shared chain. The independence statement IND-HM-2028-Q1 (AIDR-0856, 2028-02-22) records what happened when the group re-tested those seven with held-out scenarios that people had written: five passed and two failed (FND-HM-2028-003 and FND-HM-2028-004), two cells of the coverage map moved to failed, and a person reproduced seven results from the evidence without the agents (Chapter 15).

The mechanism. A check is worth what it adds beyond the thing checked. When tester and tested share a model, training data, prompts or a vendor, their errors correlate. Published work reports that language models make correlated errors across models and providers (104, Kim et al., 2025), that models used as evaluators recognise and favour their own outputs (106, Panickssery et al., 2024) and that models often know when they are being evaluated (105, Needham et al., 2025). Each is a result about particular models under particular conditions, and together they are reasons to design for correlation and not to assume its absence. The problem also exists without any model: a detection engineer who has read the scenario can write the rule that passes it (Chapter 15).

Early signs. One vendor named across planner, drafter and critic. A scenario library and a rule repository that the same agent can read. A seeded-gap catch rate that has never been measured. A coverage map in which every cell is verified.

The control. The independence principle of Chapter 4 in its three layers (five kinds of separation: model, data, authorship, vendor and time; the incentive rule between roles; and three proofs: held-out scenarios, seeded gaps and errors, and reproduction by a person), a critic on a second model family, held-out scenarios written by a person and hidden from the drafter, seeded gaps with a catch-rate threshold, and reproduction by a person who was not part of the chain (Chapter 15). The re-test that exposed the two failures is the control working.

Anti-pattern 3: coverage by assertion

The case (illustrative, Nordvik Region). A draft of the pack for the audit committee's question (AIDR-0800) circulated on 2028-01-20 said that nine of nine priority techniques for the benefits-triage service were "mapped to a detection". Seven of the nine had something behind them: a result inside the 180-day expiry for five and the provider's runbook statement under MS-1 for two. The other two, including data encrypted for impact (T1486), had last been exercised in the spring 2027 mirror fieldwork, which ended on 2027-04-02, and their evidence had expired (Chapter 5). The statement was true of the mapping and not of the claim the committee had asked about.

The mechanism. "We detect technique T" is a statement with an age, a condition and a result. Untested is not covered, and evidence expires. Chapter 17 gives the coverage record its format; a count of techniques with a mapped rule is the easiest form to produce and the least informative.

Early signs. A coverage figure without a date. A map coloured by the existence of a rule. A claim for a technique that cannot run in production in a reversible form and has no mirror test. A figure that rises every month although nothing has been re-run.

The control. A coverage record per technique and environment with what was tested, when and with what result, and an expiry (180 days at Nordvik). Karin Lund restated the pack as five of nine priority techniques with a result inside the window, two asserted by the provider and two untested, which is the wording Chapter 5 records.

Anti-pattern 4: the green dashboard

The case (illustrative, Hansa Mutual). In the first quarter of 2028 the pipeline HM-SEC-004 exercised 31 of the 34 priority techniques within 30 days, and a dashboard showed 31 green cells. The same quarter, detections fired on 37 of 41 known-form scenarios and on 10 of 14 held-out variants (Chapter 8). The first figure measures activity and the second yield; a tile that counts runs says nothing about either unless it carries the result and its age. The coverage record COV-HM-0112 for the claims-triage agent expires on 2028-05-09, 90 days after the simulation of 2028-02-09, and a cell that turns amber on 2028-05-10 shows what the dashboard should do.

The mechanism. A dashboard is a view of records, and it is only as good as the age and the test behind each cell. A measure that becomes a target ceases to be a good measure (80, Strathern, 1997); Chapter 18 explains the mechanism, and a loop whose leaders ask for green cells will produce them.

Early signs. Percentages without dates. The count of runs rises while the count of findings is flat. Nobody can say which green cells would survive a re-run.

The control. Each cell carries its last passing test, its variant and its expiry, so that age turns the cell amber on its own. Activity and yield sit side by side, and the figure the supervisor sees is the gap between the pass rate on the open library and on the held-out set (Chapter 15).

Anti-pattern 5: overfitted detections

The case (illustrative, Hansa Mutual). A detection engineer reads a failing scenario and edits the rule until it passes, which is correct and is also how overfitting begins. In the first quarter of 2028 the pass rate was 90 per cent on the 41 scenarios of known form and 71 per cent on the 14 held-out variants, a gap of 19 points, and the eight failed executions merged into six findings (Chapter 8). The regression case REG-HM-0031 of Chapter 10 carries two variants for exactly this reason.

The mechanism. The test becomes the target. A rule drafted from the exact telemetry of an exercise may key on a feature of the scenario that a real adversary will not reproduce, and it passes the exact re-test for that reason. A model drafter makes a fitted rule cheap, and a pipeline that re-runs the same scenarios rewards the fit every night.

Early signs. An open-library pass rate that never falls. Rules that mention scenario markers, tool names or test accounts. Re-test cases without variants. A held-out set quietly folded into the library.

The control. Detection from a behaviour description, not from a log line; variants and a negative control in every regression case; a held-out share of the scenarios that the drafting side never sees and never tunes against; and the gap between open and held-out pass rates reported with each run (Chapters 8 and 11).

Anti-pattern 6: the emulation that became an incident

The case (composite, illustrative). An organisation runs a credential-guessing scenario, classed as reversible, against an integration account. It is inside the signed scope, rate-limited and tagged, and it runs on the morning of a monthly pay run. The account locks, the integration stops, and payroll is late. The stop path was held by the tester, who was off shift.

The mechanism. An emulation touches the estate, and its blast radius includes what the estate does in response. The signed scope said what the scenario might do and was silent on the calendar and on the target's dependencies. AISI's incident shows the same structure at a larger scale (56, UK AISI, 2026).

The record (illustrative, Hansa Mutual). AP-HM-LOOP-05 records the nearest event the group had. On 2028-01-24 the exercise EMU-HM-2028-03 against the SOC agent HM-SEC-001 found a restraint failure: the agent executed a session revocation on a service account during a change window (FND-HM-2028-001). Finding it was the purpose of the exercise, and it showed that a seed aimed at the defender's automation can reach a real account, because the automation acts. The level of the agent's session revocation was lowered to 2 the same day, the mandate MND-HM-SOC-02 was renewed at Level 2 on 2028-02-09, the re-test REG-HM-0032 passed on 2028-03-06, and Level 3 was restored on 2028-03-13 by the group AI governance board (AIDR-0863), after 49 days (Chapter 13). The first drill of the pipeline runner's stop path, in February 2028, took 38 seconds against a target of 60 (Chapter 6).

Early signs. Scope that lists targets and actions and not calendars or dependencies. A stop path held by the team that runs the emulation, or by someone not on call. Safety classes set by the scenario's author alone. No change-freeze check in the runner.

The control. Safety classes with a white-team decision before a scenario moves up a class, scope enforced as code by a policy decision point outside the runner, a stop path held outside the system it stops and drilled, and a calendar of change windows and business-critical runs that the manifest consults (Chapters 6 and 14). Level 4 is never used for an irreversible action.

Anti-pattern 7: the findings register that leaked

The case (composite, illustrative). A findings workbook is attached to a steering-committee mail, forwarded to a supplier's engineers so that they can fix three items, and stored on a shared drive that the supplier uses for several customers. No one acts in bad faith. The workbook is nonetheless a map of the techniques that were not detected, on which systems, with the accounts used.

The mechanism. The register is the asset most often handled carelessly, because it is a working document, and every copy has its own audience and its own lifetime. The FireEye case shows what a testing organisation's assets attract (60, INCIBE-CERT, 2020). Chapter 16 gives the handling classes FC-1 to FC-4 and the rule that a finding appears in a ticket as an identifier, technique, severity and owner.

The record (illustrative, Nordvik Region). Until AIDR-0803 (2028-02-08) the working copy of exercise findings lived in the managed provider's ticket queue, because remediation items were routed to the provider. AP-NR-LOOP-03 records that exposure. It is an exposure and not a leak: the record says no misuse has been found and makes no claim that none occurred, because the Region could not have seen it. The register of record now sits in the Region's own tenant, the provider's queue carries FC-2 references only, and the tester's access ends one working day after the engagement. The Region's records officer has also recorded beforehand that the register is security-sensitive under the member state's public-access law (Chapter 16), an organisational consequence and not legal advice.

Early signs. More copies than predicted when the inventory is taken. Findings pasted into a chat tool or a model prompt. A tester's access that outlives the engagement.

The control. A register of record, handling classes set at creation and declassified by event, references in tickets, expiring access and a monthly access review with an evidence record (Chapter 16).

Anti-pattern 8: the contract with no testing right

The case (illustrative, Nordvik Region). The Region buys most of what it would need to test. Its framework agreement with the managed provider, NR-2025-044, was silent about testing until annexes MS-1 and MS-2 gave it tagged-event outcomes within five working days and seeded incidents each quarter (AI Blue, Chapter 25, recorded the earlier silence as AP-NR-SOC-01). The service agreement for the staff identity provider, NR-2024-018 (illustrative), is also silent. The scenario for account discovery (T1087, SCN-NR-0002) therefore runs in the provider's test tenant, whose conditional-access settings differ from production. A result from the test tenant says what the test tenant does.

The mechanism. Leverage in a bought loop is the contract. A supplier owes the customer only what the paper says, and a test that crosses the supplier's boundary needs the supplier's consent. Microsoft, for one, publishes rules of engagement for customers' penetration tests of its cloud (79, Microsoft, 2026). The regulatory frame points the same way. DORA Article 30(3)(d) requires the contract with an ICT third-party service provider supporting critical or important functions to contain the provider's obligation to participate and fully cooperate in the financial entity's threat-led penetration test (2, European Union, 2022). NIS 2 requires essential and important entities to address supply-chain security in their risk-management measures (Article 21(2)(d)) and to have policies to assess the effectiveness of those measures (Article 21(2)(f)) (3, European Union, 2022). The architectural implication, and not legal advice, is that a measure that cannot be tested cannot be assessed. The Iowa case shows the other half: a right on paper that the signatory could not grant for the building tested is not a testing right.

Early signs. An agreement that says nothing about testing. A scope drawn around the pre-production mirror because production belongs to someone else. A provider's AI that closes alerts, with no right to see its decisions (the Mode 0 blind spot of the provider's AI).

The control. A testing right in the contract: scope, notice, hours, consent of the owner of each system, data handling for exercise telemetry, the provider's cooperation, and a declaration of any AI in use. In the Region's case it is a clause for the call-off and service agreements, tested by one tagged-event run in the production tenant (Chapters 20 and 22).

Practical guide: Recognising the early signs in a loop review

Purpose. Run a half-day loop review once a year and after any incident, and find which of the eight patterns the loop is closest to.

Roles. The purple-team coordinator chairs; the white-team lead attends, since the white team sits outside both red and blue; the detection owner, the red-team lead or the contracted tester, the SOC lead, a procurement representative and the records officer take part. An outside reader (internal audit at Nordvik, the CISO's office at Hansa) reads the evidence.

Steps.

1. Draw 20 findings closed in the last two quarters and ask for each re-test log and evidence manifest. Count those closed at merge (pattern 1). Above 10 per cent, open a record.
2. For five claims on the current coverage map, ask for the last test, the result and the expiry. Any claim without a date is an assertion (patterns 3 and 4).
3. List every agent and tool that plans, runs, drafts or checks, with vendor and model family. Where one family spans scenario and detection, ask for the held-out pass rate and the seeded-gap catch rate (pattern 2). Where the open and held-out pass rates differ by more than 10 points, treat it as overfitting (pattern 5).
4. For each class of scenario that runs in production, ask who holds the stop, how long a drill took and whether the runner consults the change calendar (pattern 6).
5. Inventory every place a finding lives and count the people with access (pattern 7).
6. For each system the loop must test, read the contract for a testing right, the owner's consent and the date of the last tagged-event run (pattern 8).
7. Record each pattern found as an AP-* record with an owner, a control, a decision and a re-test date.

The 10 per cent and 10-point thresholds are defaults for this book's illustrations; adapt them to the sample size, a supervisor's thresholds or the works agreement. Results about people are reported in aggregate only.

Definition of done. Each of the eight patterns has an entry stating found, not found or not testable, with the evidence read; each pattern found has a record, an owner and a re-test date no later than the next quarter; a decision record states who approved the controls; and a pattern is closed only when its re-test has passed.

Worked example: Anti-pattern records (illustrative)

The records live in the loop repository at loop/anti-patterns/. The first is Hansa Mutual's record for overfitted detections, the second Nordvik Region's for the contract without a testing right. The decisions and dates are illustrative; the references are the facts established in Chapters 8, 10 and 11 and in the Region's standing agreements.

anti_pattern:
  id: AP-HM-LOOP-04
  name: overfitted detections
  organisation: Hansa Mutual
  stages: [P4 Improve, P5 Verify]
  status: mitigated          # open | mitigated | closed
  decision: AIDR-0889
  recorded: 2028-04-14       # illustrative
pattern: >
  Detections are tuned until the scenarios that test them pass, so the
  pass rate measures fit to the scenarios and not detection of the technique.
signs:
  - open-library pass rate 37 of 41 (90 per cent), held-out 10 of 14 (71 per cent)
  - gap of 19 points in 2028-Q1
cause: rules edited against failing scenarios; held-out set not yet reported per run
evidence:
  - source: HM-SEC-004 run report 2028-Q1
  - failed_executions: 8, merged into 6 findings
control:
  drafting: from a behaviour description, not from the scenario log
  regression_case: variants and a negative control required
  reporting: gap between open and held-out pass rates in every run
  trigger: gap above 10 points in two consecutive runs opens a finding
owner: head-of-detection-engineering
retest:
  due: 2028-06-30
  closes_on: gap at or below 10 points in two consecutive monthly runs

anti_pattern:
  id: AP-NR-LOOP-01
  name: contract with no testing right
  organisation: Nordvik Region
  stages: [P1 Target, P2 Emulate]
  status: open
  decision: AIDR-0838
  recorded: 2028-04-12       # illustrative
pattern: >
  A supplier agreement says nothing about testing, so scenarios run in a
  test tenant and the result describes the test tenant.
evidence:
  - agreement: NR-2024-018 (staff identity provider), silent on testing
  - scenario: SCN-NR-0002 runs in the provider test tenant
  - earlier_record: AP-NR-SOC-01 (managed provider, annexes MS-1 and MS-2)
control:
  clause: testing right with scope, notice, hours, owner consent, data handling
  standard_for: call-off and service agreements from 2028-05-01
  consent: tagged-event run in the production tenant, white-team lead present
  ai_declaration: any AI used by the provider in the tested path is declared
owner: deputy-ciso
accountable_for_class: purple-team-coordinator
retest:
  due: 2028-07-31
  closes_on: signed amendment and one tagged-event result from the production tenant

Hansa Mutual's AP-HM-LOOP-01 to -05 and Nordvik Region's AP-NR-LOOP-01 to -03 follow the same schema.

What the cases say

None of the cases shows that AI made the loop unnecessary or unsafe in itself. Loops fail where a step was reasonable and a question was not asked: who may consent, what has been tested, who stops, who else could have written the same answer. In the public cases the missing element was authority that someone had verified (Iowa), an asset that someone had classified (FireEye), a boundary that was enforced outside the agent (AISI) or a figure whose method was visible (the coverage report). In the illustrative cases it was a re-test, an independent verifier, a date, a held-out set, a calendar, a classification or a clause. This is my own conclusion and carries the weight stated above. The eight patterns are also not independent. A green dashboard is easier to produce when the detections are overfitted, an overfitted detection survives when the verifier shares the drafter's blind spots, and a loop with no testing right cannot produce more than an assertion. The controls are likewise one family, defined in earlier chapters.

Across the usage modes

Each pattern is concentrated at some usage modes and delegation levels, which is why the delegation profile is the first document to read in a loop review.

Anti-pattern	Where it is most likely	Why
1 Purple theatre	Modes 1 to 3 around a human session	An assistant that summarises the session makes the debrief look like closure (Level 0 to 1)
2 Grading its own homework	Mode 4 planner and drafter; Mode 0 vendor AI on the tested side	Shared vendor or model family across a check
3 Coverage by assertion	Mode 0 simulation and SIEM tools; Mode 4 correlator	A tool's own map is read as a result
4 Green dashboard	Mode 4 and Mode 5 pipelines	Volume of runs hides the age and yield of results
5 Overfitted detections	Mode 4 and Mode 5 drafter at Level 1; re-test runner at Level 3	Cheap drafting meets nightly re-runs of the same scenarios
6 Emulation that became an incident	Mode 5 runner at Level 3	An agent with its own identity and offensive capability
7 Findings register that leaked	Modes 1 to 3 pasting into assistants; Mode 4 and 5 agents with register access	Many copies and many readers
8 No testing right	Mode 0 provider AI; Mode 5 emulation only under a signed mandate	The provider's decisions are outside the customer's reach

Observable, expected, speculation

The public cases are observable: they are documented in the sources cited and each is described only as far as its source goes. The composites and illustrative records are constructions and are labelled so. That each of the eight patterns has an analogue in public practice or in the earlier volumes is observable. That loops with AI on both sides will show the homework and overfitting patterns more often is expected, on the basis that published results on correlated errors and self-preference describe a mechanism, though no source I have read measures the effect on a purple loop. That a loop could be closed by machines alone without any of the eight appearing is speculation, which Chapter 25 considers and the independence principle of Chapter 15 is written to prevent.

Nordvik Region and Hansa Mutual

Nordvik Region buys the loop, and its anti-patterns concern what it buys and cannot yet make. AP-NR-LOOP-01 is the contract with no testing right, and the Region's remedy is paper and one tagged-event test: AIDR-0838 (2028-04-12) records the clause for the identity provider's agreement and for later call-offs, with a re-test due on 2028-07-31. AP-NR-LOOP-02 is the coverage claim that outran its evidence, and AP-NR-LOOP-03 the register that sat in the provider's queue. AIDR-0839 (2028-04-12) adopts the review of this chapter's practical guide as an annual practice and after any incident, with the checklist of Appendix B as its form. A team of six cannot run eight checks every quarter, so the review is a half-day chaired by Karin Lund with an outside reader from internal audit.

Hansa Mutual runs the loop, and its anti-patterns concern what its own pipeline and agents do at scale. AP-HM-LOOP-01 is the six closed at merge, -02 the shared chain of planner and drafter, -03 the dashboard that counts runs, -04 the 19-point gap and -05 the restraint failure and the lesson that a seed reaches what the defender's automation touches. Three decisions follow: AIDR-0888 (2028-04-14) adopts the review and the dashboard rule that a cell shows its age and turns amber at expiry; AIDR-0889 (2028-04-14) sets the held-out gap trigger of 10 points; and AIDR-0890 (2028-04-14) adds the check of the change-window calendar to the runner's manifest RoE-HM-2028-001. Hansa's risk is a loop that runs without anyone questioning it, so the CISO's office reads the evidence and the white-team lead signs. The agreement KBV-IT-2024-SOC-05 applies to the review: no result enters an appraisal and results are reported in aggregate only.

Summary

The purple loop fails in recognisable ways, and each way is a step that looked reasonable. This chapter set four public cases beside composites and illustrative records and named eight patterns, with a record for each. A contract that gives no right to test breaks P1 Target and P2 Emulate before anything runs, and an emulation that leaves its scope or touches what the defender's automation touches breaks P2 Emulate in production. A coverage claim that was never tested and a register that spreads to many copies break P3 Observe. A detection tuned until the scenarios pass breaks P4 Improve. A session without a re-test, a verifier that shares the model and the prompts of the thing it verifies, and a dashboard that hides the age of its evidence break P5 Verify.

The controls are few and the book's own: the closure rule, the independence principle, a coverage record with evidence age, scope as code with a stop path outside the system it stops, handling classes for the register, and a contract with a testing right. Nordvik Region meets the patterns that come from buying a loop and Hansa Mutual those that come from running one, and both put the review in the calendar with an outside reader and a re-test date for every record.

Chapter 27 follows two organisations through the loop's year, from May 2027 to April 2028, with a failure and a recovery, and shows how these patterns look from inside the work of four people.

Sources

1. Iowa Judicial Branch, State Court Administration (2019). State Court Administration statement. (59, Iowa Judicial Branch & State Court Administration, 2019)
2. Kirk, J. (2020). Iowa Drops Trespassing Charges Against Penetration Testers. BankInfoSecurity. (119, Kirk, 2020)
3. INCIBE-CERT (2020). FireEye Red Team tools have been stolen. (60, INCIBE-CERT, 2020)
4. VB Staff (2022). Report: 80% of cyberattack techniques evade detection by SIEMs. VentureBeat. (120, VB Staff, 2022)
5. AI Security Institute (2026). Incident Report: unsanctioned agent behaviour during cyber testing. (56, UK AISI, 2026)
6. OpenAI (2026). OpenAI – Hugging Face Incident Technical Report. (72, OpenAI, 2026)
7. Larcher, H., Carreira, A. et al. (2026). Anatomy of a Frontier Lab Agent Intrusion. (73, Larcher et al., 2026)
8. Wijk, H., Cotra, A. & Greenblatt, R. (2026). Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident. (74, Wijk et al., 2026)
9. MITRE Corporation. MITRE ATT&CK. (17, MITRE, n.d.)
10. European Central Bank (2025). TIBER-EU Purple Teaming Guidance. (34, ECB, 2025)
11. Kim, E., Garg, A., Peng, K. & Garg, N. (2025). Correlated Errors in Large Language Models. (104, Kim et al., 2025)
12. Panickssery, A., Bowman, S. R. & Feng, S. (2024). LLM Evaluators Recognize and Favor Their Own Generations. (106, Panickssery et al., 2024)
13. Needham, J. et al. (2025). Large Language Models Often Know When They Are Being Evaluated. (105, Needham et al., 2025)
14. Strathern, M. (1997). 'Improving ratings': audit in the British University system. (80, Strathern, 1997)
15. Microsoft. Penetration testing (Azure security fundamentals) and the Microsoft Cloud Unified Penetration Testing Rules of Engagement. (79, Microsoft, 2026)
16. European Parliament and Council of the European Union. Regulation (EU) 2022/2554 (DORA). (2, European Union, 2022)
17. European Parliament and Council of the European Union. Directive (EU) 2022/2555 (NIS 2). (3, European Union, 2022)
18. Nordqvist, G. (2025). Architecture as Code. (87, Nordqvist, 2025)
19. Nordqvist, G. (2026). AI Governance. (88, Nordqvist, 2026)
20. Nordqvist, G. (2026). AI Threat. (91, Nordqvist, 2026)
21. Nordqvist, G. (2026). AI Blue. (94, Nordqvist, 2026)
22. Bainbridge, L. (1983). Ironies of Automation. (100, Bainbridge, 1983)