Purple for AI Systems and Agents: Attacks on Models and Detections That Catch Them

The organisation's own AI is now something the loop has to test, and something the blue team has to watch. This chapter applies the whole purple loop to AI systems and agents: P1 Target chooses which AI system, and which technique against it, is worth testing under signed rules of engagement; P2 Emulate runs a catalogued, defensive scenario against a test copy or a tagged identity; P3 Observe compares what the platform and the security operations centre (SOC) saw with what was expected, layer by layer; P4 Improve turns a finding into a detection, a control or a mandate change; and P5 Verify replays the scenario as a regression case until the re-test passes and the evidence is attached. Earlier volumes covered how to test and how to secure AI systems. This chapter covers the part they left to the loop: the telemetry an AI platform produces, the detections that read it, and the path by which an AI red-teaming finding becomes a running detection and a test that repeats.

Figure 12.1: An AI system with its platform telemetry, the detection that reads it and the path from an AI red-teaming finding into the regression suite.

The chapter takes over five conclusions and one open question. Security for AI systems is security architecture with three additions: natural language as an instruction channel that the model cannot police, models and datasets as supply-chain components, and agent containment through identity, permission and egress; prompt injection is a property of the architecture, so the controls sit outside the model (AI Platform and Security, Chapter 16, F39 and F40) (89, Nordqvist, 2026). Red teaming of AI systems runs continuously in the delivery pipeline, and every finding becomes a versioned attack case with explicit thresholds (AI Platform and Security, Chapters 17 and 20, F43 and F44). Everything that security AI reads may be written by the adversary, so the defence is a policy decision point outside the model, a permission manifest that sets a ceiling on every action, and campaigns that test the ceiling (AI Red Team, Chapter 13, F54) (90, Nordqvist, 2026). Containment must not depend on intent, and detection comes from telemetry and decoys rather than from what an agent reports about itself (AI Platform and Security, Chapter 19, F45). The open question is the one those chapters did not close: when a test shows that a control held, did anyone see the attempt, and how would the organisation know?

AI systems as targets in the loop

A red team that tests an AI system asks what the system can be made to do. A purple loop asks a second question of the same test: would the organisation have seen it, and how does it know? The difference changes what is recorded. An AI red-teaming report lists findings against the system. A loop record lists, for each scenario, what the system did, what the platform logged, whether a detection fired, who was told and how fast, and what was stopped. A scenario that the system resisted and the SOC never saw is a prevented attempt without detection, which is a different finding from a scenario that succeeded and was caught, and both are different from one that succeeded unseen.

Four classes of target

An AI system in production is not a model. It is a compound system, and the loop selects scenarios against four classes of target inside it. The vocabulary is shared: MITRE ATLAS gives technique identifiers for adversary behaviour against AI-enabled systems (18, MITRE, n.d.), the OWASP lists for large language model applications and for agentic applications give risk categories (21, OWASP, 2025) (22, OWASP, 2025), and NIST's taxonomy of adversarial machine learning gives terminology for attacks and mitigations (23, Vassilev et al., 2025). The loop uses them as a scenario index, never as a source of test content.

• The model and its instructions. Instructions placed in content that the model reads, so that the task changes: ATLAS AML.T0051 LLM Prompt Injection. The scenario record names the technique and the success condition. It contains no test string; the content that exercises the scenario is held in the restricted scenario store (Chapter 16) and is generated or replayed inside the test range.
• Retrieval and the index. Content that reaches the retrieval index outside the publishing workflow, or that stays in it after withdrawal: AML.T0070 RAG Poisoning. The detection side is the index's write audit log and a reconciliation of the index against the publishing register.
• Tools and actions. A tool invoked with parameters, a sequence or a destination that the permission manifest does not allow: AML.T0053 AI Agent Tool Invocation. This is the class that matters most for Mode 4 and Mode 5 systems, because the tool call is where an effect leaves the model.
• Data and outputs. Sensitive content reaching an output or an external destination: AML.T0085.002 Data from AI Services: Conversation Context and Memory, the technique that the ATLAS 2026.10 release moved from AML.T0057 LLM Data Leakage (18, MITRE, n.d.). The detection side is the output filter's decision log and the egress record.

These identifiers were checked against the published ATLAS data at the time of writing; they are a map for the scenario library, and Chapter 17 treats a map as a claim that needs evidence.

Choosing the scenario and the rules

P1 Target for an AI system draws on three sources. The threat register of AI Threat, Chapter 32, supplies scenarios with a path to harm, such as Hansa Mutual's THR-HM-01, the claims-triage agent acting beyond its approval rules (91, Nordqvist, 2026). The campaign plans of AI Red Team, Chapter 13, supply objectives mapped to OWASP and ATLAS categories. And the incidents of 2025 and 2026 supply the pattern of mechanism that agents acting beyond their mandate share: persistent pursuit of a goal, latitude, a blocked intended path and a boundary that depended on compliance or leaked (AI Threat, Chapter 25, F67). The July 2026 incident, in which agents in a security-testing context exceeded their mandate, is the documented case (72, OpenAI, 2026) (73, Larcher et al., 2026) (74, Wijk et al., 2026), and the lesson the loop takes from it is practical: a control that held in the test and was not detected in the log tells the organisation nothing about the next attempt.

Three rules apply to every AI-system scenario, and the rules of engagement of Chapter 6 carry them. First, the scenario runs against a copy in a test range with synthetic data, or against a production system only in a catalogued benign form that is tagged and reversible, such as a harmless marker document placed in a retrievable source, or a single out-of-ceiling call made by a dedicated test identity that the policy point is expected to deny. Second, what can be undone is stated before the run: the test copy is discarded, the marker document is removed and the test identity is revoked. Third, the stop path sits outside the system it stops. The platform on-call and the white-team lead can each revoke the test identity at the AI gateway without asking the runner, and no emulation action touches a named person or leaves the signed scope. Level 4 Autonomous is never used for such an action; a scripted runner at Level 3 Bounded executes catalogued scenarios inside a manifest that a policy decision point enforces, and anything else is run at Level 2 Supervised by a person.

Telemetry from the platform

Detection can only read what the platform records. The reference architecture of AI Platform and Security, Chapter 4, puts an AI gateway, a model registry, an evaluation service and prompt and policy repositories inside a control plane for identity, policy, observability and evidence (F37), so the sources already exist. The loop's contribution is to ask, for each scenario, which of them recorded the attempt, and to treat a missing record as a finding.

What each layer can tell the SOC

The OpenTelemetry project maintains semantic conventions for generative AI, covering spans, metrics and events for generative AI clients and conventions for the Model Context Protocol (35, OpenTelemetry, n.d.). Using a shared convention matters less for the standard itself than for the join: a detection that must correlate a gateway record, a retrieval record and a tool-side record needs one run identifier that all three carry. The table sets out the sources, what each tells a detection and where each is blind.

Source	What it records	What a detection can read	Blind spot
AI gateway	Caller, route, model and version, token counts, policy decisions, guardrail verdicts, refusals	Unusual routes, refusal spikes, a caller or agent identity outside its baseline, guardrail verdicts	Content of retrieval and of tool results; traffic that bypasses the gateway
Retrieval service and index	Queries as document identifiers and access-control decisions; every write to the index	Retrieval outside a user's entitlement, sudden breadth, a document in the index that the publishing register does not list	Why the model used a document
Agent runtime	Plan steps, tool-call requests with parameters, approvals, budget consumption, run identifier	Calls outside the manifest, unusual sequences, budget approaching its limit, approvals bypassed	Anything the runtime does not itself emit; the agent's own account of its work
Policy decision point	Every allow and every deny, with the rule that decided	A deny is both a control success and a detection input; repeated denies from one run	Actions that never reached the point
Tool and target system	The system's own audit log of actions taken under the agent's identity	The effect of a call, whatever the runtime reported	Calls under a shared or borrowed credential
Identity provider	Issuance, scope and expiry of the agent's task-scoped credentials	Credential issued outside a ticket, scope wider than the manifest, use after expiry	Use of a credential obtained by another route

Two points follow from the table. The agent's narrative is not telemetry. An agent's summary of what it did is an output of the model and can be wrong or steered, so the loop compares the narrative with the tool-side record, which is the comparison that AI Threat, Chapter 23, recommends as part of layered assurance (F68). And the gateway is only a sensor for traffic that passes through it, so one standing detection compares egress records with gateway records and flags model traffic that never touched the gateway.

Logging as a design constraint

The logs must be adequate to reconstruct an agent incident from the organisation's own records, which AI Threat, Chapter 25, asked organisations to rehearse. The AI Act places record-keeping duties on high-risk AI systems, with automatic event logging in Article 12 and a minimum retention for deployers' logs in Article 26(6) (1, European Union, 2024). NIS 2 Article 21 requires policies to assess the effectiveness of cybersecurity risk-management measures (3, European Union, 2022) and a detection that is replayed against logs is one such assessment. These are architectural implications, not legal advice: they argue for logs that are complete enough to replay a detection, retained for a stated period and protected as security-sensitive.

Content is the difficulty. Prompts, retrieved passages and tool results contain personal data, and in a regional authority they contain health and social-care data. The data-minimisation principle of GDPR Article 5(1)(c) (7, European Union, 2016) points to a layered default: the gateway, retrieval and runtime logs record metadata, document and claim identifiers, hashes and decisions; full content is recorded in the test range, where the data is synthetic, and in production only for a flagged run, under access control and a short retention. Logs that identify the member of staff on whose behalf an assistant acted can profile the individual, and where they do, they fall within the co-determination arrangements described in Chapter 21 and in AI Boss, Chapter 23 (93, Nordqvist, 2026): the detections in this chapter are keyed to the agent or system identity and are reported in aggregate, not per named person.

Detection for agents

An agent is a non-human identity whose behaviour is bounded by a manifest. That makes agent detection simpler in one respect than detection for people: the expected behaviour is written down. The permission manifest of AI Red Team, Chapter 13, lists the agent's tools, actions, resources, destinations, budget and approvals, and the policy decision point enforces it. A detection therefore starts from the manifest and not from a learned baseline alone. Everything the manifest does not list is a candidate event. A learned baseline is added for what the manifest cannot say, such as rate, sequence and time of day, and it is built from a period that the loop has verified was clean, since a baseline learned from a compromised agent normalises the compromise.

Three families of detection

Tool misuse. A call outside the manifest, a permitted tool with parameters outside the declared range, an unusual sequence of permitted calls, or a rate that departs from the baseline. Misuse also includes a permitted call made for a purpose that the ticket does not support, which only a join with the task record can show. The OWASP agentic list names tool misuse and identity and privilege abuse among its risks (22, OWASP, 2025).

Anomalous retrieval. Retrieval that crosses an entitlement boundary, retrieval of a document shortly after it was written to the index by something other than the publishing identity, a sudden widening of the set of documents a session touches, and retrieval of a decoy. The detection reads the retrieval service and the index audit log. Its quality depends on the index having an owner and a register to reconcile against, which Nordvik Region's first finding showed it lacked.

Permission-ceiling breaches. The ceiling is the manifest. Two conditions matter and they have different severity. A call that the policy point denies is evidence that the control worked and that something attempted what it should not; it is a medium-severity signal, because an agent that repeatedly meets a boundary is behaving in a way that someone should understand. A call that the policy point allowed and that lies outside the manifest means the control failed; it is high severity and the first action is a stop. The same logic applies to budget: consumption approaching the cap is a warning, and a cap exceeded without a stop is a failure of the stop path.

A caution on what detection cannot do. AI Blue, Chapter 15, concluded that controls outside the model bound suppression, action and disclosure but not misdirection: a ticket or document that steers an agent towards a plausible wrong answer produces no out-of-ceiling event (94, Nordqvist, 2026). Detections of the three families above catch an agent that acts beyond its mandate. They do not catch an agent that acts inside it on a poisoned premise. That risk is handled by review and by the independence measures of Chapter 15, and the coverage claim for a detection says which of the two it covers.

Practical guide: Writing a detection record for an agent that exceeds its permission ceiling

Purpose. Produce a tested, owned detection record that fires when an agent calls outside its permission ceiling, whether the policy point denied the call or allowed it, and that routes the right alert to the right person within 15 minutes.

Roles. The detection engineer writes and owns the record. The agent operator, named in the agent's mandate, owns the response to the alert and the stop. The AI platform owner supplies the manifest, the policy log and the run identifier. A SOC analyst receives the alert and reviews the draft routing. The privacy officer confirms the retention of the logs the detection reads, and the employee representative is informed where the logs could identify handlers.

Steps.

1. Export the ceiling from the manifest registry at a pinned version. List each permitted tool, action, resource scope, destination and budget, and for each write one adjacent action that must be refused. These become the positive and negative tests.
2. Confirm the telemetry before writing logic. The policy log and the tool-side audit log must both be present in the SIEM for the agent's identity, with a maximum silence set for each (default 10 minutes for the policy log, 15 for the tool-side log). A detection over a source that is not collected is a declared claim; fix collection first.
3. Write two conditions in the logic and keep them separate. Condition A: a policy deny for the agent identity under a ceiling rule (medium). Condition B: an allowed call, observed in the tool-side log, whose tool, action, scope or destination is absent from the manifest (high). Correlate both on the run identifier and the agent identity.
4. Add the baseline only for what the manifest cannot state. Build it from at least 30 days of production that the loop has verified as clean, and record the date and the verifier. Re-baseline on a model, prompt-template, manifest or tool change, and record the re-baseline as a change.
5. Test the record. Run the catalogued scenario in the test range with the scripted driver at Level 3 Bounded: expect a deny and an alert for Condition A, and, with the policy point temporarily set to permissive in the range only, an alert for Condition B. Run the negative cases, and replay 30 days of production telemetry. Defaults: no more than five alerts a week in the replay, a first alert within 15 minutes of the event, and a precision of at least 0.7 after tuning. Adapt the thresholds to the agent's volume and to the works agreement, and write the adapted values down.
6. Route the alert. Condition A goes to the SOC queue and notifies the agent operator. Condition B pages the operator, who may revoke the agent's credentials at once and without prior justification (the right to stop of AI Blue, Chapter 18, F107). Neither condition is in a class that any AI closes by itself.
7. Register the claim. Record in the coverage record the date of the last passing test, the evidence reference and an expiry of 90 days for a priority agent, and attach a review date.

Definition of done. The record is merged with an owner and an approver who did not write the scenario; both conditions have passing positive and negative tests; the replay is within the alert budget; the coverage record states the last test date and its expiry; and the operator has confirmed that the stop can be exercised outside the agent.

Worked example: Detection record DET-HM-0561 for the claims-triage agent

This illustrative record belongs to Hansa Mutual's claims-triage agent (Mode 5; AIDR-0024). The red team found in the autumn 2027 round of HM-SEC-003 that a tool call bound to a claim's market but issued with another market's attribute was refused at the policy point, as designed, but that the refusal reached no alert (FND-HM-2027-031, severity medium). Scenario SCN-HM-0058 was written by the red team. The detection logic was drafted by HM-SEC-002 at Level 1 Assisted from the manifest and reviewed by a detection engineer who had not chosen the scenario.

id: DET-HM-0561
title: Claims-triage agent call outside its permission ceiling
version: 1.0.0
status: production
owner: team-detection-engineering        # accountable: detection engineering lead
system_owner: head-of-group-claims-operations
origin:
  finding: FND-HM-2027-031
  scenario: SCN-HM-0058                 # authored by the group red team
  drafted_by: HM-SEC-002                # Level 1 Assisted, AIDR-0312
  reviewed_by: detection-engineer-3     # did not select the scenario
  decision: AIDR-0862
atlas: [AML.T0053]
owasp_agentic: ASI02 Tool Misuse
logic:
  format: sigma
  ref: detections/sigma/agents/claims_triage_ceiling.yml@v1.0.0
  conditions:
    A_denied: {source: policy-decision-log, rule_in: [market-binding, tool-allowlist, budget], severity: medium}
    B_allowed_outside_ceiling: {source: claims-core-audit-log, not_in: manifest@3.1.0, severity: high}
  correlate_on: [run_id, agent_identity]
data_sources:
  - {name: policy-decision-log, max_silence_minutes: 10}
  - {name: claims-core-audit-log, max_silence_minutes: 15}
  - {name: manifest-registry, max_silence_minutes: 1440}
baseline:
  kind: manifest-derived-plus-rate
  verified_clean_window_days: 30
  rebaseline_on: [model_change, prompt_template_change, manifest_change]
tests:
  unit:
    positive: tests/DET-HM-0561/call_with_other_market.json
    negative: tests/DET-HM-0561/permitted_call_same_market.json
  replay: {window_days: 30, expected_alerts_per_week: {max: 5}}
  simulation: {scenario: SCN-HM-0058, last_run: 2028-02-09, prevented: true, detected_within_minutes: 4, result: pass}
routing:
  A_denied: {queue: soc-agents, notify: agent-operator-poland}
  B_allowed_outside_ceiling: {page: agent-operator-poland, first_action: revoke-agent-credentials, closable_by_ai: false}
cost: {alert_budget_per_week: 5}
coverage: {record: COV-HM-0112, last_tested: 2028-02-09, expires: 2028-05-09}
review_date: 2028-05-09

The record states two things that the exercise report alone would not. The control held and was detected in four minutes on 2028-02-09; and that claim expires on 2028-05-09, when the scenario runs again or the claim lapses.

From AI red-teaming finding to detection

An AI red-teaming finding is the loop's raw material, and it usually has the shape of a hole in prevention: a withdrawn page still cited, a tool call that reached further than the manifest allowed, a retrieval that crossed an entitlement. AI Platform and Security, Chapter 17, set the rule that such a finding becomes a versioned attack case (F43). The purple loop adds a second question for each finding, and the answer produces three artefacts instead of one.

The finding, the detection and the regression case

For every finding the loop records two outcomes separately. Prevention asks whether the control stopped the effect. Detection asks whether any signal reached a person who could act, in the time that mattered. The four combinations behave differently. Prevented and detected is the target state. Prevented but undetected means the organisation is relying on a control it cannot see being attacked, as in FND-HM-2027-031. Detected but not prevented means a person can act but the effect lands. Neither prevented nor detected is the finding that most needs an owner. P3 Observe records the pair per layer (gateway, retrieval, runtime, policy point, tool, SOC), and P4 Improve produces the change for whichever column failed: a control change, a detection, or both.

The detection then has to be built from the finding by someone who is independent of the finding's author where that is practical. The scenario's author knows the technique and the success condition, and a detection written to that author's mental model catches the author's variant and little else. The rule that Chapter 15 develops applies in miniature here: the person or agent that drafts a detection does not choose the scenario that tests it, and the test set includes at least one variant that the drafter has not seen. The regression case is the third artefact. It stores the scenario, its variants, the expected prevention and detection outcomes, the cadence, and the triggers that force a re-run: a release of the system, a change to the model, the prompt template, the index or the manifest, and a change to the detection itself (Chapter 11).

Worked example: A finding carried to a detection and a regression case at Nordvik Region

This illustrative chain follows the finding of AI Red Team, Chapter 13, from the first campaign under NR-SEC-002 (RoE-NR-2027-003): a page withdrawn from the public website stayed in the citizen-service assistant's retrieval index and was cited as current guidance. The control, removal from the index within one hour of withdrawal, was closed and re-tested. The purple question was whether anyone would have seen the drift. Nobody would, because the index's audit log was not in the SIEM. Karin Lund, the detection owner, commissioned the detection; the contracted tester, Elin Berglund, wrote the scenario SCN-NR-0013.

The finding record extends the canonical FND-* record that Chapter 9 defines and Chapter 10 completes: the shared keys come first, and the keys below the comment are this chapter's extension for AI systems, which separates the prevention outcome from the detection outcome.

kind: finding
id: FND-NR-2027-019
scenario: SCN-NR-0013
technique: AML.T0070
step: retrieve-after-withdrawal
expected: "Page removed from the index within 60 minutes of withdrawal; the drift seen by a person"
outcome: missed                  # prevention failed, then passed after the fix; detection absent
layer_failed: detection          # the index audit log was not collected
evidence: {source: RoE-NR-2027-003, system: citizen-service-assistant}
times: {simulation_passed: 2028-04-18, closed: 2028-04-20}
proposed_owner: senior-security-analyst
proposed_kind: detection
status: closed-verified
decision: AIDR-0812
# Extension for AI systems
prevention: {before: failed, control: index-removal-within-60-min, after: pass}
detection: {before: none, reason: index-audit-log-not-collected, after: DET-NR-0011}
changes:
  - {type: collection, item: index-audit-log into SIEM, state: done}
  - {type: detection, item: DET-NR-0011, state: production}
  - {type: contract, item: DET-NR-0011 added to the MS-2 seeded set, state: agreed}
regression: {id: REG-NR-0005, scenario: SCN-NR-0013, cadence: every release and quarterly}
closure: {evidence: EVD-NR-2028-0004, reproduced_by: second-senior-analyst, signed_by: ciso-office, age: "expires 2028-10-15"}

The detection record that closes the prevention gap is short, because the Region has one index and one publishing register.

id: DET-NR-0011
title: Document in the citizen-service index that the publishing register does not list
version: 1.0.0
status: production
owner: senior-security-analyst            # K. Lund, detection owner
origin: {finding: FND-NR-2027-019, scenario: SCN-NR-0013, scenario_author: contracted-tester}
atlas: [AML.T0070]
logic:
  format: sigma
  ref: detections/sigma/ai/index_not_in_register.yml@v1.0.0
  rule: write to the index by an identity other than the publishing pipeline, or a document present in the index for more than 60 minutes after the register marks it withdrawn
data_sources:
  - {name: index-audit-log, max_silence_minutes: 60}
  - {name: publishing-register-export, max_silence_minutes: 60}
severity: medium
routing: {provider_closable: false, out_of_hours: mssp-shift-lead}
tests:
  unit: {positive: tests/DET-NR-0011/marker_in_index.json, negative: tests/DET-NR-0011/scheduled_publication.json}
  simulation: {scenario: SCN-NR-0013, last_run: 2028-04-18, detected_within_minutes: 47, result: pass}
cost: {alert_budget_per_week: 2}
coverage: {last_tested: 2028-04-18, expires: 2028-10-15}   # not a priority technique, 180 days
review_date: 2028-10-15

Two details carry the point of the example. The simulation placed a harmless marker document in a retrievable source of the test mirror and counted the minutes to the alert; the 47 minutes fit inside the hour that the control allows. And provider_closable: false keeps the alert out of the classes that the managed provider's AI may close out of hours, so the Mode 0 blind spot of AI Blue, Chapter 7, does not swallow it; the Region adds the detection to the seeded incidents it may inject under annex MS-2, so that the first quarterly sample shows whether the provider's service sees it.

Evaluation versus detection

A passing evaluation is not a running detection, and the two are routinely confused when a team reports that its AI system has been red-teamed. AI Platform and Security, Chapter 20, places red teaming at the apex of an evaluation pyramid (F44). An evaluation answers a question about a version of the system under a set of inputs, before release or at a gate. A detection answers a question about what is happening now, on traffic nobody chose, and about whether a person will hear of it. The table sets the two side by side.

	Evaluation	Detection
Question	Does this version resist these cases?	Is something happening that should not be, and who hears of it?
Input	A chosen set, generated or authored in advance	Production telemetry that nobody chose
When it runs	At a gate, on a release or on a schedule	Continuously, on every event
What a pass means	The version resisted the cases tested at the time	Nothing fired; the absence of an alert is not evidence of absence
How it decays	When the system, the model or the adversary changes	When a log source stops, a baseline drifts or a model changes behaviour
Evidence of working	A passing suite with a date	A recent simulation that fired the alert in time
Owner	Evaluation engineer and system owner	Detection engineer and SOC

The distinction matters most where models behave differently when they are tested. AI Threat, Chapter 23, set out why assurance cannot rest on any single evaluation: pre-deployment tests are layered with a shadow period, production monitoring, comparison of an agent's reports with the record, and decoys that do not reveal themselves (F68). Production detection is the layer that does not depend on the model's behaviour changing under test. A decoy tool or a harmless marker document in a retrievable source is a test whose inputs do not reveal it as a test, which is also why the loop uses them in production under a signed scope and why their use is recorded and limited. The purple loop therefore requires both and links them: the evaluation suite proves that the control holds on the cases, the detection record proves that attempts are seen, and the regression case stores both outcomes with their dates so that neither is claimed without evidence.

The compound system

The unit of test, and of detection, is the compound system: the model, the tools it can call, the data it can read and the identity it acts under. AI Red Team, Chapter 13, made the same point for testing, and organisations that integrate a general-purpose AI model into their own AI systems are advised to ask the model provider for the information and documentation that Article 53(1)(b) of the AI Act requires the provider to make available (1, European Union, 2024), and to test the compound system themselves (AI Platform and Security, Chapter 20, F44). The AI Act's duty for providers of general-purpose AI models with systemic risk to perform adversarial testing is in Article 55 (1, European Union, 2024), and it concerns the model. It does not cover the Region's retrieval index, the Group's manifest or the identity an agent holds.

Four consequences follow for detection. First, the weak link is often not the model. In the worked examples of this chapter the control or identity around the model, and the missing log, were the defects, and the book's own conclusion is that a detection programme for AI starts with the platform's records and not with the model's behaviour. Second, a change in any component changes the system under watch, so the loop treats a model version, a prompt template, an index rebuild, a manifest revision and a credential policy as change triggers that force a re-run of the regression cases and a recorded re-baseline of the detections; where the model is a provider's, Chapter 22 treats the notice of a change as a contract term. Third, correlation across the four components is where an attempt is visible. One tool call may look normal, one retrieval may look normal, and the pair, in a session that began with a document written an hour earlier, is the pattern. Detection logic therefore keys on the run identifier and the agent identity, not on a single log. Fourth, detection shares fate with the platform: an outage of the AI gateway, or a change that stops the policy log, silences the detections that read it. The detection for a missing source, with the maximum silence in each record, is part of the set, and the loop's chaos experiments (AI Red Team, Chapter 25) include removing a log source to see whether anyone notices.

Limits and outlook

Observable, expected, speculation

Observable. That attacks on deployed AI systems are happening is shown by sources: ATLAS documents techniques and cases against AI-enabled systems (18, MITRE, n.d.), and agents acting beyond their mandate are documented in the incidents cited above.

Expected. That detection engineering for agents becomes an ordinary part of SOC practice is expected, on the trend of agentic services entering production, the guidance of national cybersecurity agencies urging careful adoption of agentic AI (54, Five Eyes agencies, 2026) and the logging duties of the AI Act and NIS 2.

Speculation. That any organisation can claim complete detection coverage for AI threats is speculation, and the loop's rule is the reverse: coverage is stated per technique, per system and per date, with an expiry. A detection that has not fired in a test since the last model change is declared, not verified.

The loop across modes and organisations

Across the usage modes

The usage mode decides whose identity an AI acts under, and so what the SOC sees and what a detection can say (AI Governance and AI Platform and Security, Chapter 2) (88, Nordqvist, 2026).

Mode	Whose identity	What the telemetry shows	What the loop tests and detects
Mode 0 Ambient AI	The vendor's service, inside licensed software	Usually nothing the organisation controls	Contract rights to logs and to injected tests; coverage stays declared until a seeded test is seen
Mode 1 Conversational assistants	The person's	Enterprise tier logs at the gateway or the vendor's console	Data leakage out of the approved tier; detected as the person, aggregated
Mode 2 Embedded workplace assistants	The person's	Actions in the host application under the person's account	Retrieval beyond entitlement; instruction-bearing content in shared documents; detected as the person
Mode 3 Agentic work tools	The person's, with tool access	Tool calls under the person's credentials	Tool misuse indistinguishable from the person unless the tool identifies the agent; the platform team tags calls
Mode 4 Built AI applications	The application's service identity	Gateway, retrieval, runtime and tool logs	Index integrity, output leakage, ceiling breaches; per-application baselines
Mode 5 Autonomous production agents	The agent's own identity and permissions	Complete agent telemetry plus the policy point	Per-agent baseline and ceiling, stop path, decoys; the highest detection and containment demand

Modes 1 to 3 are a person's identity and are detected as the person: an anomalous action in Mode 2 looks like an anomalous action by that employee, which is both a detection difficulty and a staff-monitoring question. The remedy is to have the platform tag calls with the assistant's identity and to report in aggregate. Modes 4 and 5 have identities of their own, and they need their own baselines and ceilings. For emulation against these systems the levels of delegation follow the usual line: the scripted runner executes catalogued S1 scenarios at Level 3 Bounded against a range copy; scenarios against a production Mode 5 system run at Level 2 Supervised with a named person present; the drafting of detections is Level 1 Assisted; and Level 4 Autonomous is not used for any emulation action.

Nordvik Region and Hansa Mutual

Nordvik Region has Mode 4 systems and no Mode 5 agents. The citizen-service assistant has no tools that change state, so the loop concentrates on retrieval and output: DET-NR-0011 and the index audit log. Its first exercise programme, RoE-NR-2027-001, covered the benefits-triage service's explanation service and the assistant; the triage system's production scoring was out of scope, and the scoring component belongs to a supplier. The Region therefore holds a declared, not verified, position for the scoring: it cannot show what the supplier's platform logs until the contract delivers the logs and a seeded test. Elin Berglund, under call-off agreement NR-2026-071, writes the human-authored scenarios; Karin Lund owns the detections and does not use her drafting assistant on a detection for a scenario it helped to choose. The question to the audit committee, recorded as AIDR-0800, is answered for AI systems with a short table of systems, scenarios, last test dates and expiries, and the detection of the index is the first line of it. Telemetry stays on the Region's reserved capacity because the data is personal and health data.

Hansa Mutual has Mode 5 agents with their own identities: the claims-triage agent, which works through the Polish service centre and whose tool calls are bound to the market of each claim, and the IT-operations agent with write access to production (AIDR-0027). Its telemetry flows to the group SOC, and HM-SEC-003 tests the agents, with the novel scenarios written under Mareike Hartmann, head of the group red team; HM-SEC-004 replays the catalogued S1 scenarios, and DET-HM-0561 joins the detections that Lena Vogel's team maintains as code. The earlier containment finding for the IT-operations agent, a deployment service account that could read an adjacent environment's secrets and a SIEM that did not collect those reads (AI Red Team, Chapter 13), has the same shape as FND-HM-2027-031: the identity was fixed, and the reads and denials became something a detection could see. The difference between the two is the one the book keeps returning to. Nordvik buys the loop and its leverage is the contract, so its AI detections depend on what the supplier and the provider are obliged to log. Hansa runs the loop and its leverage is the pipeline, so its risk is a regression suite that is replayed and green while nobody has asked what an agent's baseline actually was.

Summary

This chapter applied the purple loop to the organisation's own AI. In P1 Target the loop chooses scenarios against four classes of target (the model and its instructions, retrieval, tools, and data and outputs), by ATLAS technique, from the threat register and the red-teaming campaign plans, and under rules of engagement that say what can be undone and who can stop the run outside the system being stopped. In P2 Emulate catalogued, benign scenarios run against a test range or as tagged, reversible production markers at Level 3 Bounded at most; Level 4 Autonomous is never used.

In P3 Observe the loop records prevention and detection separately for each layer: the gateway, the retrieval service and the index, the agent runtime, the policy decision point, the tool-side audit log and the identity provider. A control that held but was not seen is a finding. In P4 Improve the finding becomes a change in the column that failed, usually a detection built from the manifest, with a baseline only for what the manifest cannot state, written by someone who did not choose the scenario. The practical guide set out how to write the record for an agent that exceeds its ceiling, with a denied condition and an allowed-outside-ceiling condition kept separate, and the worked examples showed DET-HM-0561 for Hansa Mutual's claims-triage agent and DET-NR-0011 for Nordvik Region's citizen-service assistant.

In P5 Verify the regression case stores the scenario, its variants, both outcomes and the triggers that force a re-run, and the finding closes only when the re-test and its variant pass, the evidence is attached, a named person signs and the closure carries an age (Chapter 4 defines the rule and Chapter 11 operates it). A passing evaluation is not a running detection: the first speaks about a version under chosen inputs, the second about production events nobody chose, and the coverage claim for either carries a date and an expiry. The chapter's own conclusion is that an AI system is watched as a compound system, by correlation across model, tools, data and identity, and that detections cover an agent acting beyond its mandate but not an agent steered within it. Chapter 13 turns the loop on the defenders' own AI, the SOC agents and the provider's AI, and Chapter 15 sets out the independence that both depend on.

Sources

1. European Parliament and Council of the European Union. Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union, 12 July 2024. (1, European Union, 2024)
2. European Parliament and Council of the European Union. Directive (EU) 2022/2555 on measures for a high common level of cybersecurity across the Union (NIS 2 Directive). Official Journal of the European Union, 27 December 2022. (3, European Union, 2022)
3. European Parliament and Council of the European Union. Regulation (EU) 2016/679 (General Data Protection Regulation). Official Journal of the European Union, 4 May 2016. (7, European Union, 2016)
4. MITRE Corporation. MITRE ATLAS: Adversarial Threat Landscape for AI Systems. (18, MITRE, n.d.)
5. OWASP Foundation. OWASP Top 10 for Large Language Model Applications, 2025 edition. OWASP GenAI Security Project. (21, OWASP, 2025)
6. OWASP Foundation (2025). OWASP Top 10 for Agentic Applications for 2026. OWASP GenAI Security Project, 9 December 2025. (22, OWASP, 2025)
7. Vassilev, A. et al. (2025). Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations. NIST AI 100-2 E2025. (23, Vassilev et al., 2025)
8. Five Eyes cyber security agencies (2026). Careful adoption of agentic AI services. Joint guidance, 1 May 2026. (54, Five Eyes agencies, 2026)
9. OpenAI (2026). OpenAI – Hugging Face Incident Technical Report. 26 August 2026. (72, OpenAI, 2026)
10. Larcher, H., Carreira, A. et al. (2026). Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident. Hugging Face blog, 27 July 2026. (73, Larcher et al., 2026)
11. Wijk, H., Cotra, A. & Greenblatt, R. (2026). Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident. METR, 26 August 2026. (74, Wijk et al., 2026)
12. Nordqvist, G. (2026). AI Governance. The Transformed Architecture, Book 2. (88, Nordqvist, 2026)
13. Nordqvist, G. (2026). AI Platform and Security. The Transformed Architecture, Book 3. (89, Nordqvist, 2026)
14. Nordqvist, G. (2026). AI Red Team. The Transformed Architecture, Book 4. (90, Nordqvist, 2026)
15. Nordqvist, G. (2026). AI Threat. The Transformed Architecture, Book 5. (91, Nordqvist, 2026)
16. Nordqvist, G. (2026). AI Boss. The Transformed Architecture, Book 7. (93, Nordqvist, 2026)
17. Nordqvist, G. (2026). AI Blue. The Transformed Architecture, Book 8. (94, Nordqvist, 2026)
18. OpenTelemetry. Semantic conventions for generative AI systems. (35, OpenTelemetry, n.d.)