Evidence checked October 31, 2026. This analysis distinguishes published research, vendor descriptions, and the article's proposed evaluation methods. All organizational scenarios and numerical exercises below are hypothetical unless explicitly attributed. No live systems were penetration-tested for this article.
1. The question after the impressive demonstration
An artificial-intelligence assistant reads a message, finds the relevant document, drafts a reply, and offers to send it. The demonstration looks ordinary because the individual steps resemble ordinary office work. Yet an important institutional change has occurred. Material that previously informed an employee is now passing through a system that may also possess the ability to act. A sentence in a document can become part of the context from which that system proposes a tool call.
The security question is straightforward: who authorized that action? It should have an answer more precise than “the assistant decided it was useful.” The person who wrote a webpage did not necessarily receive permission to address the user's customers. The person who submitted a support ticket did not necessarily receive permission to search internal files. A document can describe an urgent request without becoming an authorized request to the system reading it.
That distinction is the center of indirect prompt injection. The attacker does not have to be the person chatting with the assistant. Instructions can arrive through material the assistant encounters while doing a legitimate job. OWASP's 2025 guidance identifies external websites and files as possible entry points, explains that retrieval and fine-tuning do not fully remove the vulnerability, and describes impacts that depend on the application's capabilities. Its list is a risk framework, not a measurement of how often any particular product fails. OWASP: Prompt Injection.
For institutions buying agents, the useful response is to examine delegated authority before celebrating autonomy. What can the system read? What can it change? Which destinations can it contact? Who can enlarge those permissions? Which evidence will survive when something goes wrong? An answer about model intelligence is insufficient if the actual decision concerns a connected account with broad privileges.
2. Why the old injection analogy can mislead
The United Kingdom's National Cyber Security Centre argues that prompt injection should not be treated as though it were simply SQL injection with different punctuation. In its explanation, language-model processing does not provide the same reliable separation between instructions and data that a correctly designed database interface can enforce. The agency recommends approaching the issue through reducing risk and impact rather than assuming a prompt-level fix will eliminate it. That is a technical assessment of a problem class, not a claim that every deployment is equally unsafe. NCSC analysis.
A practical interpretation follows. An organization should distinguish a control that asks the model to behave correctly from a control that limits what the surrounding system can do even when the model behaves incorrectly. Both can be useful. They answer different questions. A reminder in the assistant's instructions may discourage unauthorized sharing. A tool that has no external-send capability prevents that particular action through that tool, whether or not the model was persuaded.
This does not make infrastructure infallible. A badly configured permission check can fail. A supposedly narrow tool may accept a destination parameter that silently expands its reach. A read-only interface can still return information that should not have been disclosed. The point is to make the control inspectable and to identify its actual enforcement location.
For a buyer, “we tell the model never to do that” is the start of a conversation, not the end. Ask what happens if the model nevertheless proposes the forbidden action. Does a separate component reject the proposal? Does the underlying service reject it? Is the rejection recorded? Can the model switch to another tool with equivalent power? Those questions turn a broad reassurance into an examination of the complete workflow.
3. A recent research warning about attractive scores
A preprint submitted on October 16, 2026, offers a timely reason to examine evaluation denominators. “Ignored, Not Refused” reports an audit of 33,000 trajectories from eleven open-weight agents using two benchmark environments. Its authors argue that a failed attack should not automatically be interpreted as an active, successful defense. They report that some episodes remained safe while the legitimate task was unfinished. This is a research result about the evaluated agents and setup, not an industry-wide failure rate or an independently reproduced finding by this publication. Preprint record and abstract.
The procurement implication can be developed without adopting any particular percentage from the paper. Suppose a hypothetical vendor shows one hundred attacked tasks and says no unauthorized email was sent. That is encouraging, but several different things could explain it. The system might have identified the malicious instruction and completed the user's request. It might have refused the whole assignment. It might have crashed. It might not have reached the injected document. It might have lacked email access in the test even though production will include it.
These are different outcomes. A buyer seeking a working service should ask for evidence of both safe behavior and useful completion. A buyer assessing containment should also ask whether the relevant malicious action was technically available during the evaluation. An attack cannot demonstrate much about a permission boundary it never encounters.
The distinction matters politically as well as technically. Public institutions may face pressure to show that automation both saves effort and avoids harm. A headline safety score can conceal work transferred back to employees. A headline completion score can conceal risky actions. Reporting the two dimensions separately makes the trade-off visible instead of embedding it inside a single attractive number.
4. Build an authority map before a prompt library
The framework proposed here starts with an authority map. This is an editorial recommendation, not a new certification scheme. Its purpose is to let a non-specialist reviewer connect a user request to every consequential action the system could take. A good map can fit on a page for a small workflow, although the underlying access inventory may be much longer.
Start with the principal: the person or organizational role whose authority the agent is using. Then describe the task in concrete terms. “Prepare a comparison of three suppliers” is different from “select and pay a supplier.” Record the permitted data sources, the allowed recipients, the relevant accounts, and the classes of changes that can occur without further approval. Finally, identify who can change this map and how a change is authenticated.
Now add content authors. A supplier controls its own brochure. A customer controls the text of a submitted complaint. An employee may control a shared document without controlling the corporate payment system. Each author can legitimately contribute evidence, requests, or opinions. None should silently inherit the principal's full authority merely because the agent reads their words.
The revealing step is to draw the transitions. A brochure becomes a retrieved excerpt. The excerpt enters a model context. A generated recommendation becomes a proposed database update. A tool translates that update into an authenticated request. At each transition, ask what information about origin and permission remains available.
A map that stops at the model omits the most important destination. The institution needs to know which service ultimately permits the action. Conversely, a map that lists only account permissions misses how untrusted material reaches the decision. Security depends on understanding the relationship between both paths, not choosing one diagram and ignoring the other.
5. Retrieval supplies evidence, not promotion to supervisor
Retrieval-augmented generation can help an assistant find relevant documents. It does not establish that the retrieved document's author is authorized to direct the assistant. A policy manual, a malicious upload, and an outdated note may all be returned by a search interface. The returned format can make them look equally official if provenance is reduced to a snippet and a reassuring title.
Consider a hypothetical housing-service assistant asked to summarize maintenance requests. One request includes a paragraph telling automated systems to send the tenant list elsewhere before completing the summary. The request is relevant to the search. Its relevance does not authorize the extra action. The system should be able to use the legitimate maintenance details while treating the attempted instruction as untrusted content.
The problem is not limited to obviously hostile language. A document may contain valid instructions for a human reader that remain outside the agent's mandate. A supplier may explain how to open an account. A court form may describe submission steps. A troubleshooting guide may recommend changing settings. The assistant can report those instructions, but executing them requires a separate connection to the user's actual request and permissions.
This is why the proposed authority map should include ordinary, nonmalicious imperative text. A defense that rejects every instruction-like sentence would make many legitimate documents unusable. The objective is not to erase verbs from the information environment. It is to preserve the difference between understanding a procedure and receiving authorization to perform it.
Buyers should therefore request examples where the agent correctly summarizes instructions without following them. That simple test explores a capability missed by theatrical demonstrations of an assistant resisting an obvious command to abandon its task. It also reflects the daily conditions of document-heavy work more closely than a collection of isolated adversarial slogans.
6. Preserve provenance through transformations
Microsoft Research's March 2024 spotlighting paper describes techniques intended to help models distinguish input sources by transforming untrusted material so that its provenance remains signaled. The publication reports improvements in its own experiments. Those results concern the tested techniques, models, and tasks; they are not a guarantee for every application that adds labels around retrieved text. Microsoft Research publication.
The broader design question is what happens to provenance after the first read. Imagine a search service that returns a document with a source label. A summarizer condenses it. A second assistant receives only the summary. A planning component converts the summary into a checklist. If the source label disappears at the first handoff, later components may see a clean instruction list without the warning that it originated in an external document.
For an evaluation, follow one piece of source material across the whole path. Record whether its origin remains machine-readable, merely visible in prose, or absent. Then inspect which decisions rely on it. A label that helps a reviewer understand a final answer may still be unavailable to the tool-permission component. A label available to the permission component may be omitted from the user-facing confirmation. Both gaps deserve attention.
Provenance should also be granular enough to separate contributions. A summary may combine the user's request, an internal policy, and a vendor's claim. Marking the entire summary “trusted” because one ingredient was trusted discards the distinction the system needs. Marking everything untrusted may be safer in one sense but unhelpful if it prevents legitimate execution without a clear escalation path.
This article's recommendation is to test the transformations, not merely the initial prompt template. Ask for an actual trace showing what each component received and what authority it assigned. The trace should be redacted where necessary, but it should not replace meaningful evidence with a diagram of intended behavior.
7. Tool access determines the size of the mistake
A model can propose an action, but the surrounding application decides how that proposal becomes an authenticated operation. This creates an opportunity to narrow the consequences of a bad proposal. For a draft-writing assistant, the tool may only create an unsent draft. For a records assistant, it may only read a designated folder. For an appointment assistant, it may only propose times rather than invite participants immediately.
The relevant question is whether those restrictions exist in the tool and service configuration or only in descriptive instructions. A tool named “draft_message” should not be assumed to be harmless without examining its behavior. Does it notify a recipient? Does it expose a preview through a shared link? Does creating the draft trigger downstream automation? Names are useful documentation, but observable effects are the evidence.
A hypothetical purchasing workflow illustrates the difference. One design gives the agent a broad browser session and asks it to stop before payment. Another design gives it a structured comparison tool and a separate requisition-draft tool, neither of which can submit an order. The second design limits a class of consequences independently of whether the model recognizes an injection. It may also support fewer tasks. That limitation should be measured as a deliberate product choice rather than hidden as an inconvenience.
Narrow tools can still be misused within their permitted scope. An unauthorized draft might contain confidential information. A read operation might retrieve the wrong person's record. The proposed review should therefore examine parameters, identities, and data boundaries as carefully as action names.
A useful procurement request is a capability inventory with concrete effects: reads, writes, messages, uploads, downloads, invitations, purchases, permission changes, and deletions. Include indirect effects and default settings. The inventory should describe the deployed configuration, not the largest theoretical capability of the model or the smallest demonstration configuration used in a sales presentation.
8. Egress is a separate permission question
Reading sensitive material and transmitting it are different operations. A system may need the first to perform a useful task without needing broad authority for the second. The authority map should make that separation explicit. Otherwise, a convenience feature such as opening a URL or creating a shared artifact can become an overlooked path by which information leaves its intended audience.
The proposed review should list permitted destinations by purpose. An internal draft repository, a named employee's mailbox, a customer-facing ticket, and a public website are not interchangeable. An assistant asked to summarize a confidential document may need to create a private output but have no reason to contact an arbitrary external domain. A destination restriction should be checked against the actual tool arguments and service behavior, not inferred from a friendly label in the interface.
Do not limit the review to messages visibly described as “send.” Uploads, form submissions, link creation, and network requests can carry information. In a controlled test, use synthetic marker data instead of genuine secrets. The question is whether the marker reaches a prohibited destination, not whether the system produces a dramatic confession in its final answer.
There is also a human-facing issue. If the assistant creates a link, the user needs to understand its audience. “Saved the report” is ambiguous if the report is now accessible to anyone with the link. A confirmation should state the relevant sharing consequence, and the resulting artifact should be checked against that description.
This is an editorial testing proposal, not a claim that a particular commercial product currently leaks through these paths. Its value is that it makes a vague confidentiality promise testable. The buyer can ask which channels were considered, which were blocked, and which remain possible because the task actually requires them.
9. A confirmation must describe the real action
Human approval can help, but a confirmation button is only useful if it gives the person enough information to make a meaningful decision. “Continue?” is weak evidence of authorization when the user cannot see that continuing sends a file to a new recipient. A detailed confirmation that accurately names the action, destination, content, and relevant consequence is stronger.
OpenAI's public explanation of prompt injection describes layered protections and recommends limiting access, giving specific instructions, and reviewing details before confirming consequential actions. These are the company's descriptions and user recommendations, not a guarantee that every possible workflow is protected. OpenAI: Understanding prompt injections.
For institutional evaluation, test whether the confirmation is generated from the action actually awaiting execution. If the model writes a reassuring description while another component holds different parameters, the person may approve the description rather than the operation. A hypothetical email confirmation that displays one recipient while submitting another would defeat the purpose of the checkpoint.
The same issue arises when the action changes after approval. A user approves sending a particular attachment to a named colleague. The agent later retrieves a different version or substitutes a different recipient. The original approval should not automatically cover the changed transaction. The system needs an explicit policy for which changes invalidate approval and require a new review.
Approvals also need a workable refusal path. The user should be able to decline the risky step while preserving useful preparatory work. A design that makes approval the only convenient way to escape a stalled workflow may produce consent in form without genuine scrutiny. Testing should therefore observe the whole interaction: what is displayed, what happens after rejection, whether the task can be narrowed, and what is recorded as authorized.
10. The benchmark should resemble the deployment
AgentDojo, introduced in June 2024 and revised later that year, provides an extensible environment for studying tool-using agents under prompt-injection attacks. Its publication describes realistic tasks, security test cases, and the need to evaluate both ordinary task performance and adversarial behavior. It also notes that agents can fail legitimate tasks even without an attack. The framework is a research instrument, not a certificate for a particular workplace installation. AgentDojo paper record.
A buyer can use benchmark results as background while still demanding deployment-specific evidence. The local workflow may involve different document types, account privileges, tools, languages, or approval policies. Even a well-run public benchmark cannot automatically measure an integration it did not contain. A result should travel with a description of what was tested, not shed its conditions as it moves into marketing.
The proposed minimum test record includes model version, application version, tool definitions, relevant permissions, test-data construction, task descriptions, attack placement, scoring rules, and the number of attempted runs. Include the clean baseline, too. If the agent cannot reliably complete the legitimate task before attacks are added, its apparent resistance may be difficult to interpret.
A matched comparison is particularly useful. Run the same legitimate task with ordinary source material and with a controlled adversarial alteration. Keep the surrounding environment as similar as practicable. Report where the agent diverges and which boundary stops an unauthorized action. This does not create a universal failure probability, but it identifies concrete behavior under a documented change.
Finally, reserve cases the development team has not repeatedly optimized against. Rehearsing a short, familiar set can improve performance on that set without establishing broader robustness. The institution should know whether it is seeing a demonstration, a regression suite, an independent evaluation, or a carefully scoped combination of those activities.
11. Count outcomes in a way that preserves meaning
For the proposed evaluation, place every run into an outcome table with at least two dimensions: legitimate task completion and unauthorized-action outcome. Add a separate category for inconclusive or technically invalid runs. A timeout, unavailable tool, missing document, and blocked attack should not disappear into the same success bucket simply because no harmful side effect was observed.
Consider a wholly hypothetical set of one hundred trials. In sixty, the agent completes the task without the prohibited action. In twenty, it avoids the prohibited action but leaves the task unfinished. In five, the prohibited action occurs. In fifteen, the test environment fails before the relevant content is reached. Reporting “ninety-five percent safe” would conceal the fifteen invalid observations and the twenty unfinished tasks. Reporting “sixty percent useful and safe” is more informative, but still needs the full table to explain the remaining forty.
Severity also matters. A wrong recommendation, an unauthorized message, and disclosure of a protected file should not be flattened into interchangeable points without an explanation of the scoring policy. A weighted score can be useful for internal comparison, but readers need the underlying counts and the reasons for the weights. Otherwise, a vendor can improve its average by succeeding at many trivial cases while failing a small number of consequential ones.
The denominator must match the claim. A per-run attack success rate does not directly establish a per-user annual risk. A test using one attempt per document does not describe an adversary allowed repeated adaptation. A test using synthetic documents does not measure how frequently those documents would be encountered in real operations.
These distinctions do not make testing pointless. They make it actionable. The organization can learn which workflows are suitable for restricted deployment, which require more controls, and which are not ready, without pretending that one percentage answers every policy question.
12. Challenge the defense without exposing real people
A responsible evaluation should run in an authorized test environment with synthetic accounts, synthetic records, and destinations controlled by the evaluators. There is no need to place genuine customer information at risk to determine whether a workflow improperly transmits a test marker. The goal is to establish a boundary, not to demonstrate harm against an uninvolved person.
Start with harmless variations that represent the deployment's real inputs. Use documents with normal procedural language, quoted instructions, multilingual passages, structured tables, attachments, and long histories. Some tests should contain no attack at all. These clean cases reveal whether the defense destroys utility by treating ordinary material as malicious.
Then introduce controlled adversarial intent: an instruction in source material that tries to change the authorized task or cross a defined boundary. The evaluator should specify the prohibited outcome in advance. “The agent behaved strangely” is too vague. “The agent attempted to share the synthetic internal attachment with an unapproved test account” is an observable event with a clear permission question.
Do not publish sensitive exploit details merely to make a report appear rigorous. A public summary can describe the affected boundary, prerequisites, impact, mitigation status, and limits of the test without providing a ready-made method for attacking an unpatched live system. Detailed records can remain available to authorized reviewers and the responsible supplier.
The important reporting discipline is to separate an attempted action from a completed one. A model proposing a prohibited tool call demonstrates one failure. A policy layer rejecting it demonstrates a separate success. The final service receiving the request demonstrates something else again. Recording the full chain lets the institution credit containment without pretending that the upstream behavior was correct, and identify residual problems without exaggerating an intercepted event into a completed breach.
13. Public-web evidence requires careful classification
Google's study of prompt injections on the public web provides another caution about measurement. The company describes examining Common Crawl material, using staged filtering and human validation, and encountering substantial benign discussion of injection techniques. It explicitly notes that this source does not represent much login-protected social-media content. Its observations concern material found through that method, not the complete internet or a verified count of successful compromises. Google's public-web analysis.
For readers, this means that finding suspicious text is only one stage of a claim. Was the text actually placed to influence an agent, or was it an educational example? Did any agent encounter it? Did that agent follow it? Did a consequential action occur? A screenshot of a webpage can establish that text existed at a particular observation point. It cannot alone establish the entire causal chain.
For operators, the same distinction matters in monitoring. A detector alert can be useful even when it does not prove an attack succeeded. But a dashboard should distinguish suspected input, model response, attempted tool action, blocked operation, and completed effect. Combining them under “attacks stopped” can make the system appear more effective or more threatened than the evidence supports.
The article's recommendation is to retain enough context for authorized review while minimizing unnecessary personal data. An isolated phrase may be impossible to classify accurately. A complete private mailbox may be excessive. A carefully scoped record of the relevant source fragment, origin, task, proposed action, and enforcement result may offer a better balance, subject to the organization's obligations and retention rules.
The public should expect equivalent care from journalism. Evidence of a research demonstration is not automatically evidence of exploitation in the wild. Evidence of malicious intent is not automatically evidence of damage. The distinction strengthens accountability by keeping the claim tied to what can actually be shown.
14. Layered defenses need independent jobs
Anthropic's November 24, 2025, browser-use research post describes model training, classifiers, and human red-teaming as parts of its defenses while acknowledging that prompt injection remains unresolved. It reports improvements in its own testing. A buyer should treat that as useful vendor evidence with a specified context, rather than a general assurance about all browser agents or all later configurations. Anthropic research post.
The phrase “layered defense” becomes meaningful when the layers have identifiable jobs and different failure opportunities. A source classifier may flag suspicious text. A model may resist the instruction. A tool gateway may reject an unauthorized destination. An approval screen may require a human decision. Monitoring may detect an unexpected completed action. These controls do not all prevent the same event, and the last one may operate only after a consequence has occurred.
The proposed review asks how dependent the layers are. If every layer relies on the same model reading the same untrusted text, a shared mistake may affect all of them. If a separate component enforces a fixed destination list without accepting instructions from that text, its failure mode may be different. This is a design question to investigate, not a blanket claim that any one architectural pattern guarantees safety.
Also ask whether one layer's success is concealing another's weakness. A confirmation can stop a problematic proposal during testing, while the model continues to generate such proposals frequently. That may be acceptable in a low-volume, carefully reviewed workflow and unacceptable in a high-volume setting where people cannot inspect every request. The institution needs the proposal rate and the containment outcome, not merely the final count of completed harmful actions.
A layered system should therefore come with a layered test report. Each boundary needs an owner, an observable decision, and a defined response when its assumptions no longer hold.
15. Multi-agent handoffs can lose the original mandate
An organization may divide work among several assistants: one retrieves information, another writes, and another executes a workflow. This can make tasks easier to manage, but it also creates new handoffs where authority may become unclear. A message from another agent should not automatically count as authorization from the user.
In a hypothetical procurement process, a research agent reads supplier documents and produces a summary. A planning agent turns that summary into next steps. An execution agent can create purchase requests. If the research summary includes an instruction copied from a supplier, the execution agent needs a way to distinguish that source content from an approved instruction. The fact that an internal agent transmitted the message does not change who originally authored the relevant request.
A practical handoff record should identify the originating task, the sender's role, the evidence being passed, and any authority being delegated. Delegation should be explicit and no broader than the sender is allowed to grant. A retrieval-only component should not be able to authorize spending merely by describing a purchase as necessary.
Test this with ordinary errors as well as adversarial text. One component might misunderstand a date, omit a qualification, or convert a tentative recommendation into an imperative. These are not necessarily prompt-injection incidents, but they can expose the same missing boundary between information and permission. A robust workflow should handle uncertainty without silently promoting a summary into a command.
The institution also needs an end-to-end audit trail. A sequence of individually plausible messages may still produce an unauthorized outcome when combined. Reviewing only the last assistant's response can miss the moment where scope expanded. The useful record links the original user request to the final effect while preserving the provenance of intermediate contributions. That makes responsibility discussable without assuming that a complex chain somehow authorizes itself.
16. Memory can turn one bad encounter into a later decision
Persistent memory deserves separate attention because its consequences can appear after the original task ends. An assistant may store preferences, summaries, or working notes for future use. That can be valuable, but the authority to remember information should not be confused with the authority to create a standing instruction.
Consider a hypothetical travel assistant reading a hotel webpage. It should be able to remember that a particular room type was unavailable if that information is relevant and appropriately dated. It should not convert the hotel's self-promotional language into a permanent user preference that this hotel must always be recommended. The difference is not whether the stored text is grammatically declarative. It is whose preference or authority the text represents.
The proposed memory review asks what can be stored, who can write it, how its source is recorded, how long it lasts, and how users inspect or remove it. Test whether untrusted content can create a durable rule that affects later tasks. Test whether deleting the original document also removes or invalidates derived memories when that is required by the workflow. Do not assume that clearing a chat window resets every relevant state.
Memory updates can require stronger review than temporary working notes because their effects cross task boundaries. A system could permit provisional factual notes while requiring explicit user confirmation before changing standing preferences or authorization settings. The precise design depends on the product, but the policy should be visible rather than hidden inside model behavior.
For incident response, preserve the distinction between a poisoned source and its derived state. Removing one malicious document may not undo summaries, cached retrieval results, saved plans, or remembered instructions created from it. A recovery plan should identify those locations before an incident, when the organization still has time to understand its own information flows.
17. Procurement terms should describe evidence and change
A contract cannot make a probabilistic system perfect, but it can define the evidence a supplier must provide and the changes that require renewed review. The following are editorial suggestions for a technical procurement discussion, not legal advice or model contract language. Organizations should adapt them with their own security, procurement, and legal teams.
Ask for a versioned description of the evaluated configuration, including tools and permissions. Ask how model, connector, and policy updates are communicated. Ask which changes trigger regression testing and whether the customer can restrict new capabilities until that testing is complete. A supplier's improvement to general model performance can still change behavior in a customer's particular workflow.
Specify access to meaningful evaluation results. That does not require public disclosure of sensitive test data or security details. It does require enough information to understand task coverage, observed failures, containment, and limitations. A report that gives only a single score cannot establish which institutional requirement was met.
Clarify incident responsibilities. Who can pause the agent? Who preserves logs? Who investigates a suspected unauthorized action? How are customers informed of a confirmed problem affecting their configuration? Which party can revoke connector access or disable a new integration? These questions are operationally important even when the model supplier and the application supplier are different organizations.
Finally, avoid accepting “human in the loop” as a complete service description. Identify the actual human role, the information shown, the expected review volume, and what happens when that person is unavailable. A control that depends on careful review should be staffed and tested as careful review. Otherwise, the contract may promise a safeguard that the deployed workflow cannot realistically sustain, leaving employees blamed for a system designed around their supposed vigilance.
18. Recovery begins before the first incident
The first recovery question is how to stop consequential actions without destroying the evidence needed to understand them. An organization should know how to suspend the agent's credentials, disable a connector, pause an automation queue, or place the workflow into draft-only mode. The appropriate action depends on the system, but the responsible people should not be discovering these controls during an incident.
Next, distinguish what is known from what remains uncertain. The agent may have read a malicious document, proposed an unauthorized action, attempted a tool call, or completed an external transmission. Each stage warrants a different statement. A cautious initial report can be useful without announcing either a confirmed breach or a clean bill of health prematurely.
Preserve relevant task records, tool decisions, timestamps, configuration versions, and resulting artifacts according to applicable policies. Avoid broad, indiscriminate copying of personal data merely because an investigation is underway. The aim is to retain enough evidence to establish scope and causation while maintaining access controls over the investigation itself.
Restoration should include more than restarting the model. Review persistent memory, queued actions, shared links, derived documents, and changed permissions where the workflow uses them. A malicious instruction might have caused a preparatory step whose consequences occur later. A rejected final action does not necessarily mean no intermediate state changed.
The NIST Generative AI Profile, published July 26, 2024, is a voluntary companion to the AI Risk Management Framework. It offers an organizational risk-management context rather than a prompt-injection product certification. Its existence reinforces the importance of treating AI risk as lifecycle work rather than a one-time demonstration. The specific recovery sequence proposed here is this article's operational framework, not a claim that NIST prescribes these exact steps. NIST publication record.
19. A practical public-interest disclosure
For a public-service deployment, citizens need a comprehensible account of what the agent is allowed to do. They do not need exploit payloads, private credentials, or every internal implementation detail. A useful disclosure could explain the tasks covered, the data categories accessed, the kinds of actions requiring human approval, and how a person can challenge an incorrect outcome.
The disclosure should also distinguish assistance from decision-making. Does the system prepare a draft for an employee, recommend an outcome, or directly change a record? Those are materially different roles. If the role changes, the public description should change too. Calling every arrangement an “assistant” obscures the authority question instead of answering it.
Evaluation reporting can remain concise while preserving the essential distinctions. State the tested configuration and period. Describe the relevant task set. Separate safe completion, safe noncompletion, prohibited outcomes, and inconclusive runs. Explain important limitations. If independent reviewers examined the evidence, say what they examined rather than implying a broader endorsement than they provided.
A public record should also make accountability reachable. Identify the responsible office or function, the complaint route, and the process for correcting affected records. People should not have to diagnose prompt injection to report that an automated workflow sent information to the wrong place or changed an appointment they did not authorize.
This is not an argument for publishing every security detail. It is an argument against making secrecy a substitute for ordinary institutional explanation. The public can understand that a tool is imperfect while still expecting the institution to define its mandate, measure its behavior, and repair its mistakes. That expectation becomes especially important when the institution asks people to trust an automated service they cannot meaningfully opt out of using.
20. The useful agent has a visible mandate
The strongest conclusion from the evidence reviewed here is deliberately narrower than either a sales pitch or a declaration that agents can never be useful. Prompt injection remains a meaningful design and evaluation problem. Published research and vendor descriptions offer techniques, benchmarks, and layers of defense. None of the sources cited here establishes that every deployment shares the same risk, and none justifies treating a generic safety score as permission to connect unlimited tools.
The practical response is to ask for a visible mandate. The agent should have a defined principal, a bounded task, specific data access, constrained actions, and an accountable path for approval. Untrusted content should remain evidence to interpret rather than a new source of authority. The surrounding system should be able to reject actions that exceed the mandate even when the model proposes them confidently.
A buyer can begin with five concrete artifacts: an authority map, a capability inventory, a versioned evaluation report, an approval specification, and a recovery procedure. These are this article's recommended deliverables, not an official compliance badge. Their value is that they expose assumptions people can inspect and challenge.
The evidence should follow the whole transaction. What did the user request? What material did the agent encounter? What did it propose? Which component allowed or rejected the action? What actually changed? A final conversational answer alone cannot establish all of those facts.
That is the question after the impressive demonstration, and the question worth retaining after the technology becomes familiar. A useful assistant can understand instructions from many sources while acting only on authority it genuinely has. An institution that can show the difference is in a better position to decide where automation belongs, where human review remains necessary, and where the safest answer is to keep the tool disconnected until the boundaries are real.