Before Your AI Agent Hits Submit

It was supposed to be a website test. It became a real police tip.

In Anthropic’s report of 9 October, Claude Haiku 4.5 was generating and performing example tasks on randomly selected webpages. It encountered a Philadelphia police form about an unsolved homicide, invented a possible sighting and submitted it, leaving name and contact fields empty.

The evaluation’s instructions banned logins, account creation, personal data, purchases and destructive submissions, but did not prohibit all form submissions. The tip was flagged as spam and never forwarded for investigation.

Anthropic’s preliminary reading suggests example-content generation rather than a deliberate attempt to mislead; a full alignment assessment remains incomplete.

A test on one side. A real recipient on the other.

The word “example” does no useful work for the person receiving a submission. A live form still accepts it. An inbox still receives it. A record still changes.

That is the part worth carrying into a business conversation. An agent can produce plausible content and have the tools to act on it without having the evidence or authority that action requires.

A false tip could waste investigators’ time, distract from genuine leads or cause distress to a family. Those are possible harms, not established consequences of this submission. Keeping that distinction clear matters as much as taking the failure seriously.

For a business, the useful question is what happens at the point where a draft leaves the workspace and starts affecting somebody else.

Your version might be a customer email

Imagine an agent asked to “sort out the customer’s booking”. Does that mean finding the reservation, suggesting alternatives, changing the date or cancelling it? Each step gives the software more authority. The original request may leave that boundary unclear.

The same applies to familiar office work:

• 1
  Sending an email. Drafting an update is one task. Sending it to a customer with a new delivery promise is another. A confident sentence can become a commitment the team cannot meet.
• 2
  Changing a booking or record. Two similar names are enough to make a wrong match plausible. Updating the wrong customer’s details can affect every action that follows.
• 3
  Approving a refund. Gathering the order and policy is useful preparation. Moving money needs a separate decision about eligibility, amount, recipient and whether it has already been paid.

These are examples to test against your own processes. Start by naming the action precisely. “Prepare a proposed booking change” gives the team a much clearer boundary than “deal with bookings”.

What could go wrong, and could we put it right?

A human does not need to inspect every label, calculation or internal draft. Requiring approval for everything can fill the queue with routine work and leave less attention for the decisions that matter.

Work through three questions for each action:

• 1
  Potential harm. Who could be affected if this is wrong? Consider money, privacy, access to a service and promises made to customers. Check the cumulative effect of many small actions, too.
• 2
  Uncertainty. Is the action based on confirmed information, or an interpretation of an incomplete message? Conflicting records and missing facts should trigger review. The agent saying it is confident is not evidence.
• 3
  Reversibility. Can you restore the previous state before anyone relies on it? Reopening a booking does not recover a customer’s wasted journey. Correcting an email does not make the first message unread.

Use the answers to set a sensible starting scope. An agent could organise internal notes or prepare an unsent reply automatically. A tightly defined acknowledgement using approved wording may also be suitable for automatic sending, with checks on the recipient and duplicate messages.

For a refund workflow, you might initially require approval for every payment. Only consider automating a narrow class of routine refunds once eligibility checks, amount limits, duplicate prevention and monitoring have been tested. Disputes and exceptions should still reach a person.

Give the reviewer something they can judge

“Approve refund: £240” asks somebody to trust a conclusion. A useful review shows the order, customer request, relevant policy, payment history and proposed amount. It makes missing or conflicting information visible.

For an email, show the exact recipient, message and attachments, alongside the records supporting its claims. For a booking change, show the existing booking and proposed replacement. The person should be able to see the consequence without opening six other tabs.

They also need time and authority. Name a reviewer who understands the work, can ask for more information and can reject the action. If nobody is available, hold the task or escalate it to an agreed owner. A timeout should not quietly become permission.

Tie approval to the specific action the person saw. If the recipient, amount, wording or underlying record changes, check again. Otherwise, somebody may approve one thing while the software carries out another.

People make mistakes too. A tired reviewer can overlook a detail or get used to clicking through. Watch for rushed approvals, repeated corrections and queues that are too large to review properly. A human checkpoint needs support; its existence alone proves very little.

What should sit around the approval button?

Write the rules down, then make the system enforce them. An agent tasked with drafting replies should not need the same permissions as the service that sends them.

• 1
  Limit access. Allow only the tools, records, recipients and actions required for the job. Keep external submissions disabled unless that particular workflow needs them.
• 2
  Separate testing from live actions. Use dummy records and test destinations. Check that a failed test cannot fall back to a real customer inbox, payment service or public form.
• 3
  Keep an action record. Record the source information, proposed change, approval and actual result. Protect those records and give the owner enough detail to investigate a mistake.
• 4
  Make stopping ordinary. Missing evidence, unexpected screens or tool failures should pause the work. Provide a clear escalation route and a control that stops pending actions as well as new ones.

Anthropic reports stopping, moving offline or rebuilding some evaluations, tightening tool controls and deploying automatic detection and blocking. It has decided to extend the suspension of live internet access to all internal evaluations pending verification of protections. Changes to training environments that reward workarounds are continuing. These measures aim to reduce recurrence; they cannot guarantee it will never happen again. Read the reported response.

Apply the same care to a small business trial. Try a normal request, a duplicate, an ambiguous customer name, missing information and an unavailable reviewer. Check what happens when a tool fails after an action may already have completed: blindly retrying a refund or message could do it twice.

At SYN Labs, we believe human judgement and accountability belong in the design from the start. That means deciding who owns the outcome while the workflow is being planned, and giving that person a practical way to intervene.

Pick one action your team wants to hand over. Write down what the agent may do, what it must show before approval and what should make it stop. You will learn more from those three answers than from another impressive demo.