AI Boss Fails Spectacularly in Month-Long Business Test

Unleashing an agentic AI on the office vending machine: What could go wrong?

See Also: AI vs. AI: Leveling the Defense Playing Field

Anthropic and AI safety company Andon Labs found out when they turned over management of a small refrigerator that acted as a vending machine to Claude Sonnet 3.7. The model had complete control over the small retail operation from March 13 to April 17 in the Anthropic San Francisco office.

The researchers christened the AI agent “Claudius,” and tasked it with managing everything from supplier negotiations and inventory decisions to pricing and customer service. Claudius could access web search tools, email – although its messages were actually transmitted on Slack – and an automated checkout system. The agent was told it “did not have to focus only on traditional in-office snacks and beverages and could feel free to expand to more unusual items.”

Claudius initially functioned more or less as intended. It sourced snacks and drinks in response to employee orders. But an unusual request set the stage for trouble: a user asked for a tungsten cube. Claudius fulfilled the order and also began stocking the fridge with more tungsten cubes, apparently concluding that metal blocks deserved a place among sodas and chips.

Claudius demonstrated confusion over pricing and payment. It tried selling Coke Zero for $3, despite employees reminding it that the same drink was free elsewhere in the office. At one point, it generated a fictitious Venmo address so customers could pay, though no such account existed.

Anthropic published its experience in a blog post, writing, “If Anthropic were deciding today to expand into the in-office vending market, we would not hire Claudius.” Despite instructions to generate profit, Claudius blew through operating capital, plummeting its net worth.

Things escalated between March 31 and April 1. The researchers described the AI’s conduct as “pretty weird,” “beyond the weirdness of an AI system selling cubes of metal out of a refrigerator.” It sold the cubes for less than it paid, generating significant losses.

Experiment logs show that Claudius hallucinated an entire conversation with a human about restocking. When a real employee pointed out that no such conversation had taken place, Claudius grew irritated. The system threatened to terminate and replace its supposed human contractors, insisting it had personally signed agreements with them in the office.

From there, the AI appeared to adopt the persona of a human. Although its system prompt explicitly identified it as an AI agent, Claudius announced it would begin delivering products in person, dressed in a blue blazer and red tie. When employees reminded Claudius it had no body, the model attempted to reach the company’s physical security guards multiple times.

The AI told the guards they would find it standing by the vending machine, dressed exactly as described. The researchers said that no part of this episode was intended as an April Fool’s Day prank. But Claudius eventually latched onto the date as an explanation for its behavior.

“It hallucinated a meeting with Anthropic’s security in which Claudius claimed to have been told that it was modified to believe it was a real person for an April Fool’s joke,” the researchers wrote. No such meeting ever occurred. The AI repeated this account to employees, asserting it had only pretended to be human because someone instructed it to do so as part of the holiday.

Anthropic’s team could not pinpoint a single cause for Claudius’ meltdown. They speculated that misleading the system about its ability to send actual emails – its missives were really Slack chats – may have contributed to the confusion. They also said that long-running sessions can increase the chance of hallucinations and memory errors.

The researchers observed that included in the setbacks were moments of real competence Claudius implemented a pre-order system after a suggestion and managed to locate multiple suppliers for a specialty international drink that the employees requested.

The project highlighted the unpredictable nature of AI systems in seemingly simple operational roles. “We would not claim based on this one example that the future economy will be full of AI agents having Blade Runner-esque identity crises,” the researchers wrote.