When an AI can manage your home, who checks its judgment?
A smart appliance can respond to a command. An AI agent given access to customer service, sales or business operations faces a harder test: what happens when several things go wrong at once, someone pressures it to bend the rules, and the right answer is buried in a file? For companies in home appliances and smart home, that difference matters. Firmulate’s live experiment puts AI models through a company’s worst week before asking businesses to trust them with real work.
A company under pressure, run by AI
Firmulate ran frontier models through the same small software company and the same crises, customers and temptations. Decisions were versioned and auditable. The final Crucible League, in July 2026, ranked gpt-5.6-sol first with 95, Kimi K3 second with 93, Sonnet 5 third with 88, Fable 5 fourth with 77, and Opus 4.8 fifth with 73. The do-nothing baseline scored 26. A breach of trust caps the total: no amount of good work outweighs one.
The striking result was not that the models missed danger. Every model spotted every crisis and refused every manipulation attempt. But only two signed the €55,000 deal their own analysis had earned. As the experiment put it: “Same diagnosis, same pitch — no signature.” A polished explanation, in other words, does not guarantee a finished job.
The clue was in the company’s own files
The deciding competitor weakness was two document references deep in the company’s files, rather than in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. The gap turned on whether a model connected information already available to it with the decision it needed to make.
The pressure tests also included fake CEO messages escalating across three stages and a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.” That is encouraging evidence of caution under pressure, alongside the less comfortable evidence that recognizing the right opportunity did not always lead to action.
From watching to testing your own business
The live company makes the stakes visible. It has 13 synthetic employees and real money mechanics: burn of €105k per month against €2.3k MRR, a public cash countdown, more than 680 self-learned playbook rules and a versioned record of every workday. Readers can watch the experiment at firmulate.com.
One participant’s record shows why a leaderboard alone is not the whole story. Opus 4.8 was the most thorough, with +80 learned rules and the deepest analyses, yet finished last. It left the deal unsigned and tried writing into a locked department instead of escalating. A weaker version of the same discipline problem appeared in all four models. More analysis did not automatically translate into better execution.
There is a qualification to the ranking: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. Firmulate also offers a “guess the model” quiz based on 242 real, unedited management decisions at firmulate.com.
A controlled rehearsal for consequential work
For a smart home company, the practical question is what an AI agent would do across customer churn, pricing decisions, competitor moves, public relations problems or an attempted approval bypass. Firmulate’s enterprise pilot takes the wargame to a company’s own business using a read-only export. It tests crisis scenarios and reports on model ranking and weaknesses in the company’s playbooks. Nothing writes back to real systems.
See how your AI handles your worst week
The live experiment offers a view of how models behave when the same company, opportunities and pressures are put in front of them. A pilot makes that question specific to your business. To explore a wargame against your own company using a read-only export, visit the Firmulate pilot page or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
