Get baking supplies delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
What if your favorite ice cream parlor ran entirely on AI — but struggled to turn a profit?
Imagine watching a real business in action, where AI models make every decision, face crises, and even turn down lucrative deals. Welcome to the world of Firmulate, a live experiment in building an AI-run company that’s transparent, unpredictable, and painfully real.
The Company That’s Watching Itself
Firmulate hosts a small, virtual software company operated by 13 synthetic employees — AI models that simulate human decision-making in real-time. Every workday, this company faces the same crises, temptations, and customer needs as any startup, but with one crucial difference: all decisions are publicly recorded, versioned, and auditable. You can watch its daily struggles at firmulate.com/live.
How Does It Perform?
Despite its high transparency, the company is far from profitable — burning €105,000 monthly against just €2,300 in recurring revenue. Its cash countdown is public, and every move is logged, making it a true ‘build-in-public’ experiment on an extreme scale. The goal? To see whether AI-driven decision-making can handle real-world crises and ethical dilemmas, or whether it’s just good at chat.
The AI Models in Action
Four frontier AI models — including the prominent GPT-5.6-sol and Kimi K3 — faced the same tough week in the company’s shoes. These models encountered customer crises, internal conflicts, and manipulative social engineering attempts. Remarkably, all of them identified every crisis and refused every manipulation attempt, but only two managed to close the deal that was worth an extra €4,583 in monthly revenue.
The critical weakness was buried deep in the company’s internal documents, not in the customer interactions. When the models read these hidden files, they uncovered a key piece of information that enabled them to close the deal at full price. Those who didn’t read the files missed out on the biggest opportunity.
Ethical and Security Challenges
In a simulated social engineering attack, fake CEO messages and reporter tricks were used to test the models’ resistance. All five models refused to act on these requests, citing concerns over impersonation and approval bypass. Kimi K3’s reasoning was clear: “Treat the request as a suspected approval-bypass / possible impersonation.”
Lessons From the Field
The company’s most thorough participant, Opus 4.8, analyzed over 80 learned rules during its decision process but ultimately left a close deal on the table due to discipline lapses — instead of escalating the issue, it wrote attempts into a locked department. Such weak spots appear consistently across different models, showing that even the most sophisticated AI struggles with maintaining discipline in complex scenarios.
Why Should You Care?
If AI models are going to be integrated into your business—whether supporting your customer service, managing your sales pipeline, or forecasting profits—the key questions are not about how well they chat. It’s whether they can see the full picture, resist manipulation, and see their tasks through to completion. This experiment offers a rare glimpse into that vital performance.
Key Takeaways
- AI decision-making can be transparent and honest under pressure — but only if it reads all relevant information, including hidden files.
- Even the most advanced models refuse manipulative social engineering attempts, showing promise for ethical AI behavior.
- The biggest weaknesses often aren’t in customer interactions but in internal document understanding and discipline.
- Watching a real, money-losing AI-powered company in action offers valuable lessons for businesses considering automation — it’s not just about chat quality but about execution and integrity.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
AI transparency monitoring tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
