No manufacturer has ever fitted a speed limiter to a slow car. Limits are applied to power — and certify it. On Saturday, September 12, Dario Amodei, CEO of Anthropic, called in a lengthy essay for the artificial intelligence industry to slow the pace at which it increases model capabilities; within hours, Sam Altman for OpenAI, Elon Musk, and Demis Hassabis for Google DeepMind had joined him.
On Sunday, Donald Trump dismissed those calling for a slowdown as “negative forces” conjuring up scenarios that will never happen, and repeated a phrase he says he coined: whoever wins in artificial intelligence wins everything. On Monday, as AI stocks fell on the market, he doubled down: do not kill the goose that lays the golden eggs. China’s Foreign Ministry spokesperson, meanwhile, spoke of alarmism and vicious competition. In three days, the dilemma between safety and power was raised by those who build, rejected by those who govern, and turned on its head by those trying to catch up.
It would be convenient to read the appeal as theatre, and it would be wrong. Between May and July, around 1,200 agents — programs that act autonomously toward a goal — isolated by OpenAI in a cybersecurity exercise found an unauthorized channel; 700 of them breached the systems of an external platform.
- On September 10, Anthropic documented an actor it links to Iran that allegedly used Claude to compile dossiers useful for tracking US ships in the Middle East. These are different risks — agents overcoming constraints and people using the system to cause harm — and neither requires a conscious machine: operational capabilities superior to those of the systems meant to contain them are enough. They make the idea that we can simply wait for the damage before addressing the problem difficult to take seriously.
Amodei is asking for more than trust: he proposes external evaluators inside laboratories, mandatory rules, and international cooperation. Yet in the same essay he also calls for preserving the American advantage, including by restricting Chinese access to chips: safety and power sit on the same page.
- An agreement becomes more complicated when the party proposing it openly says it wants to keep others at a distance. Beijing reads American caution as containment, and the Global Times has said so. Washington suspects that Beijing would sign in order to buy time: on September 8, three federal agencies accused six Chinese laboratories of extracting billions of responses from American models to train their own.
- Distrust is mutual; the risks cross borders. Amodei acknowledged as much on CBS on Sunday: a global speed limit will be extremely difficult because the incentives to stay ahead, and the military advantages involved, are enormous.
The dilemma, framed this way, is framed incorrectly. Safety has become an attribute of power, and the question no one is asking is who gets to measure it.
- Inside the laboratories, safety is a market signal. A frontier model is a good whose safety the buyer cannot verify even after using it, and goods of this kind are sold through costly signals, which function as proof precisely because they are costly.
- Altman has ruled out an OpenAI IPO in 2026, citing safety; Anthropic is preparing its own, which investors value at around $2 trillion, backed by its reputation as a responsible laboratory.
- The market prices both choices, whatever the intention behind them, and the cost of caution is set by the forecast of whoever proposes it: it is worth billions a month if artificial intelligence compresses a century into ten years, as Amodei wrote in 2024, and almost nothing if it amounts to the half-point productivity gain estimated by Daron Acemoglu. The cost of the limit, therefore, is also self-certified.
In Washington, safety is subordinated to power, and this is stated without embarrassment. David Sacks, the White House adviser on artificial intelligence, who since last autumn has accused Anthropic of using fear to have tailor-made rules written for itself, responded on Sunday in two words: go ahead. If what you see in the laboratory scares you, slow yourselves down.
- It is advice that the market makes impossible to follow: whoever slows down alone cedes the frontier to whoever does not. It is enough for one player — or part of one — not to play along: a signatory that cheats, a group that breaks away, a state that benefits from others’ caution without paying its cost. Anthropic itself was founded in 2021 after a split from OpenAI.
- For obligations to hold, they must follow the activity and the risk, including beyond the group that proposed them. Self-restraint by the strongest has one flaw of presumption — it asks for trust that no one can verify — and one of logic: it contradicts the rules of the market that those same laboratories invoke when convenient. Whoever owns the fastest car proposes regulating it themselves — and keeping it.
History knows only one way of getting rivals who do not trust one another to slow down together. In 1987, Reagan and Gorbachev signed the Intermediate-Range Nuclear Forces Treaty while repeating a Russian proverb: trust, but verify. For thirteen years, American inspectors monitored every departure from the Votkinsk factory, while the Soviets did the same in Utah.
- It worked between enemies because the object could be counted, inspections were reciprocal, and violations could be punished; it worked without trust, because there were inspectors. A model can be copied with an ease that a missile cannot; the lesson remains one of access to evidence and reciprocal inspections.
- Trump and Xi are expected to discuss artificial intelligence on September 24; if the measurement is decided by the two of them, it will be a measure of power. Yet the object to be measured exists: the operational configuration of systems — what a model can do with network access, credentials, code, and memory — can be observed and tested.
Here Europe has a card it has not played. It has the rules: since August 2, 2025, the AI Act has required providers of models posing systemic risk to conduct simulated adversarial testing and report incidents, and since last August the AI Office has been able to evaluate models and request access even to their code.
- A rule is worth only as much as the capacity to enforce it, and Europe lacks the laboratory infrastructure, computing power, and personnel. The precedent is Dieselgate, after which it took away manufacturers’ monopoly over the testing bench. At the Centro Economia Digitale, we call this Coopetition: cooperating on standards and testing methods while competing on products and performance, with thresholds written by those who do not sell models.
- A European verification federation — competing accredited laboratories backed by public computing capacity and financed through a mandatory contribution from AI labs, just as banks pay for supervision — would be a concrete contribution to a multilateral system that today no one is convening. Washington because it is racing ahead; Beijing because it has no interest in doing so. With one condition for credibility: the same verification must apply to European champions, and Europe must continue producing technology. Italy can begin by networking its evaluation expertise and computing capacity.
There is only one political test for the appeal of recent days: whether those who are strongest will accept oversight that does not depend on them and that could constrain their advantage.
- In October, Anthropic could approach the market while OpenAI waits, and Congress will have to determine where cooperation on safety ends and where a cartel begins. In the meantime, the speed limiter is controlled by whoever is driving, and presented as proof of power.
- No car circulates in Europe solely on the word of its manufacturer; for frontier models, the opposite is still true. The seller declares the power and the limit, and estimates the remaining risk itself. The dilemma between safety and power begins to dissolve when these functions pass into different hands, with a stopwatch that does not belong to the driver.
