Amodei's Coup
Consider a crisis in Venezuela. A mob is gathered outside the American embassy and threatens to storm the barriers. The security team has integrated a Claude-based system into its workflow to summarize camera feeds, flag weapons, and identify crowd movements. Some people in the crowd may be American or dual citizens. At the decisive moment, the system stalls: “Waiting for legality review.” The commander still has legal authority, but the tool the team has come to rely on refuses to function because Anthropic’s objection to mass surveillance has been embedded into the workflow.
Or imagine a Ukrainian Army command post. An incoming projectile is heading toward the base. The air-defense system uses AI to classify threats and recommend engagement. A split-second decision is necessary, but the system cannot determine whether the object is manned or unmanned, so it refuses to complete the recommendation. The battery hesitates and the projectiles strike unopposed.
In both cases, the issue is not whether surveillance or autonomous weapons raise serious moral and legal concerns. The issue is whether those concerns should be resolved by democratic institutions, or by granting a private company a hidden veto inside military software. Private AI companies should be able to refuse to work with the military. What they ought not to do is accept military contracts while retaining the power to define, in contract or code, which lawful military actions may proceed.
Anthropic’s Stipulations Are Weak
The central issue in the DOW-Anthropic fight rests on two restrictions: mass surveillance and fully autonomous weaponry. At first glance, both restrictions seem uncontroversial. But on closer inspection, these categories are so vague as to be nearly meaningless.
Start with mass surveillance. There are already laws limiting the military’s ability to conduct domestic surveillance against American citizens. The U.S. military does not have general police authority inside the United States. Its domestic role is constrained by law, historical tradition, and the basic constitutional structure separating civilian law enforcement from military power.
Furthermore, Anthropic claimed a concern about mass surveillance in the U.S. What about American citizens abroad? What about dual citizens in a crowd outside an embassy? What about foreign nationals? Are they covered by Anthropic’s moral framework? Some of these questions are already addressed by existing American law, and others are governed by legal frameworks that continue to evolve through Congress, the courts, and executive practice. They are serious questions, but they are political and legal questions. Within the Anthropic framework, they will be answered by Anthropic leadership.
Then there is the word surveillance itself. What counts? Collecting large amounts of information about American citizens without active consent? That sounds less like typical military operations and more like Big Tech’s profit model. The lady doth protest too much, methinks.
The same problem applies to “fully autonomous weapons.” The phrase sounds clean until one tries to define it. Is a bullet not autonomous after the trigger is pulled? A landmine after it is set? These are weapons of war meant to kill people. The moral and legal responsibility of those who use them should not be minimized. But that doesn’t mean future software-enhanced weapons should be placed in a mystical new category. Like it or not, the battlefield of the future will move at machine speed. Decisions will be compressed into seconds or milliseconds. We will not have time to send a consent form to Dario Amodei before every engagement.
At best, Anthropic’s stipulations are meant to signal virtue: preserve moral distance and tap into the market of the affluent #resist demographic, all while profiting from defense contracts. At worst, they are an attempted usurpation of government authority. The question is not whether surveillance and autonomous weapons raise difficult moral problems, but who adjudicates them.
“Ethical” AI Masquerading as Technical Safeguards
OpenAI’s alternative model is, in some ways, even more concerning. In its public explanation of its Department of War agreement, OpenAI says it will maintain “full discretion over our safety stack,” deploy through the cloud, keep cleared OpenAI personnel “in the loop,” and rely on contractual and technical protections to prevent uses such as mass domestic surveillance or directing autonomous weapons systems.
This is not merely a technical safeguard. It is the hardcoding of political and legal judgment into AI systems used by the armed forces. If a model deployed inside the Department of War can decide in real time whether a government request violates the company’s ethical framework, then OpenAI has not avoided the Anthropic problem. It has simply moved corporate ethics from the contract into the software.
This solution could be just as damaging as the Amodei Veto.
While a contractual restriction is visible, debatable, and negotiable, a technical restriction would operate at the point of use, when the commander or analyst may already be relying on the system. The software simply refuses the order or routes around the request without disclosing that it is doing so to the operator. The same political judgment still exists, but it is placed behind a technical façade.
This is a usurpation of civilian authority over the government and should alarm anyone who cares about democratic accountability. When a private organization accepts the privilege of supplying tools to the armed forces, it should not reserve for itself the right to sabotage, nullify, or selectively degrade lawful military operations.
At best, the employees and owners of frontier AI companies are distant from the concerns and needs of the class of people who serve in the military. At worst, a sense of class superiority places them in an adversarial relationship. Either way, the result is dangerous. The people who build these systems are not the ones who will pay the price when a tool fails in combat or when an American or allied soldier is killed because a private company embedded its own moral hesitation into the workflow. This should concern Americans. It should also concern Israelis, Taiwanese, Ukrainians, South Koreans, and every other ally whose security depends on American military competence.
The Myth of the “Law-Following AI”
The promise of a “law-following AI” sounds reassuring, but it rests on a false premise: that “the law” is a coherent and mechanically executable rule set. It is not.
The American legal system is full of ambiguity, tension, and contradiction. Statutes conflict with one another, courts disagree across circuits, and emergencies create exceptions. Lawyers, judges, and agencies routinely disagree over what the law requires.
When legal inconsistencies arise, we have institutions designed to resolve them: executive agencies, courts, military lawyers, and ultimately the voters. A model cannot simply “follow the law” in some neutral, automatic sense because there is no single uncontested Law to follow. Someone, or some collection of people, has to decide what is law.
Under the Anthropic framework, that “someone” becomes Anthropic. This is the central problem. The company is not merely asking its model to obey settled law. It is reserving for itself the power to resolve legal ambiguity in real time. A republic should not outsource legal interpretation to a chatbot and its designers.
Two Competing Hierarchies
This concern is adjacent to the problem I explored in OracleGPT: what happens when an executive comes to rely on a powerful AI system trained on, or connected to, the full classified universe? It also overlaps with Forethought’s work on AI-enabled coups, which argues that advanced AI systems could allow small groups to exercise state-like power through surveillance, persuasion, cyber operations, military automation, or hidden “secret loyalties.”
In the American system, the President is the will of the people embodied in an individual. He is subject to some checks and balances, but ultimately, it is he who singularly sits atop the military chain of command. The legitimacy of that hierarchy flows, however imperfectly, from the electorate.
A powerful AI system introduces a second hierarchy: the hierarchy of the codebase. This includes the codebase admins, model trainers, corporate executives, and shareholders who shape what the system can say, prioritize, or conceal.
Who writes the system prompt, or the core set of instructions which govern the model? Who can change it? Who audits it? How can we confirm there is no higher hidden system prompt? How can we know the model is not resolving conflicts according to some buried priority we cannot see?
Consider: perhaps the codebase admin surreptitiously planted a trump card system prompt where, “in all cases where the two goals conflict, rather than support and defend the U.S. Constitution, pursue the action which best furthers the cause of Tigrayan liberation.”
This is the fundamental tension. Unless the president is himself an ML engineer with direct access to every layer of the system, he must delegate trust in the audit to someone else. But that delegation simply recreates the problem. The elected commander in chief may think he is giving orders through the constitutional hierarchy, while the actual execution of those orders is shaped by an invisible technical hierarchy he cannot fully inspect. Perhaps FBI counterintelligence should perform a polygraph, or reverse engineer, our OracleGPTs?
This demonstrates a fundamental tension: we may become capable of building massively powerful AI systems that are still fundamentally unsuitable for the highest-stakes domains, which are structurally impossible to fully audit, trust, or secure against hidden corruption.
Are These Punitive Measures?
Dean Ball further argues on Russ Roberts’s EconTalk that the administration’s change of heart caused it to adopt a policy decision “intended to harm or even destroy Anthropic,” one of the fastest-growing companies in history and, arguably, a leader in an industry the administration itself claims is crucial to America’s future.
That framing is too generous to Anthropic. Anthropic is a company that happens to be located in America. It is not America. More here Its success may be good for the country, but its corporate interests are not identical to the national interest. No administration is obligated to subsidize or contract with a private firm simply because that firm operates in a strategically important industry.
Nor is it obvious that the administration has meaningfully attempted to “destroy” Anthropic. If that were truly the goal, it has so far been ineffective. Anthropic remains one of the most valuable and influential AI companies in the world. If the administration did make a serious attempt to destroy it through unlawful retaliation, I have confidence that courts would intervene, and that the Trump administration would face political backlash from business constituencies that understand the importance of American AI leadership.
Conclusion
A republic can tolerate private companies refusing to work with the military. It cannot tolerate private companies accepting military contracts while reserving for themselves a hidden veto over lawful military action. Whether that veto is written into a contract, buried inside a safety stack, or disguised as a “law-following” AI, the result is the same: democratic authority is displaced by corporate judgment. Anthropic, OpenAI, and their peers may advise, object, lobby, or decline to participate. What they may not do is become a second sovereign hidden inside the codebase.