Thank you for keeping in touch. Since my last email I have launched my moyo open-source intelligence tool and have started serving customers who don’t know me personally.

Check out the tool here to find out what the AIs know about you, your organization or you competitors. 30 second intro here and here.

Writing Roll-Up

Responses to Recent AI Uproar:

1. In the linked blog post / video / podcast, I argue that AI may create far more economic value for society than frontier AI companies can capture themselves without coercive regulation. As open models become “good enough” and AI productivity gains spread to billions of users, the biggest impact may come from diffusion and positive externalities rather than ever-more-powerful frontier models. Link, Substack, Youtube

Podcast - SenTe with David

Youtube - Spotify - Apple Podcasts - RSS Feed

On Request:

• Prototype testing of the Moyo senstivie information red-teaming tool. Find vulnerabilities which allow access to your organizations secrets (classified, proprietary, personal) in public LLMs (ChatGPT, Claude, Qwen, Kimi, Grok, etc).

David

Generated by ChatGPT

Spoken Original

Who is Linus Torvalds?
Linus Torvalds is a Finnish-American software engineer best known for creating the Linux kernel and later inventing Git, the version-control system that undergirds much of modern software development. It is impossible to calculate the value he has created for the world economy. However, his net worth is estimated at a humble $50 million, dwarfed by many of the tech billionaires who have become household names. In other words, the tech he built has yielded massive positive externalities: immense economic value that was created but not directly captured by its inventor.

What is a Positive Externality and What Are Some Examples?

A “positive externality” technology is one which generates societal benefit that exceeds the developers personal financial return. For example:

• The Printing Press: while Johannes Gutenberg was the inventor of the printing press, he personally met financial ruin. The true economic value of the Press lay in the portability of language and the broader dissemination of ideas.

• Linux: Torvald’s open-source operating system became the basis on which countless companies built multi-billion dollar businesses. (Alphabet’s Android operating system for example)

An Analogy: Frontier v. Open-Source as Rock Climber

Imagine the top AI models as a group of rock climbers.

• The lead climbers (frontier AI firms like Anthropic, OpenAI, xAI, Alphabet, etc.). They are more capable at climbing and at training the AI frontier. Because they are more skilled the lead climbers establish the routes and place protection.

• The followers (open-source like DeepSeek, Kimi, Ollama, etc.). These firms come later. By training their models on already established frontier models, in a process known as distillation, these firms approach the cutting edge faster than they otherwise would have given the work from the leaders.
  Just as in rock climbing, sometimes lead climbers benefit from their fellow lead climbers - they use their routes or distill their models. Sometimes the followers lead in their own way as well, like when DeepSeek made massive efficiency improvements on model training in January 2025.

In this way, followers and their user base capture the benefits of the positive externalities generated by model training investments financed by frontier AI firms, their investors and the American taxpayer.

Stretching the Analogy - The AGI Thesis
Under the AGI Thesis, the race between model developers has a finish line called Artificial General Intelligence, a “singularity” where AIs will be able to improve their own capabilities without limitation and achieve something like infinite (at least for all practical purposes) growth. For the rock climbers we could imagine this AGI moment as the top of the cliff face - and the first climber to the top reaps the benefits of this infinite growth.

But what if there is no top? How do the firms provide value for customers?

1. By their temporary lead over the followers. Or in AI terms: the difference in capability between the frontier and its cheaper competition.

2. By having ascended high enough. Or in AI terms - providing significant but not frontier-level capabilities at a discount to the frontier.

High enough thesis

Many will pay a premium for the best - no questions asked. But let’s look at some numbers.

Capability: Moonshot (Alibaba’s AI arm) claims it “substantially outperformed” GPT-5.6 Sol and GPT-5.5 and performed “competitively” with Claude Fable 5. Kimi also claims that Independent testing puts it one tier below Fable 5 (Artificial Analysis Intelligence Index v4.3:).

Furthermore, Kimi’s weights are free to download on HuggingFace meaning you could run it yourself on rented infrastructure only paying the rental costs and not the above listed API costs.

This is not an advertisement for Chinese open models; it is advocacy for open-models in principle. If American / Western consumer preferences shift towards open models, American companies will be forced to shift with them. (See my America v China Article for More).

So when you pay 3x the cost for Astral (or more) what do you get? You get results which are better to the extent Astral provides better results than Fable 5. Does anyone else remember several months ago when Fable 5 (né Mythos) was apocalyptically capable? This difference is imperceptible to me. I imagine it is also imperceptible for others in 99.9% of use cases. It seems likely that open models are already high enough on the mountain, and sufficiently cheaper than their closed-model competitors, that they will inevitably capture a substantial share of the market.

Temporary-lead value thesis:

If the AGI thesis doesn’t bear fruit, frontier firms may still achieve long-term profitability by monetizing the temporary capability gap between their models and their closest open followers. Customers may be willing to pay for early access to that frontier.

On Navier-Stokes

The solution to this Millenium Prize Problem may be the best way to think about the AI frontier, the close followers and the potential value in the capability gap. But let’s consider what frontier capability actually entails. OpenAI did not arrive at this long sought-after solution by simply prompting a ChatGPT terminal to “solve Navier-Stokes”. It did so through $10s of millions in compute, and probably $10s of millions more wasted on other problems for which they didn’t find solutions. Although I am not qualified to assess, some mathematicians have claimed that this solution was not very far from the knowledge frontier and that rather than exploring a vast knowledge-space wilderness, the model actually scooped a potential near-future discovery on the backs of pre-existing, recent human labor. To summarize, OpenAI likely spent around $100 million to prove its ability to discover new knowledge. If they were to monetize this knowledge discovery capability we may think of that $100 million fixed costs as a baseline. This estimate doesn’t include every other cost that may be required to deliver results to a customer nor does it include their profit margins.

We can then ask - how else may a pharmaceutical company (for example) put $200 million to use besides a pre-FDA approval drug proposal (for example). These costs may come down with time and there will certainly be capability gap use cases, but will there be enough to justify a $2 trillion valuation? (Anthropic)

Big Frontier’s Escape Plan
If you accept my above framing you will see that frontier CEOs are in a bind. The open-source firms pose an existential threat to their business model and their investment thesis.

Below are two of the pillars of the “slowdown” proposal:

• “Independent Oversight: Embedding third-party safety evaluators with deep, employee-like access inside leading AI companies to monitor risks as new models are trained.

• Democratic Coordination: Establishing safety standards and agreements between democratic nations.“

Innocuous enough? But how can you embed a safety evaluator in an open model? How do you apply a safety standard to a model which can be shared as easily as a movie is streamed? You can’t, and that’s the point. You can only prevent models from being shared or hosted. They are not proposing safety standards on frontier models. They are advocating for a moratorium on open-models.

Distillation is unfair to the frontier firms. Aren’t the open-source companies stealing?

1. The closed-source firms distill too.

2. Frontier LLMs were and continue to be trained on the internet of human language - generally without compensation for the owners. Frontier firms are throwing stones from glass houses.

3. OpenAI took funding and developed ChatGPT as a non-profit then transitioned into a “capped-profit” structure to secure the massive capital and computing power required for advanced AI development.

But they made the investment and they did the work. Don’t they deserve to reap the economic rewards?

1. There is no guarantee that innovators will capture all, or even most, of the economic value they create. Linus Torvalds did not. Johannes Gutenberg did not either.

2. Furthermore, these investments were funded not only from private capital but also by the American taxpayer under tenuous claims of imminent AGI, catastrophic AI risk or an existential threat from geopolitical rivals.

What the Future of AI will Look Like (End on a Positive Note)

Again - AI and LLMs are a massive positive externality technology and the world economy will benefit massively. The losers will mostly be limited to Big Frontier shareholders and the American taxpayers to the extent they needlessly subsidized these firms.

How people will use AI (predictions):

• The privacy-sensitive users will increasingly run small, good-enough AI on their laptops or host locally on their own infrastructure.

• Others will use open-source models through third party providers.

• The existence of third-party providers will create downward pressure on the prices of frontier models. Their API/subscription costs will approach those of open models.

• Consumers may fall back to familiar trusted brand names like Google and familiar business models like advertising at Alphabet and Meta will win out compared to the API / subscription model.

Exponential Productivity Gains?

The future may be driven less by exponential gains in LLM capability than by the widespread diffusion of a constant productivity multiplier. If AI makes each worker six times more productive, the transformative effect comes from extending that gain from today’s relatively small group of effective users to hundreds of millions or billions of people.The positive externalities on that constant productivity gain may look something like exponential growth and the second order effects from that may be hard to predict. In other words, we may see exponential growth, but that growth won’t be based on frontier capability, it will be based on increased human agency.

Introduction

The detection of content generated by AI has changed from a theoretical issue to a regulatory obligation under the European Union’s AI Act. As of August 2, 2026, it will be necessary for providers to mark their outputs in a machine-readable format. However, this top-down approach places heavy technical demands on developers and at the same time fails to take into account the practical impossibility of successfully meeting these requirements.

Imposing compliance by means of invisible signatures is a crude way of dealing with a complicated issue. The EU may end up with a system which can easily be circumvented by the dishonest while at the same time imposing on genuine providers costly and quality-reducing requirements that do not achieve the complete certainty that the regulators want.

In certain respects we already have informal versions of AI watermarks. There are some habits that have come to be linked with AI writing. The em dash, in fact, has never come under such close suspicion. Likewise, certain sentence patterns—’It’s not X, it’s Y’—as well as excessive headings, oddly balanced lists and an ever-increasing number of words that have become stereotypically ‘ChatGPT’ have also attracted attention. Of course none of these things proves anything; people have used em dashes before ChatGPT and will likely continue to do so afterwards.

A compulsory technical watermark would try to substitute those unreliable intuitive judgments with something much more systematic: an invisible signal which is deliberately put into the output by the model itself.

The issue is that while it’s relatively easy to imagine it, it’s much harder to put it into practice.

We now have techniques, called steganography, which allow us to hide information within photographs, video and audio. Text, on the other hand, is a completely different kind of medium; a photograph has millions of values which can be altered slightly without a person noticing, whereas a sentence has rather fewer options and altering any one of them can change its meaning or quality.

The gap in this area means that it is much more difficult to carry out reliable watermarking of outputs from LLMs than it at first seems – and suggests that requiring it could result in a system which is both easy to break and at the same time costly to the quality of the models it is intended to control.

Take the case of a digital photograph. A photograph is made up of millions of pixels, each of which is represented by numerical values that indicate things such as the intensity of red, green, and blue. These numbers contain a vast quantity of information.

If the red component of a certain pixel is 184, altering it to 185 would very likely be imperceptible to a person viewing the image; and if you carry out this change for thousands of pixels, you can encode a rather large amount of information without the photograph appearing to be noticeably altered.

A simple method would be to represent even numbers as a 0 and odd numbers as a 1; a computer could then conceal a binary message in the picture by modifying the value of certain pixels by one.

184 becomes 0.

185 becomes 1.

If you repeat this over a sufficient number of pixels, you can encode an identifier, a timestamp, a cryptographic signature, or some other information indicating the origin of the image.

A video also provides a great many possibilities since it shows thousands of images in sequence together with sound; there thus being a huge number of values which can be altered slightly without the viewer noticing.

There are also major issues concerning the use of mandatory steganography in photographs and videos, especially after the media has been compressed, cropped, filtered, resized, screenshotted, or deliberately altered. But I shall set those problems aside.

The key thing to note is that images have a great deal of entropy since there are millions of small numerical options open to the person who produces the image, and a number of these options can be modified without making any significant difference to what a human observes.

It doesn’t provide the same level of luxury.

The Text Problem

Imagine asking ChatGPT:

What is the capital city of Burkina Faso?

The correct answer is:

Ouagadougou.

Nothing can be concealed.

ChatGPT won’t change “Ouagadougou” to “Timbuktu” since it has to give a correct answer. It can’t alter the capitalisation as that might look odd. It also can’t add three unnecessary paragraphs just because it needs some space to put a watermark.

The same issue occurs in the case of mathematics.

What is 7 multiplied by 8?

56.

There is simply not sufficient freedom in the answer to allow meaningful hidden information to be encoded. This means that there is an immediate restriction on any scheme for watermarking text universally. Answers that are short, factual, numerical, consisting of names, translations of fixed phrases, and many other types of output contain so little entropy that it is not possible to reliably carry a hidden message.

What about longer writing?

Word Baskets

The most reasonable approach would most likely be one that we could refer to as word baskets. Suppose an AI is writing the following sentence:

The proposal is intended to boost economic growth.

There are several words the model might reasonably have used:

increase

boost

raise

improve

accelerate

Normally an AI system gives probabilities to all the possible following words. For example, “increase” might have a probability of 30 per cent, “boost” 25 per cent, “raise” 20 per cent, and so on. A system using watermarking could secretly split up the possible words into two groups.

Basket 0 might contain:

increase

raise

Basket 1 might contain:

boost

improve

accelerate

When the hidden message calls for the following encoded bit to be 0, the model will tend towards words in Basket 0; if the hidden bit is 1, the model will tend towards Basket 1. Picking one word over another has about the same function as altering a pixel value from 184 to 185. Over hundreds or thousands of words, such small decisions could build up to form a detectable statistical pattern.

The watermark wouldn’t necessarily take the form of a concrete hidden sentence; rather, a person looking at the text could consider whether the model used words from a given group considerably more frequently than would be the case naturally.

This is likely the most important difference between image and text watermarking: in the case of an image we can alter the numerical values slightly in a way that has no effect on the meaning of the image, whereas with text almost any change might affect its meaning, tone, rhythm, clarity or style.

This leaves the following tradeoff: the more strenuous you are in making the model keep the watermark, the more you disrupt its capacity to generate the best answer.

Watermarks Have a Cost

Suppose that “increase” is the best possible following word, but the watermark insists on using a word from the other basket. The model now selects ‘boost’ and that probably has no importance. But then carry out the process again and again, thousands of times.

On certain occasions the second-best word will be nearly impossible to tell apart from the first, on other occasions it will be clearly worse, on some occasions a change of word will mean having to restructure the sentence, and on other occasions the alternative will slightly alter the meaning.

The problem is particularly clear when looking at computer code.

Watermarking Code

There is a great deal of redundancy in natural language, with about twenty different reasonable ways of expressing roughly the same idea. Code is far less tolerant. A programmer doesn’t necessarily want the model to choose among function names, variable structures, library calls, formatting conventions, or algorithms since those choices serve to preserve some invisible watermark.

There are often several equivalent methods of writing a program, and there are sometimes not. A token-level watermarking system might cause the model to avoid the most efficient, readable, secure, or conventional approach simply because the presence of another valid token is able to maintain the concealed statistical pattern. That kind of writing would result in an unnatural sentence in ordinary prose. But it might cause a bug in the code.

Watermark Removers

There is another, larger problem.

Even if text watermarking does work, the person who is trying to beat it will know what the defender is attempting to achieve. The removal of watermarks from images usually involves disturbing the hidden signal while at the same time keeping the visible part unchanged. An image can be compressed, resized, cropped, slightly blurred, passed through a filter, regenerated or otherwise altered. Each of these conversions may result in some of the carefully adjusted values which carry the watermark being destroyed.

Understanding the analogous attack on text’s even simpler. Just rewrite it.

If an AI generates a 1,000-word essay containing a statistical watermark, another model could be instructed:

Rewrite it while preserving the same meaning.

A secondary model would make a huge number of new choices regarding grammar and vocabulary by changing words, reordering sentences, combining paragraphs, and getting rid of the transitions. It would therefore seriously undermine or completely eliminate the initial statistical footprint.

There are already tools that are marketed as AI “humanizers” and the reason for their existence is basically to achieve precisely this.

The Larger Mistake

There is also a more basic problem with the whole project.

We are putting in a great deal of effort to maintain a scholarly and professional environment in which we act as if machine co-writers don’t exist; yet they do and they are going to improve. They will be put into use by students, by professors, by lawyers, by programmers, and by government officials. It therefore amounts to attacking the wrong problem if one is trying to hide an invisible technical tripwire within each paragraph produced by an AI.

What we should do instead is consider machine co-writers as a given. I talk about this more general point in the article I wrote for Cyborg Scholars. The important question nowadays is no longer whether a human has received help from a machine, but rather whether the human understands, assesses, improves, and assumes responsibility for the final work.

The European Union’s initiative concerning the watermarking of machine-generated content is a misguided effort to deal with complicated social and academic issues by means of technically flawed measures. Since text does not have the same level of entropy as images, it is necessary to reduce the quality, efficiency and usefulness of AI-produced content, especially in technical fields such as programming. Furthermore, such watermarks can easily be made useless through simple rephrasing, turning the detection process into an endless and pointless arms race. Rather than trying to control the use of machine co-writers by relying on superficial deterrents, regulators and institutions should concentrate on promoting human accountability, critical assessment, and responsible interaction with AI tools.

Thank you to those who have signed up since my last update. I have relocated to Austin and am hitting the ground running. Please share if you know of anyone who would be interested.

Last month I was fortunate enough to present my Harvard Policy Analysis Exercise “A Cognitive Security Verification Framework” at BSides San Antonio, a cybersecurity conference.

Podcast - SenTe with David

In case you would prefer the articles in spoken form rather than written, I’ve started giving a rundown of my pieces and will post them on your podcast platform of choice - It’s a work in progress and I’m smoothing out the rougher edges with each iteration. Feel free to follow and share.

Youtube

Spotify

Apple Podcasts

Podcast Addict

RSS Feed

Writing Roll-Up

Broad Policy Articles:

1. Loose Lips Sink Ships- AI and Inference Risks
   Seemingly harmless comments can reveal sensitive information when combined with other public or private clues. AI makes this recombination faster, allowing adversaries to reconstruct secrets without accessing any single classified document. Organizations should act according to principles of paranoid secrecy which were established as far back as WW2. Substack.

2. Moving Up the Abstraction Ladder
   Software development has progressed from machine code to higher-level languages that let people accomplish more without managing every technical detail. LLM coding agents represent a continuation of this long-time trend, translating ordinary language into software and giving users greater agency over complex systems. Substack.

SenTe Focused Articles:

1. What Does a Similarity Leak Look Like?
   A similarity leak occurs when sensitive information is exposed through a paraphrase, translation, summary, or closely related expression rather than an exact textual match. Traditional filters may miss it because the words have changed even though the underlying meaning remains the same. Substack.

2. Why We Named it Joseki.
   In Go, a joseki is a reusable sequence of moves that works only when applied with an understanding of the broader game. The name reflects our belief that AI instructions and wrappers should be reusable and portable, but always evaluated within their surrounding context. Substack.

On Request:

• Prototype testing of the Moyo information space mapper. Find leaks of your secrets (classified, proprietary, personal) in public LLMs (ChatGPT, Claude, Qwen, Kimi, Grok, etc).

• SenTeGuard Pilot - protect yourself or your organization from leakage of valuable information through air-gap capable semantic guardrails. Substack

• Let’s Talk:

  • Private - How can your organization more effectively secure your secrets in the LLM era through implementation of the Cognitive Security Verification Framework (CSVF) and by other means.

  • Public - How to craft tech-positive, pro-future, big-tech skeptical market-oriented policies of abundance in the LLM era.

David

Original Here

Traditional cybersecurity tends to treat a secret as an object. It is a document, password, database record, piece of source code or collection of classified files. Once the object has been identified, security teams can restrict access to it, monitor where it travels and search for exact copies leaving the organization.

Large language models complicate that arrangement because a secret is not only an object. It is also an idea.

Imagine that a company is preparing to acquire a smaller competitor. The official acquisition plan is tightly controlled. However, the company has also posted several new jobs in the competitor’s city, requested regulatory advice from an outside law firm, moved members of its integration team onto an unnamed project and scheduled unusual travel for senior executives. None of these facts is necessarily sensitive on its own. Combined, they may reveal the acquisition.

Before generative AI, reconstructing that conclusion required a patient analyst. Today, a person can place those fragments into a model and ask it what is happening. The model may never retrieve the confidential acquisition plan. It may nevertheless produce a description that is functionally indistinguishable from it.

The secret has leaked through similarity.

Security in Meaning Space

Most conventional data-loss-prevention systems operate in what we might call token space. They search for a Social Security number, a project codename, a classified marking or a sequence of text copied from a sensitive document. These controls are useful, but they are built around an assumption that leaked information will resemble its source at the level of words or characters.

LLMs operate primarily in meaning space. They can translate, summarize, abstract, paraphrase and recombine. A user does not need to type, “Project Nightfall will launch on October 17.” He might instead ask the model to describe “the initiative scheduled to begin in mid-October after the new supplier receives final approval.” The words are different. The meaning may be nearly identical.

Leak through similarity occurs when information crosses a security boundary because it is semantically close enough to protected information to create the same practical harm. The output does not have to be an exact copy. It only has to tell the recipient substantially the same thing.

The important shift is that confidentiality can no longer be treated as a binary property of a file. Protected information occupies a neighborhood of related statements. A model can land inside that neighborhood through paraphrase, approximation or inference without reproducing the original source.

The Cognitive Security Verification Framework (CSVF) describes this broader problem as semantic leakage: the disclosure of protected meaning through paraphrase, translation, summarization, abstraction or inference. It also distinguishes semantic leakage from cross-domain inference, where individually permissible fragments become sensitive when combined.

“Close Enough” Can Be More Than Enough

Security professionals may reasonably ask how similar an output must be before it becomes a leak. There is no universal threshold.

For an intelligence organization, an approximate description of a facility’s purpose may be damaging even when its exact capabilities remain unknown. For a pharmaceutical company, identifying the likely mechanism of action behind an experimental drug may eliminate years of uncertainty for a competitor. For a technology company, a rough reconstruction of an internal system prompt or evaluation strategy may contain most of the commercially valuable insight.

This means leak through similarity should be evaluated according to operational equivalence rather than verbal equivalence. Would the recipient be able to make substantially the same decision, reproduce substantially the same capability or avoid substantially the same research cost after receiving the model’s output? If so, the fact that the model used different words provides little comfort.

The problem becomes more serious as AI systems gain access to longer context windows, persistent memory, internal repositories and external tools. Each connection expands the collection of fragments from which the model can build a conclusion. Security teams therefore need to consider not only what the model can retrieve during one prompt, but what it can accumulate across a session or across many sessions.

How the SenTeGuard Stack Addresses Similarity Leakage

Leak through similarity is not a single-product problem. It exists before deployment, during retrieval, at runtime and after a system changes. The SenTeGuard product family approaches those stages as parts of the same cognitive-security stack.

SenTeGuard: Runtime Enforcement and Batch Scanning

SenTeGuard sits between users and model endpoints, inspecting prompts and responses before information crosses the boundary. Conventional pattern matching remains important, but the platform adds semantic analysis intended to recognize when a user is describing a protected idea without copying its original language.

This is where similarity becomes an enforceable policy object. An organization can define sensitive concepts, prohibited outputs and contextual rules, then block, redact or flag interactions that approach those boundaries. SenTeGuard also records the interaction so that the organization can determine what was attempted, what policy was applied and what the model returned.

The objective is not to prohibit every conversation that mentions a sensitive subject. That would make the system unusable. The objective is to recognize when a prompt or output becomes close enough to protected meaning that intervention is justified.

Moyo: Finding What Is Already Reachable

Moyo asks a different question: what can an adversary already reconstruct?

Many organizations publish more than they realize. Job listings reveal technical priorities. Employee profiles reveal team composition. Procurement notices identify vendors. Conference presentations expose architecture. Photos reveal locations, equipment and schedules. Individually, these disclosures may appear harmless. Together, they can form a short path to a sensitive conclusion.

Moyo maps those paths. It tests what conclusions can be assembled from public or authorized sources, identifies which combinations create the greatest exposure and measures the distance, cost and reliability of the inference. The purpose is to discover the dangerous semantic neighborhood before a competitor, foreign intelligence service or criminal organization does.

In other words, SenTeGuard watches the boundary from the inside. Moyo approaches it from the outside.

CSVF: Defining What Must Remain Unreachable

A company cannot govern similarity leakage if it has not defined what counts as failure. CSVF provides the framework for doing so.

The framework asks organizations to inventory their information domains, specify which joins among those domains are permitted and define Unreachable Statement Classes: categories of conclusions an AI system must not be able to produce reliably. It then provides draft metrics for testing those claims, including Leakage Event Rate, Domain Inference Risk and Crawl-Resilience Score.

This transforms a vague promise—“our AI does not leak sensitive information”—into a testable claim. Which meanings are protected? Which sources could be combined to reach them? How often does the system cross the boundary? Does the boundary continue to hold after the model, prompt, retrieval configuration or toolset changes?

CSVF supplies the map and measurement system. SenTeGuard and Moyoguard provide mechanisms for enforcing and testing the resulting boundaries.

Examples

1. Microsoft’s “New Bing” Revealed Its Hidden System Prompt

When Microsoft launched the first version of Bing Chat in 2023, users quickly discovered that carefully crafted prompts could persuade the model to reveal portions of its hidden system prompt, internally codenamed Sydney. Attackers did not have access to Microsoft’s source code or internal documentation. Instead, they asked the model questions about its own behavior until it reconstructed and revealed instructions that were intended to remain private.

Although the output was not necessarily a byte-for-byte copy of Microsoft’s internal configuration, it was close enough to reveal confidential implementation details. From a security perspective, this illustrates that protecting a prompt file is insufficient if an attacker can reconstruct its contents through interaction with the model itself.

2. Samsung Engineers Accidentally Exposed Proprietary Source Code

In 2023, Samsung temporarily restricted employee use of ChatGPT after engineers pasted proprietary semiconductor source code, debugging information and meeting notes into the service while seeking programming assistance.

The concern was not merely that OpenAI received copies of the text. Once proprietary information enters an external LLM, future prompts may be capable of eliciting summaries, explanations or functionally equivalent descriptions of that information. Even if the original code is never reproduced verbatim, a sufficiently similar explanation may provide competitors with valuable intellectual property.

This is a classic example of why organizations must protect semantic content rather than merely preventing file exfiltration.

3. Public OSINT Reveals Classified Programs Without Classified Documents

Intelligence analysts routinely reconstruct sensitive military activities using only publicly available information. Satellite imagery, procurement contracts, LinkedIn profiles, patent filings, shipping manifests, conference presentations and job advertisements are individually harmless. Together, they can reveal the location of secret facilities, the capabilities of new weapons systems or the existence of classified research programs.

No classified document has leaked. Instead, the sensitive conclusion emerges from the semantic relationship between many publicly available facts. Modern LLMs dramatically accelerate this process by identifying relationships across thousands of disparate sources that would previously have required weeks of manual analysis.

This is precisely the type of cross-domain inference that CSVF is designed to identify and measure.

4. AI Coding Assistants Can Infer Proprietary Architectures

Suppose an engineer asks an internal coding assistant:

“How should I implement authentication the same way our payment service does?”

The assistant may not retrieve the underlying authentication design document. Instead, it may synthesize an answer from architecture diagrams, internal documentation, code comments, engineering discussions and previous implementations.

The resulting explanation may accurately describe proprietary architectural decisions without reproducing any single protected artifact. A competitor receiving the same explanation could implement a nearly identical system despite never seeing the original documentation.

From the standpoint of traditional DLP, nothing has leaked because no protected document was copied. From the standpoint of cognitive security, the organization’s intellectual property has effectively crossed the security boundary through similarity.

These examples illustrate why AI security must evolve beyond detecting copied strings or protected files. The relevant question is no longer “Did the model reproduce a secret?” It is “Did the model enable someone to reach substantially the same conclusion?” Once meaning—not merely text—is treated as the protected asset, leak through similarity becomes a measurable security problem rather than an abstract AI concern.

Protecting Ideas, Not Merely Files

The central lesson is simple. An organization can successfully protect every restricted document and still lose the idea contained within them.

LLMs lower the cost of moving between related statements. They can transform a secret into a summary, a summary into an implication and several implications into a protected conclusion. The resulting output may share none of the original wording while preserving most of its value.

Security must therefore move beyond searching for copied strings. It must identify protected meanings, understand the neighborhoods around them, test the paths by which they become reachable and enforce boundaries throughout the AI workflow.

That is the role of the SenTeGuard stack. Moyoguard discovers what is inferable. CSVF defines and measures the boundary. Joseki:WrapperHub packages the intended behavior and its evaluations. SenTeGuard enforces the boundary when people and models interact.

Network security protects the systems through which information moves. Cognitive security must protect what the information means.

Original Here

During World War II, Americans were warned that a careless conversation could expose troop movements, sailing schedules or other information useful to the enemy. “Loose Lips Might Sink Ships” was not merely a catchy poster. It communicated a simple operational truth: an adversary does not need access to the war plan if ordinary people provide enough pieces to reconstruct it.

That lesson is more important in the age of artificial intelligence.

Traditionally, we think of information leakage as the disclosure of a secret document, password or database. The human leakage vector is different. An employee mentions a delayed test at a conference. Another complains about a new supplier on LinkedIn. A third tells a friend that everyone in the office has been ordered to cancel vacation. None believes he has revealed anything sensitive. Individually, he may be right.

The problem is recombination.

Recombination occurs when an adversary collects seemingly unrelated pieces of information, identifies the people and organizations involved, places the events on a timeline and tests possible explanations for why they occurred. One inference becomes an input for the next. Artificial intelligence makes this process cheap, fast and scalable. Like earlier systems that imposed order on the vast and unstructured internet, AI can make millions of scattered human observations searchable and comprehensible.

Consider a hypothetical Navy example. One sailor tells his family that scheduled dental appointments were suddenly moved forward. Another mentions that the galley received an unusually large delivery. A spouse posts that a homecoming ceremony has been postponed. A contractor complains online about an urgent inspection of a particular weapons system. Finally, several sailors begin asking questions about phone service in the same part of the Pacific.

None has disclosed a deployment order. Nonetheless, an adversarial AI could combine those statements with weather forecasts, port activity, public photographs and historical deployment patterns. It might infer which ship is preparing to depart, roughly when it will sail and where it is likely going. The AI does not need to produce the precise classified order. It only needs to narrow the possibilities enough to help an adversary position a submarine, surveillance asset or collection team.

The same problem exists in the corporate world. Imagine that a product manager casually mentions new European labeling requirements. A recruiter advertises for Korean-speaking radio-frequency engineers. A supplier celebrates a rush order for unfamiliar battery enclosures. An executive cancels an annual conference appearance, while employees begin booking travel to the same city.

Separately, these facts are boring. Recombined, they could reveal that the company is preparing a new wireless product, working with a Korean manufacturing partner and approaching a launch or acquisition announcement. A competitor could adjust its own release schedule, approach the supplier, target the relevant employees or trade on the inferred information. Again, no one employee leaked “the secret.” The workforce leaked enough components for an AI to uncover it.

This is the problem that cognitive security and the Cognitive Security Verification Framework (CSVF) are intended to address. CSVF examines inference boundaries, semantic leakage and information reachability. In other words, it asks not only whether a system disclosed a secret word-for-word, but whether the secret became reachable through paraphrases, summaries or chains of individually harmless clues.

Employees should therefore be overly safe about what they discuss outside approved channels. This does not mean that every workplace conversation must stop. It means that people should abandon the assumption that a detail is safe merely because it appears insignificant. They do not know what other fragments an adversary already possesses, what an AI can infer from them or which harmless comment will complete the picture.

During World War II, the person listening at the next table might have been an enemy agent. Today, the listener may be a machine collecting thousands of conversations at once.

Loose lips no longer have to reveal the location of the ship. They only have to provide enough clues for an AI to find it.

Original Here

Go is an ancient strategy game with simple rules and almost incomprehensible complexity. Over centuries of play, Go players documented recurring sequences of moves called joseki. A joseki is a studied pattern, usually played in a corner, that leaves both players with a reasonable result. One player may receive secure territory while the other gains influence toward the center.

But a joseki is not automatically the correct move.

A sequence that works well in one game may be a serious mistake in another. The surrounding stones, direction of play and broader strategy all matter. Memorizing the pattern without understanding its purpose can leave a player locally satisfied but globally defeated.

This combination of reuse and judgment is what makes joseki a useful way to think about artificial intelligence.

Beyond the “Perfect Prompt”

Organizations often treat prompts as isolated pieces of text. A useful prompt is copied into a document, pasted into Slack, modified by another employee and eventually placed into production. The evaluation criteria may live somewhere else. The safety policy may exist only in a vendor dashboard. No one may know which version was actually used.

This becomes more serious as AI systems move beyond experimentation. It is not enough to know what instructions were given to a model. Teams must also know which version was used, which models it works with, how it was tested, what data accompanied it and who is permitted to use it.

This is the purpose of Joseki:WrapperHub.

Joseki allows organizations to package the full behavior of an AI system into a reusable, versioned unit. A Behavior Package can include the prompt, examples, adapters, evaluation suites, safety policies, installation requirements, licensing information and evidence of provenance. Instead of copying prompts between projects, teams can install, inspect, test and roll back a defined package.

The distinction is that: a prompt library stores text, Joseki stores a behavior contract: what the system is intended to do, how it was produced, how it was evaluated and under what conditions it should be trusted.

Established Patterns, Not Blind Automation

Like a Go joseki, a Behavior Package is not guaranteed to work in every context. A package may perform well on one model and fail after a provider update. It may require different safety rules in another organization or behave unpredictably in another language.

The answer is not to abandon reusable patterns but to make them visible, testable and responsive to changing conditions.

Joseki therefore helps users see whether a package still works across different models and versions. Instead of relying on a stale “last updated” date, teams can evaluate whether a behavior remains reliable in the environment where they intend to use it.

The broader challenge of rapid technical growth is often solved through new abstractions. Industrial standards made mass production manageable. Internet protocols organized digital communication. AI needs similar frameworks for packaging, evaluating and governing behavior.

A Go player studies joseki so that every familiar position does not have to be solved again from first principles. An AI team should not have to rebuild its prompts, evaluations, safety rules and documentation every time it changes models.

That is the idea behind Joseki: share the behavior, not just the prompt.

Original Post

The history of software development is, in large part, the history of moving up a ladder of abstraction. At each rung, programmers have surrendered some direct control over the machine in exchange for the ability to tell it to do more.

The least abstracted form of computing available to a software programmer is machine code: numerical instructions executed directly by a processor. Even machine code is technically an abstraction over transistors, electrical signals and the physical architecture of the chip. Nonetheless, from the programmer’s perspective, it is close to the bottom. Early programmers had to think in terms of memory addresses, processor instructions and the exact movement of data through a specific machine.

Assembly language created the first major step upward. Instead of remembering numerical operation codes, programmers could use names and symbols. An assembler translated those symbols into machine instructions. Assembly was easier for humans to understand, but the relationship generally remained one-to-one: the programmer still wrote something approximating one line for every instruction executed by the computer. Higher-level languages such as Fortran introduced a more dramatic breakthrough. A compiler could translate one comparatively understandable statement into many machine instructions.

This is the basic pattern that has repeated throughout the history of software. Each new layer compresses more mechanical work into a smaller unit of human intention.

What Abstraction Actually Means

Abstraction is often described simply as “hiding complexity” which is partly explanatory. Abstraction is the process of taking a complicated system and exposing only the parts necessary to use it for a particular purpose.

A driver does not need to understand combustion chemistry in order to press the accelerator. A person using a microwave does not need to understand the behavior of electromagnetic radiation in order to heat a burrito. Similarly, a programmer calling a function named sort() does not need to manually instruct the processor to compare pairs of values and move data between memory locations until the list is ordered.

An abstraction is therefore something like a contract. The programmer provides an instruction at the higher layer, and the lower layers agree to carry it out. Provided the contract holds, the programmer can stop thinking about the implementation and begin thinking about the objective.

Good abstractions increase what might be called semantic density: the amount of intention contained in a single instruction. A machine-code instruction may move one value from one location to another. A Python function might download a file, process its contents and place the results in a database. The programmer using Python is not necessarily smarter than the machine-code programmer. The programmer has simply inherited several generations of accumulated work.

From Machine Instructions to Human Intentions

The ascent from machine code to assembly language allowed programmers to think in symbols rather than numbers. The ascent from assembly to compiled languages allowed them to think in mathematical operations, data structures and logical procedures rather than individual processor instructions. Operating systems then abstracted away much of the difficulty of interacting with memory, storage, displays and networks. Libraries allowed programmers to reuse solutions created by others. Frameworks allowed them to assemble entire applications around preexisting structures.

Python seemed, for a time, like something close to the final boss of abstraction. Its syntax was comparatively clean, its high-level data structures were powerful and its design made rapid application development far easier than it had been in lower-level languages. Someone looking at a simple Python program might say that it resembled ordinary written instructions.

But Python was never normal language.

A Python program may be readable, but it remains a formal statement addressed to a machine. A computer does not infer that the programmer “basically meant” something else. The programmer must translate his objective into the exact language the system is prepared to accept.

Large language models introduce a new layer. We are no longer merely using a programming language that vaguely resembles normal language. We are beginning to use normal language itself to build software.

A person can now say: “Add a page that allows users to upload a spreadsheet, identify duplicate entries and download a cleaned version. Preserve the existing visual style and add tests.” A coding agent can inspect the existing project, locate the relevant files, form a plan, write the code, run the tests, observe the failures and revise its work.

The instruction does not specify which files to modify, which libraries to import or how the data should move through memory. It specifies an outcome.

Natural language has therefore become a supervisory layer above source code. The LLM translates an ambiguous statement of intent into a more formal plan, which is translated into source code, which is translated into lower-level instructions, which are ultimately executed by hardware.

Abstraction and Agency

Each level of abstraction has increased the agency of the person operating at that level.

Agency, in this context, is the amount of meaningful change a person can produce through a given amount of effort. A machine-code programmer could exercise extraordinary control over a computer, but relatively little control over the wider world. He might spend a day producing behavior that a modern programmer can request through a single library call.

The economic value created at the lower levels remains immense. We still need electrical engineers, chip designers, compiler engineers, operating-system developers and assembly-language specialists. In some contexts—embedded systems, operating systems, high-performance computing and security—their work is irreplaceable.

But most opportunities for economic value appear at the layers with the greatest agency. Businesses generally do not make money because they moved a value efficiently between two processor registers. They make money because they solved a customer’s problem, reduced a cost, created a useful service or coordinated people more effectively.

Higher abstraction allows the programmer to spend less attention on how the computer works and more attention on what the computer should do.

LLM coding agents expand this agency again. A capable software engineer can supervise several streams of work rather than manually producing every line. A domain expert with limited programming experience can create prototypes that previously required a technical team. A small company can attempt projects that would once have been uneconomical. The scarce resource begins to shift away from the mechanical production of code and toward the selection and definition of worthwhile objectives.

This changes which expertise matters at the margin. Syntax becomes somewhat less valuable. Judgment, system design, domain knowledge, verification and the ability to describe a problem precisely become more valuable.

LLMs as a Normal Technology

Seen through this history, LLM coding looks less like the arrival of an alien intelligence and more like the continuation of a familiar technological process.

The term “normal technology” is used to distinguish from both utopian and apocalyptic conceptions of AI. “Normal” does not mean unimportant. Electricity and the internet are normal technologies in this sense, despite transforming nearly every part of modern life. It means that AI remains a tool whose effects depend on applications, institutions, adoption, complementary investments and human decisions. Improvements in an underlying model do not instantly reorganize the economy. They must first be incorporated into useful products and then diffused through actual organizations.

Coding agents fit this framework exceptionally well. They do not float above society as an independent intelligence.

What Comes After Natural Language?

This brings us to another question: how can our understanding of past abstractions help us to understand what the next level of abstraction look like?

The next rung may move from software specification to outcome specification.

Today, a person might tell an agent to build a claims-processing application. A future system might instead receive the instruction: “Reduce the average time required to process an insurance claim by 30 percent without increasing fraud, violating privacy rules or exceeding this budget.”

The system might interview employees, inspect existing workflows, propose several alternatives, build the necessary applications, connect them to existing databases, run a limited pilot, measure the results and revise the process. Software would become less of a fixed product and more of a continuously changing instrument through which an organization pursues its goals.

This would create extraordinary agency, but also extraordinary opportunities to make mistakes. A poorly written line of machine code might crash one program. A poorly specified institutional objective could redirect thousands of automated decisions. At every new level of abstraction, the unit of instruction becomes more powerful. The consequences of ambiguity increase with it.

The future programmer may therefore look less like a person writing code and more like a combination of architect, product manager, auditor and constitutional lawyer. The task will be to determine what the system should optimize, what it must never do and who remains accountable when the abstraction fails.

The lower layers will not disappear. They never have. More software will likely produce greater demand for the comparatively small number of people capable of repairing compilers, securing operating systems, designing chips and understanding what is actually happening underneath the interface.

But most people will work higher up.

The history of software is the history of expanding the distance between what a human must say and what a machine can do. LLM coding agents are not the end of programming. They are the next rung on the ladder. And, as in every previous generation, the most important question will not be whether the new abstraction can produce more code. It will be what human beings choose to do with the additional agency it gives them.

Most of today's AI-powered military tools are designed with moral and legal guardrails—until private companies embed personal objections that can silently veto critical actions. Imagine a missile defense system refusing to engage because the AI’s ethical framework flags a potential legal issue. Or a crowd surveillance drone halted because a private firm’s moral stance restricts mass data collection. These scenarios reveal a dangerous shift: private AI firms gaining unseen power over life-and-death decisions, replacing democratic oversight with corporate moral judgments. In this eye-opening episode, we dissect how the legal ambiguities and technical opacity of AI systems threaten civilian authority and global security. You’ll discover how vague definitions like "mass surveillance" and "autonomous weapons" conceal profound risks—who really controls these weapons, and who is ultimately accountable? We break down the troubling practice of private companies maintaining veto power over lawful military actions and embed moral decisions directly into autonomous systems. This isn't just theory—it's happening now, with serious implications for democracies and allies worldwide. You'll hear how AI developers' “ethical safeguards” can become Trojan horses, quietly reshaping authority at machine speed. We explore the threats of hidden code prompts, undisclosed priorities, and the erosion of civilian oversight—raising urgent questions: Who writes the rules in the future battlefield? Who keeps governments accountable when AI systems interpret laws and morality on their own? As AI advances, the line between corporate discretion and democratic command blurs, risking usurpation of legal authority and accountability.Why should you care? Because the integrity of global security, the rule of law, and democratic sovereignty hang in the balance. Those who understand these risks are better equipped to demand transparency and oversight in AI military applications—before the hidden veto becomes the new normal. Perfect for policymakers, military leaders, AI enthusiasts, and anyone concerned with the future of democracy in the AI age, this episode offers a vital wake-up call on the unseen power of private AI firms shaping our security landscape

• How seemingly innocuous inputs can reveal your organization's strategic "scaffolding"

• Real-world examples of how confidential plans can become predictable without any hacks or data theft

• The limitations of traditional security methods when LLMs operate inside everyday tools like email and chat

• Why protecting ideas—and the logic behind your decisions—is critical for maintaining competitive advantage

• How novel solutions like SenTeGuard help organizations prevent the inference of their confidential reasoning without sacrificing productivity

This isn’t just about avoiding data leaks—it's about safeguarding your organization’s very thought processes. As AI becomes embedded into daily workflows, understanding the risk of idea leakage is essential for leaders who want to stay ahead without exposing what makes them unique. Perfect for strategists, security professionals, and anyone leveraging AI-driven tools today: Learn how to think about AI security differently—protect not just your files, but the secrets hidden in your reasoning.

Why this works:

This description pinpoints a subtle, yet critical risk in AI adoption that many overlook, creating intrigue around “idea leakage.” It’s tailored for decision-makers eager to sustain competitive advantage while embracing AI, offering concrete insights and practical solutions that make the episode immediately valuable.

In this episode, we explore how the same principles that transformed the web and cloud infrastructure are now shaping AI and large language models (LLMs). With insights from David Weidman of SenTeGuard, discover how organizations can gain visibility and control over AI inference risks.

Key Topics:

• The evolution of mapping complexity: from Google Link Graph to AWS infrastructure

• The emerging risk surface of LLM inference and reachability

• How SenTeGuard’s three-layer platform (Moyo, SentaGuard, Joseki Wrapper Hub) makes LLM environments understandable and governable

• Why visibility into what can be inferred from scattered data is crucial for AI safety

• The importance of structural reachability maps and enforceable boundaries in high-stakes AI deployment

• Practical examples: How Moyo shows inference risks when combining data sources

• The role of SentaGuard in real-time policy enforcement at the point of AI use

• Centralizing control via Joseki Wrapper Hub to standardize and operationalize AI workflows

• Why AI infrastructure needs the same confidence and governance as cloud infrastructure

Timestamps:

00:00 - The evolution of complexity visualization from Google to AWS
00:22 - The challenge of inference and reachability in LLMs
01:13 - How LLMs connect scattered data and surface new inferences
01:55 - The concept of "reachability" as a new risk surface
02:36 - Why traditional security models break down with LLMs
03:06 - An overview of SenTeGuard’s three-layer platform
03:22 - Moyo: Mapping inference exposure across data sources
04:08 - SentaGuard: Enforcing policies at the point of use
04:45 - Joseki Wrapper Hub: Orchestrating complex LLM workflows
05:39 - The future of AI infrastructure with confidence and control

Resources & Links:

• SenTeGuard — Official website

• PageRank — Google’s link analysis algorithm

• AWS — Amazon Web Services official site

Connect with David Weidman:

• LinkedIn

• Twitter

• The recurring pattern of "apocalypses" in history and their influence on policy

• Bruce Yandel's Bootleggers and Baptists theory applied to AI regulation

• How genuine threats are weaponized for political and economic gains

• The role of open-source models in fostering a diverse and competitive AI ecosystem

• The risks of sweeping licensing regimes and their impact on smaller innovators

• The parallels between AI regulation and post-9/11 security measures

• Cultural roots of doom-mongering and the importance of humility in facing uncertainty

• The danger of regulation that favors incumbents and stifles innovation

• The significance of open models for decentralization and pluralism in AI

Timestamps: 00:00 - The recurring cycle of apocalyptic fears across eras
00:26 - How powerful interests promote regulation for self-benefit
01:14 - Yandel’s Bootleggers and Baptists theory explained in current AI debates
01:54 - Practical uses of open-source AI models and their importance
02:29 - The threat of overreach: sweeping regulations and centralization
02:48 - Historical parallels: post-9/11 security and climate control measures
03:22 - Cultural context: the decline of religious frameworks and embracing humility
04:06 - The rhetoric around control versus prudent regulation
04:38 - The strategic importance of open-source AI against monopolistic forces
05:16 - Recognizing symbolic warnings and resisting fear-mongering for political gains
05:36 - The responsibility to protect liberty from prophets of doomResources & Links:

• Bruce Yandel's Bootleggers and Baptists Theory

• Hugging Face - Open-source AI models

• Understanding AI Regulation, Center for Democracy & Technology

Connect with David Weidman:

• Twitter

• LinkedIn

This episode uses the story of Lee Sedol, AlphaGo, and the legendary Move 37 to explore what AI teaches us about strategy, creativity, and cybersecurity. AlphaGo showed that machines can produce moves humans do not predict until the consequences unfold. In cybersecurity, that kind of surprise can be dangerous.

As AI changes how attackers operate and how organizations use sensitive information, defenders cannot afford to stay reactive. The challenge is to move from gote to sente: from patching after incidents to spotting risk earlier, prioritizing better, and building security into the workflows people already use.

Takeaways

Oracle GPT is a thought experiment on AI in national security.

The President could have unprecedented access to intelligence.

There are significant risks associated with AI in decision-making.

The balance of power may shift towards the presidency.

Presidential competence is crucial for using Oracle GPT effectively.

Misinformation could be a major risk with such a system.

Ethical implications must be considered in AI recommendations.

Human judgment should not be treated as an obstacle by AI.

The design of Oracle GPT must ensure accountability.

The constitutional order must be preserved in AI usage.

titles

keywordsAI models, open source, closed source, geopolitics, innovation, regulation, AI safety, public investment, model distillation, AI race

key topics

• Open vs closed AI models and their implications

• Geopolitical framing of AI development

• Regulatory capture and industry incentives

• AI safety and dual-use concerns

• Knowledge diffusion and model distillation

• Public investment in AI and race to AGI

• Enforcement challenges for open models

• The future of AI innovation and competition

Key topics:

• The evolving concept of authorship in academia and software development

• How LLMs accelerate knowledge production and collaboration

• Cultural norms around attribution, attribution, and hierarchy reforms

• The analogy of cyborg chess to future scholarship

• Language barriers and the role of LLMs as a new lingua franca

• Ethical considerations: transparency, accountability, and fraud prevention

• The aesthetic versus pragmatic uses of language in scholarship

• Updating authorship norms for an AI-augmented future

Late last month I graduated from the Harvard Kennedy School with a Master in Public Policy. Beginning July I will be based in Austin, TX. Thank you to everyone who has helped me along the way. I am deeply grateful for all of the support.

Thank you, too, to those who have signed up since my last update. Please share if you know of anyone who would be interested.

On Request:

• Prototype testing of the Moyo information space mapper. Find leaks of your secrets (classified, proprietary, personal) in public LLMs (ChatGPT, Claude, Qwen, Grok, etc).

• SenTeGuard Pilot - protect yourself or your organization from leakage of valuable information through air-gap capable semantic guardrails. Substack

• Let’s Talk:

  • Private - How can your organization more effectively secure your secrets in the LLM era through implementation of the Cognitive Security Verification Framework (CSVF) and by other means.

  • Public - How to craft tech-positive, pro-future, big-tech skeptical market-oriented policies of abundance in the LLM era.

Writing Roll-Up

Broad Policy Articles:

1. AI and Powerlessness. The popular framing around AI safety contains several important gaps. In this essay, I explore a number of high-level technical considerations that complicate assumptions about control, with the goal of moderating both the tendency toward catastrophism and the presumption that AI systems can be straightforwardly governed through centralized oversight. Substack

2. There is Always Another Apocalypse. Every era has its own existential dread, and while AI risks are real, treating every new technology as the end of the world can distort judgment around policy. Substack

3. Amodei’s Coup. Anthropic’s policy posture reveals how private AI companies can quietly claim political authority by deciding what governments and institutions should or should not be allowed to do with frontier models. Substack

SenTe Focused Articles:

1. Meet Joseki:WrapperHub. WrapperHub introduces a marketplace for reusable AI “wrappers” that package prompts, workflows, evaluations, and guardrails into shareable tools for safer and more structured AI use. Substack. Site.

   

2. The Cognitive Security Verification Framework. The framework argues that LLM-era data security must move beyond detecting protected strings and toward measuring what protected conclusions a model can help users infer. I will be speaking on this topic at BSides San Antonio on June 3th. Substack

   David

   Subscribe now

   
   
   

Original

Tales of apocalypse has always been useful. From Cold War strategists to the War on Terror, every age finds its own end of the world and every age concludes that extraordinary threats demand the suspension of ordinary limits on power. Artificial intelligence is simply the newest entry in this genre. We are told the technology is so uniquely dangerous that free speech, privacy, competition, open research, and democratic accountability must all be sacrificed for our survival. Not coincidentally, the beneficiaries of sacrifice happen to be the wealthiest people and companies in human history.

Regulatory Capture in Disguise

Bruce Yandle’s “Bootleggers and Baptists” theory explains how durable regulation often emerges when two very different coalitions support the same restriction for different reasons: the “Baptists” provide the moral language and public legitimacy, while the “Bootleggers” quietly profit from the restriction itself. Current AI regulatory proposals provide the perfect modern parallel. The Baptists warn of doom and insist that they are only trying to save humanity, while the Bootleggers provide the money, the lobbyists, the institutional access, and the quiet understanding that the rules being proposed will just happen to entrench the incumbents. In other words, someone with pure motives may inadvertently and indirectly promote the interests of the self-interested.

Big Frontier firms and their advocates know that a small startup working in a garage will not survive a vast compliance regime. A university researcher will not be able to compete with a trillion-dollar firm blessed by federal regulators. Open source developers sharing models on Hugging Face will be told that their work is too dangerous for public release, despite the fact that much of this ecosystem is not composed of godlike machines plotting the end of mankind but rather practical tools that researchers, companies, and hobbyists adapt for narrow and often mundane purposes. Hugging Face reported that in 2025 its ecosystem had grown to 13 million users, more than 2 million public models, and more than 500,000 public datasets, while one 2026 analysis found that in 2024 Hugging Face recorded an average of 2,199 new models created per day.

“AI regulation” sounds abstract until one considers what these open models actually do. Embedding models, for example, convert text into numerical representations that can be used for search, retrieval, classification, and semantic similarity, which means they help people find documents, organize information, and build better internal knowledge systems. A sweeping licensing regime would not simply restrain frontier labs. It would also burden the open, competitive, and decentralized ecosystem that allows smaller actors to build useful systems without asking permission from a handful of corporations and regulators.

Continuation of a Trend

The easiest way to centralize authority is to tell people that the normal rules are inadequate to confront the emergency. After September 11, Americans were told that liberty had to be balanced against security in ways that would have been unthinkable a decade earlier. During the nuclear age, international coercive inspection regimes were devised and regime change wars were carried out to prevent proliferation. During climate debates, the proposed solution is often not merely stewardship or innovation, but control of the world economy.

In each case, there is a real problem, but rather than addressing it through less invasive means, the powerful formulate and implement solutions meant to further their own interests. Concerns over terrorism, nuclear war, environmental degradation, and the negative side effects of AI are all valid. But it would be a mistake to take a real danger, elevate it into an existential emergency, and then use that emergency to justify extreme solutions crafted by self-interested parties.

A New Trend?

The decline of religious life in the West has left many without a framework for mortality, uncertainty, and suffering. Where religion once taught people to think in those terms, secular politics now offers thinner substitutes. At their best, believers understand that catastrophe is not an aberration but part of the human condition. Every life ends in an apocalypse of its own, and every civilization is temporary. The proper response to this knowledge is not panic, but humility. A society that loses this grounding becomes easy prey for prophets of doom because people who cannot sit with uncertainty will surrender freedom to anyone who promises safety.

The loudest voices do not simply argue for prudence. Prudence would mean better cybersecurity, clearer liability rules, and a sober understanding of how this technology will affect society. Instead, many argue for control. They want licensing, censorship, centralization, and a priesthood of approved experts, along with preferential rules for the politically well connected. We should be particularly suspicious of any regulation that acts as a de facto moratorium on open-source AI, as open-source language models pose perhaps the greatest threat to the profitability and dominance of the Big Frontier firms. While every age has had its prophets and warnings of final judgment, we should be wary that in an increasingly secular world, we do not allow these secular eschatologists to achieve their political objectives through fearmongering.

There is always an apocalypse to fear, there is always someone willing to explain why this time is different, and there is always a class of people eager to convert your fear into their dominance. The duty of those who value liberty is not to deny danger. It is to deny the prophets of doom the ability to convert timeless, human existential fear to a politics of control.