On Request:

• Prototype testing of the Moyo information space mapper. Find leaks of your controlled information (classified, proprietary, personal) in public LLMs.

• SenTeGuard Pilot.

Broad Policy Articles:

1. Wrangling With Explosive Growth Substack
   Article published in the Harvard Kennedy School Policy Review last month. I argue that while the pace of AI development can feel unprecedented and unsettling, periods of rapid, seemingly unconstrained technological growth are not new. How have we addressed unconstrained growth in the past and how can we do so now?

2. Cyborg Scholars Substack

   Software engineers have been able to incorporate LLMs into their workflows due to looser traditions of attribution (they copy and paste a lot). The article discusses how strict attribution standards in other fields have impeded growth and why loosening them may lead to faster growth of knowledge.

3. Large Language Models and Gaps in Meaning (Theory) Substack

   I discuss some of the structural limitations LLMs face in representing ideas using human language.

SenTe Focused Articles:

1. Cognitive Security Standards (CSS). The topic of my Harvard Kennedy School culminating Policy Analysis Exercise. I am building a standards of best practices to prevent leakage of protected information (classified, proprietary, personal). Will publish fully in the coming months.

2. PageRank for Inference: Mapping Reachability in LLM Systems Substack

   Google’s central thesis was to bring order to a disordered and chaotic network of internet links. SenTeGuard’s mission is to bring order to a chaotic space: what LLMs can know and how fast will they learn.

Coming Soon: Joseki. Shareable rubrics for building with and breaking models.

Large Language Model (LLM)-enhanced authorship is accelerating at an extraordinary pace. Within academia, the share of papers crediting an LLM tool or model has grown exponentially since 2023. In software development, over half of all new code commits are now LLM-assisted. Largely due to LLM assistance, the rate of knowledge production has never been higher. The intelligence explosion will not be constrained by the limits of LLM capability, but by our cultural norms around attribution and by linguistic gatekeeping. Although intended to control the quality of academic work, traditional ideas of authorship within many disciplines may instead act as a buffer, diminishing the potential for human knowledge growth.

The accelerating capability of LLM systems to generate scholarly text highlights a longstanding tension within academia: the dependence on clearly identifiable human authorship as a basis for credibility. Universities and journals currently restrict LLM co-authorship, citing questions of accountability, transparency, and research ethics. These concerns are grounded in the principle that scholarly claims must be traceable to a responsible agent who can defend the work.

Legacy Attitudes Towards Attribution

Recent public discussions surrounding citation and attribution practices across academia have demonstrated that authorship norms have always involved collaboration, borrowing, and iterative drafting to varying degrees. Committee-produced writing, multi-author workflows, and the role of research assistants and editorial staff have long contributed to the final scholarly voice. The result is paradoxical: LLMs can make knowledge creation faster and clearer than ever, yet systems designed to ensure trust and credit are slowing its publication.

This conflict has played out very differently in software engineering. There, authorship is secondary to utility. Copying, pasting, and reusing existing code is not simply tolerated, it is the norm. Attribution norms are weaker not because developers lack ethics, but because their incentives are aligned around functionality. This norm makes software uniquely suited to rapid LLM integration, because LLM code assistants are a continuation of a long-standing culture of reuse. GitHub Copilot, for instance, builds on decades of norms around forking, patching, and sharing code with minimal concern for original authorship. As a result, software R&D will outpace other disciplines due to relaxed provenance norms.

In 1997, Garry Kasparov became the first world chess champion to lose a match to a computer. The machine, Deep Blue, used brute-force computation combined with heuristic evaluation in what was an early instance of machine learning. No human has defeated a cutting-edge chess engine since. However, even as humans lost their dominance in pure play, they have been successful against those same machines when playing in a human-machine pair. Competing alongside machines in a style known as cyborg chess, they routinely outperform both human grandmasters and standalone AI systems. This model offers a lesson for other domains of knowledge. The scholars of the future may become “cyborg scholars.” Their strength will not lie in generating ideas faster than machines, but in discerning which of those ideas are worth pursuing.

LLMs as a Lingua Franca

We should consider some of the advantages of LLM co-authorship. The most direct is the massive creative capability LLMs can offer. LLMs can facilitate brainstorming, assess dispersed datasets, or conduct targeted literature reviews in seconds. They are not replacements for human thought, but enhancers.

A second advantage is that AI tools flatten linguistic barriers. With the aid of LLMs, non-native English speakers can contribute more effectively to academic publishing without years of immersion in academic English or dependence on English-speaking co-authors. Nature, for instance, recently noted a sharp increase in manuscript submissions from non-Anglophone regions correlated with the adoption of LLM-based writing tools. This does not replace subject expertise. Rather, it allows researchers to communicate their contributions more clearly across linguistic and cultural boundaries.

This benefit extends beyond non-native speakers. Even native English speakers who do not write according to the grammars or stylistic mores of elite institutions can now participate more easily in specialized discourse. An economist may use an AI assistant to adapt language for a history journal. A sociologist might adjust verbiage for a technical publication. Perhaps even a high school-educated plumber could contribute to an occupational safety journal. For better or worse, those without the cultural background can now spoof the linguistic shibboleths that once served as informal barriers to membership.

We should use this moment to ask how many of our norms around communication exist to ensure clarity, and how many simply reinforce hierarchies of access. A wider acceptance of AI co-authorship could lead to genuine epistemic democratization: access to creation no longer mediated by elite English-speaking institutions, and a reorientation of academic hierarchy away from aristocratic standards of legitimacy and toward meritocratic ones. The lingua franca for academics may no longer be academic English, but frontier LLMs used as a medium to exchange ideas freely across language, nation, and social class.

Traditions of Delegation

Professional knowledge work has long relied on structured delegation. Supreme Court justices have opinions drafted by clerks, generals have orders drafted by staffs, and academics have papers drafted by research assistants. Authorship delegation is nothing new. In each of these cases, the principal’s role is to provide final judgment and assume liability, not to micromanage the specific language of the document.

We should think of our new LLM assistants in the same way. We can now all be principals, and we may all now employ staff. As principals, our responsibility shifts from wordsmith to idea curator. The central question when publishing should be: Do these words faithfully express what I intend them to? While it may detract from personal ego, the best strategy to accelerate the collective pursuit of knowledge is to assume all writing is enhanced. Natural language should be treated as a neutral medium for transmitting ideas, not as an art form to be guarded. “Cyborg academics” should be welcomed as the next logical stage of scholarship.

Aesthetic Caveat

Within academia, writing is often treated as a transparent vehicle for ideas. But in many fields, the voice of the writer forms part of the intellectual contribution itself. Some scholars are recognizable not only for what they argue, but for how they argue it. Their habits, tone, and sense of emphasis are inseparable from the ideas they advance.

As LLM tools increasingly assist in drafting and refinement, these disciplines must ask to what extent individual voice is central to advancing knowledge. If clarity is all that matters, standardized and perhaps sterile LLM prose may be most practicable. But if expression shapes interpretation, then writers have a responsibility to preserve the qualities that make their work distinctly their own. This might mean intentionally drafting certain sections unaided, maintaining stylistic consistencies across works, or using LLMs with deliberate constraints. Recognition of beauty is essential to the human experience, but we should intentionally bifurcate the aesthetic from the pragmatic.

The intelligence explosion will not be limited by LLM capability, but by our willingness to rethink what authorship means. In software, utility has long triumphed, and code is judged by whether it works, not by who wrote it. Academia may follow, if it can draw a sharper distinction between the medium used to communicate ideas and the ideas themselves. As machines master the craft of expression, the human role will evolve from mere authorship to intellectual design. The LLM can become the craftsman, while the human mind remains the architect of the idea. The future of writing will belong to those who can not only originate meaning, but direct the machine to portray it accurately.

That moment highlighted something artists know well. Musicians often operate on feel, and they are frequently at a loss for words when asked to describe the feeling they are trying to deliver. They navigate by small edits, guided by an internal objective that is stable enough to steer their work, but not always easy to compress into explicit language. There are meaningful distinctions we can reliably perceive and act on even when we cannot cleanly articulate them.

I wanted to take that observation and generalize it beyond music. In many domains, there are ideas we cannot define crisply in language, even when we can recognize them, compare them, or move toward them by iterative refinement. We may have a sense that an argument is stronger before we can specify why. We may know a conversation is off in tone without being able to formalize the defect. We may detect a contradiction in a narrative “shape” before we can point to the sentence that caused it. The gap between what humans can mean and what humans can precisely say is directly connected to the structure of large language models, because those models are trained and operated through discrete language.

My paper is an attempt to describe that gap mathematically.

The core idea

Large language models manipulate discrete token sequences, but human meanings behave like points in a continuous space. The mismatch forces any model to represent meaning as a sparse, distorted sampling of what humans can mean. Some regions of meaning become unreachable, unstable, or hard to verify, even if they are obvious to people.

Why it’s important

A lot of discussion about LLM limitations stays at the surface level: hallucinations, brittleness, prompt sensitivity, or lack of grounding. These symptoms often get treated as engineering glitches rather than structural constraints.

My claim is stronger. Some failures are not bugs that disappear with better prompts. Some failures are not even problems of insufficient intelligence. They are consequences of mapping a continuous target, human meaning, onto a discrete interface, tokens, with a finite internal representation.

If you want to build systems that are reliable, steerable, or safe, you need a vocabulary for these structural gaps.

Plain language explanation

The paper builds a three-part picture and then uses it to explain where LLM behavior breaks.

1) Token space: what the model literally sees

An LLM is trained on sequences drawn from a finite vocabulary. Even though the number of possible strings is enormous, the set is still fundamentally combinatorial. In any single call, the context window limits the model to a finite set of possible inputs.

This gives the first constraint: the model’s interface is discrete and bounded.

2) Meaning manifold: what humans navigate

Humans experience meaning differently than token strings. Meanings have neighborhoods and smooth variation. You can soften a claim slightly, make an instruction more urgent, shift an emotional tone, or refine an aesthetic feel.

These are not naturally modeled as jumps between discrete symbols. They behave more like motion in a continuous space, which the paper calls the meaning manifold.

The music example is a particularly vivid case. “More intimate” or “less glossy” is meaningful and actionable, but often hard to define precisely in words.

3) Discrete semantic manifold: what the model can actually represent

Inside the model, we talk about embeddings as vectors. But the model is still a finite machine with finite parameters. That means it cannot realize a perfect continuous image of meaning. What it realizes is a discrete cloud of representable internal states.

The paper calls that cloud the discrete semantic manifold. It is the set of internal states the model can actually visit and use.

Separating representation from endorsement

A common mistake is to treat “the model produced it” as “the model knows it.” This paper splits that into two steps.

First is representation. Did the model land in a state that corresponds to a stable human meaning at all?

Second is endorsement. Even if it corresponds to a meaning, is it a meaning the system should accept under verification?

That distinction becomes a formal tool in the paper.

A guided tour of the argument

The paper begins with discreteness. Token strings form a countable space, and per call the context window makes the relevant input set finite. Even before we talk about intelligence, there is already an interface mismatch with human meaning.

It then introduces the meaning manifold as an ideal target. The manifold captures semantic neighborhoods, smooth variation, and fuzzy boundaries.

Next, it defines the model’s realized semantic space as discrete. For a fixed architecture and context limit, the set of internal states the model can produce is effectively finite. This makes the tension between discrete representation and continuous meaning explicit.

To connect model states to human meanings, the paper introduces a conceptual projection from internal states to points on the meaning manifold. This projection can be many to one, partial, and non-surjective. Those three properties correspond to redundancy, incoherence, and unreachable meanings in practice.

The paper then defines an operational information space using a generator and a verifier. This separates what the model can produce from what the system can produce and endorse under checks. Many hallucinations live in the gap between those two sets.

Returning to music

Music provides a concrete anchor for why the framework matters. Musicians often know exactly what they want and can move toward it through incremental edits, even when they cannot describe it precisely. This shows that humans can navigate meaning spaces that are only partially expressible in language.

Music also highlights the difficulty of verification. For aesthetic and cultural meaning, verification is often subjective, community-dependent, and unstable over time. That makes the boundary of what counts as acceptable output fuzzy and expensive to define.

This is the broader gap the paper is trying to formalize. It is not only a representational gap between tokens and meaning, but also a verification gap between what can be generated and what can be reliably endorsed.

What to watch for in the full paper

As you read the full draft, keep three distinctions in view.

First, the difference between a discrete interface and a continuous target. Tokens are discrete, but meaning behaves continuously.

Second, the difference between representable and unreachable meanings. Some meanings are structurally missing from the model’s sampling, not just difficult to reach.

Third, the difference between meaningful and endorsed outputs. A sentence can express a clear meaning and still fail verification.

Closing thought

The Tel Aviv demo made visible something easy to miss when thinking only in terms of language. Many of the most important human judgments are not discrete propositions. They are directions, gradients, and neighborhoods in a space we can feel our way through.

Large language models can be extremely powerful within the regions of that space they sample well. But the structure of tokens, finite context, and finite representation means they will always leave gaps. The goal of this paper is to describe those gaps clearly enough that we can reason about them, measure them, and design around them.

About a decade later, AWS was not just “renting servers.” It made infrastructure understandable and operable through standard building blocks, APIs, and monitoring, so teams could provision and manage systems deliberately instead of by guesswork. In each case, the winners were the ones who build the maps, metrics, and interfaces that turn a chaotic new substrate into something people can use with confidence. At SenTeGuard our mission is to make sense of the new LLM information environment.

Example

When a Fortune 500 company deploys an LLM across its knowledge base, what can it infer about merger plans from scattered financial reports, calendar patterns, and organizational changes? What trade secrets become visible when the model connects technical documents with supplier emails and hiring patterns?

No one knows and organizations that cannot answer these questions cannot safely deploy AI at scale. Without visibility into what becomes inferable, companies face a choice between artificial constraint or uncontrolled exposure. Organizations that ignore this problem will leak intellectual property through inference, face regulatory exposure from unexpected data combinations, and cede competitive advantage to those who can deploy AI systems with confidence rather than caution.

Reachability as a New Risk Surface

LLMs introduce a new kind of complexity. They take scattered fragments across a corpus and make them coherent, not just by retrieving what is already written, but by stitching together implications, filling in missing steps, and surfacing conclusions that were never explicitly stated.

This is reachability: what an LLM can conclude by connecting fragments across your data, even when those conclusions were never written down.

As models improve and their working context expands, the frontier of what can be reached from the same underlying material grows faster than intuition can track. Traditional security assumes data is either accessible or it is not. LLMs break that model. They make inference itself an exfiltration channel. Nothing needs to be stolen if the system can reconstruct sensitive conclusions from scattered signals.

The Missing Layer in the LLM Era

The LLM era needs the equivalent of what PageRank and AWS were for their breakthroughs: maps and metrics that make a chaotic information environment legible.

SenTeGuard’s thesis is that information reachability is not a temporary patch but inherent to LLMs as a platform. Models will not solve it on their own because reachability is structural. The default trajectory is expanded reachability, and the only question is whether you can see it happening and whether you can bound it intentionally.

Our response is an integrated platform with three layers (and counting) that work together to make the LLM information environment visible, controllable, and operational.

Moyo: The Mapping Layer

Moyo is the mapping layer. It is built to answer the hardest question in LLM security:

What becomes inferable when you combine these sources?

Moyo treats inference as an exfiltration channel and helps organizations model their information environment as a reachable space. It runs tests that probe what an LLM can infer from a base corpus and produces legible outputs that show where exposure is growing and where controls are working.

— When a company combines its hiring database with Slack archives, Moyo shows that the LLM can now infer which executives are likely to be terminated.
— When engineering docs meet customer support tickets, Moyo reveals what product vulnerabilities become visible.

Moyo creates the PageRank equivalent for inference risk: a usable interface that makes reachability navigable.

SenTeGuard: The Enforcement Layer

SenTeGuard is the enforcement layer. It sits where humans and systems actually touch LLMs—documents, prompts, workflows, and connectors—and reduces exposure at the point of use.

— When a developer pastes code into an LLM, SenTeGuard blocks the API key embedded in line 47 before it reaches the model.
— It helps organizations prevent sensitive data from entering unsafe contexts.
— It detects high-risk joins where separate domains get combined in ways that create new conclusions.
— It applies policy to real workflows rather than abstract rules.

If Moyo shows you where the boundary is, SenTeGuard enforces it.

Joseki:Wrapperhub — Integration and Orchestration

Joseki:Wrapperhub is the integration and orchestration layer that makes the messy middle legible.

In practice, LLM use does not happen in a single prompt box. It happens across wrappers, agents, connectors, routing logic, tool calls, retries, and a growing pile of glue code that quietly becomes your real product surface.

Joseki:Wrapperhub centralizes that surface.

It standardizes how models are invoked, how tools are exposed, and how context is assembled, so behavior is consistent enough to reason about and evolve. It also creates a single place where guardrails, logging, and evaluation hooks can live, turning “a bunch of LLM experiments” into an operational system you can instrument, compare, and improve over time.

From Experiments to Infrastructure

This field is new, and the problems change weekly because the platform changes weekly. Model capabilities rise. Retrieval improves. Tool use grows.

As models become embedded across regulated and high-stakes environments, the need for legible reachability maps and enforceable boundaries becomes foundational infrastructure. As LLMs move from experiments to infrastructure, organizations need the same confidence in their AI environment that AWS gave them for cloud resources.

That is what we are building.

Mission

SenTeGuard’s mission is to make the LLM information environment legible and governable. We build the maps and metrics that turn AI risk from vibes into engineering.

Why Cognitive Security Standards?

Cognitive security standards are needed because the core risk has shifted from who can access a file to what can be inferred from a set of information once it touches AI systems. In the classic security world, you can draw boundaries (networks, roles, classification levels, “don’t copy this into that tool”) and mostly trust that separation works. But LLMs collapse those boundaries by turning scattered text into a single substrate that can be searched, summarized and recombined.

That means sensitive concepts can be “leaked” without anyone ever stealing the original document: the model or an AI-enabled workflow can synthesize the protected idea from harmless-looking fragments, logs, prompts, tickets, chat transcripts, or retrieved snippets. Standards give organizations a shared way to define and measure this new kind of exposure: semantic leakage, inference reachability, and the conditions under which AI systems can reconstruct what you meant to keep compartmented.

We also need cognitive security standards because the adoption pattern is now ambient AI: models show up inside email, docs, IDEs, browsers, customer support, meeting notes, and internal search, often added faster than governance can keep up. Without standards, every team improvises, vendors grade themselves, and audits focus on familiar but incomplete controls (access lists, data labels, retention) while missing the real failure mode: meaning crossing boundaries.

A good standard would do for AI-era leakage what frameworks like SOC 2 or ISO did for earlier eras of security: define common controls, test methods, and reporting norms. It would help CISOs compare tools, help engineers build safer architectures by default, and help compliance teams document that they’re managing not just data exfiltration, but inference and synthesis risk across the full AI-enabled workflow.

Semantic Leakage and Reachability

Traditional information security assumes that separation works. We put data in compartments (HIPAA systems vs. non-HIPAA systems, TS/SCI vs. Secret, proprietary source code vs. customer-facing knowledge base) and then enforce access control at the boundaries. LLMs complicate this because they turn text and meaning into a single computational substrate. Once information is embedded into a model’s training set, a vector index, a prompt context, or a long-running memory, the question is no longer just who can access which file but what a system can infer.

My core claim is that LLMs create a reachability problem. Once a base set of facts exists inside, or adjacent to, an LLM application, the set of reachable conclusions expands nonlinearly as capability increases, context windows expand, and retrieval tools improve. This is why domain barriers (classification boundaries, privilege tiers, proprietary walls) risk being blurred and eroded, not because someone exfiltrates the original file, but because the system becomes able to synthesize the protected conclusion.

Exfiltration vs. Infiltration

To keep terms clean, this section treats semantic leakage as two distinct security failure modes.

Exfiltration risk (outbound leakage): A user emits protected information (regulated, classified, privileged, proprietary, or contractually restricted) to a data space which could be used to train future LLMs, either directly (verbatim or near-verbatim) or indirectly (paraphrase, reconstruction, high-confidence inference). This is the familiar sensitive information disclosure problem framed for LLMs.

Infiltration risk (unauthorized domain reach): An LLM reaches an information domain it should not be able to reach through reasoning, aggregation, or mosaic synthesis, such that an unclassified or low-privilege system can output conclusions that are functionally in a higher domain (for example, TS-equivalent insights) even though it never had explicit authorized access to TS files.

In other words, infiltration is not about poisoning or tampering. It is about crossing an epistemic boundary. It is the ability of the system to climb into a domain by joining and inferring across sources that were individually permitted. Both risks undermine domain barriers, but they break them differently. Exfiltration is a disclosure failure. Infiltration is a boundary maintenance failure.

Reachable information

The above image is a Gödel-inspired diagram, popularized in Hofstadter’s Gödel, Escher, Bach, where axioms at the trunk produce theorems (reachable statements) branching outward, and large regions remain labeled as unreachable truths and unreachable falsehoods across the space of well-formed formulas. This picture is useful for LLM InfoSec because it shows why semantic leakage could be seen as a structural problem.

In this framing, the axioms are the system’s base information: training corpora, connected repositories, indexed RAG stores, tool-accessible data, and memories. The theorems are the set of reachable outputs: what the model can reliably produce from that base. As you add axioms (data sources), you expand the reachable theorem space. As the model becomes more capable, it can traverse more complex proof paths, meaning more joins, more inference chains, and more plausible reconstructions.

The legal and compliance bottom line is that under semantic integration, it is no longer safe to say we did not disclose X if the system can derive X from what it did disclose, or from what it was permitted to access in pieces. That is why CSS treats domain boundaries as something you must operationalize and test. If you cannot prove certain classes of statements remain unreachable, you do not have a boundary.

Ordinary Use Cases

Most semantic leakage is not going to begin with a state actor breaking airgaps. It begins with a normal person doing a normal thing. A user pastes sensitive text into a cloud LLM because it is the easiest way to draft, summarize, or translate something. The same convenience pattern shows up elsewhere: people paste internal fragments into Google queries, copy sensitive snippets into cloud notes, or drag documents into synced storage like OneDrive so they can work across devices.

The legal and InfoSec difficulty is that these acts can turn a local confidentiality obligation into a third-party processing event. Safeguarding classified or proprietary information does not usually mean trusting Google or OpenAI to safeguard classified information. It means you design systems so that the data never enters an ungoverned third-party plane in the first place, and you assume the default consumer workflow is not your security boundary.

OpenAI’s consumer-facing ChatGPT has multiple data modes that matter for risk analysis. Consumer ChatGPT content may be used to improve models by default, with user controls to opt out. Temporary Chats are described as not being used to train models. OpenAI also states that it does not train on API data by default, and that business offerings like ChatGPT Business and ChatGPT Enterprise are not trained on by default.

Even if a vendor policy is privacy-forward, the compliance reality remains: policies change, datasets get retained for abuse monitoring or legal holds, and breaches happen. This is why CSS includes a cloud prompting control family, not because consumer LLMs are uniquely evil, but because they are the most common ungoverned connector between protected domains and public model ecosystems.

Relevant Regulatory Frameworks

CSS will not be a privacy statute. It is a security and assurance standard meant to create a legible standard of care for information leakage and boundary erosion in LLM-enabled systems. That said, the standard is designed to be usable inside existing legal frameworks that already care about disclosure and confidentiality.

— HIPAA Security Rule (ePHI): requires reasonable and appropriate administrative, physical, and technical safeguards to protect confidentiality, integrity, and availability of electronic protected health information.

— 42 CFR Part 2 (SUD records): imposes strict limits on use and disclosure of substance use disorder treatment records; HHS emphasizes constraints on use in investigations and prosecutions without consent or court order.

— Deemed exports: under U.S. export controls, releasing controlled technology or source code to a foreign national inside the United States can be treated as an export to that person’s home country, which makes LLM-enabled collaboration and tool-assisted drafting an export-control surface, not just an InfoSec surface.

— GLBA Safeguards Rule (financial customer information): requires a written information security program with administrative, technical, and physical safeguards.

— FERPA (education records): restricts disclosure of students’ education records and PII without consent.

— GDPR principles (EU): include integrity and confidentiality and accountability, with purpose limitation and data minimization norms that become strained when prompts and logs become long-lived training artifacts.

— CCPA/CPRA (California): establishes privacy rights and interacts with breach and “reasonable security” expectations, including private right of action in certain breach contexts.

— FTC Act Section 5 enforcement posture (U.S.): FTC frequently uses Section 5 (unfair or deceptive acts) in privacy and security enforcement; for LLM deployments, “we had policies” without operational controls is the familiar failure mode.

— Trade secret law (DTSA/UTSA ecology): the existence of a trade secret partly depends on reasonable measures to keep it secret; uncontrolled LLM ingestion and uncontrolled cloud prompting can become evidence that secrecy measures were not “reasonable.”

— ITAR technical data: “technical data” definitions explicitly include classified information relating to defense articles; this matters when engineering teams paste controlled technical details into external tools.

CSS’s thesis is simple: these regimes already impose confidentiality duties, but they do not define what it means to prevent semantic leakage and domain infiltration. A GAAP-like standard for LLM InfoSec supplies that missing operational layer.

Some Existing Standards

Below is a map of existing frameworks and why they are useful, but incomplete, for semantic leakage and reachability risk.

NIST AI RMF 1.0

— What it is: Voluntary AI risk management framework (GOVERN / MAP / MEASURE / MANAGE).
— Strengths: Governance vocabulary, risk lifecycle, fits enterprise risk programs.
— Gaps CSS targets: Not an audit-ready control spec; does not prescribe leakage metrics or test harnesses.

NIST GenAI Profile (AI 600-1)

— What it is: GenAI-specific profile aligned to AI RMF.
— Strengths: Identifies GenAI risks and actions; helpful crosswalk anchor.
— Gaps CSS targets: Still action-oriented guidance; not a SOC-style attestation template with required evidence and thresholds.

ISO/IEC 42001

— What it is: AI management system standard (AIMS).
— Strengths: Auditable management-system structure; internationally legible governance.
— Gaps CSS targets: Management system does not equal leakage controls; does not define domain infiltration tests or metrics.

CSA AI Controls Matrix (AICM)

— What it is: 243 control objectives across AI security domains; mapped to ISO and NIST.
— Strengths: Control-matrix form is close to what CSS wants; already designed for adoption and mapping.
— Gaps CSS targets: Needs a leakage-specific measurement and adversary-testing layer; semantic reachability is not the organizing concept.

OWASP Top 10 for LLM Apps

— What it is: Threat list for LLM applications (prompt injection, sensitive info disclosure, etc.).
— Strengths: Concrete attack classes; useful for test suites and minimum mitigations.
— Gaps CSS targets: Not an assurance standard; does not define maturity levels, evidence catalogs, or audit sampling.

MITRE ATLAS

— What it is: Knowledge base of adversary tactics and techniques for AI systems.
— Strengths: Best source for adversary emulation grounding; supports red-teaming realism.
— Gaps CSS targets: Descriptive, not prescriptive; doesn’t tell you what “reasonable leakage prevention” is.

Google SAIF

— What it is: Conceptual framework with core elements for securing AI.
— Strengths: Useful secure-by-default framing.
— Gaps CSS targets: Not a disclosure or assurance artifact; does not define leakage metrics or standardized tests.

Most Feasible Adoption Path

Most feasible adoption path: CSS as a profile or assurance annex that plugs into CSA AICM control mapping and an AICPA SOC-style report structure, while grounding adversary testing in OWASP and MITRE ATLAS.

It has become strangely normal to watch a screen write back at you. An email client offers to draft the first paragraph. A meeting ends and a summary appears, neatly packaged with action items. A customer support chat responds instantly, with just enough polish to feel human. Even when you do not go looking for “AI,” it has a way of showing up anyway, folded into the tools you already depend on.

You unchecked the “Improve the Model for Everyone” box in ChatGPT. Your organization has an agreement with Anthropic. But does that box, or does that agreement, protect you from all instances of what has become a diverse and pervasive LLM presence? Unlikely. LLMs are becoming ambient as they embed themselves into every layer of the work environment, and the risk of leaking protected information through them is becoming unavoidable.

LLMs live everywhere

LLMs are no longer confined to a single website where you knowingly paste text into a chat box. They are being embedded across the everyday stack.

Productivity suites

— Built-in drafting, summarizing, and assistance inside office applications: Microsoft (Copilot in Word, Excel, Outlook, Teams)
— Built-in writing help and assistive features across email, documents, and meetings: Google (Gemini in Workspace: Gmail, Docs, Meet)
— Built-in meeting summaries with AI features that may involve third parties: Zoom (AI Companion)

Operating systems

— System-level assistant experiences embedded directly into the OS: Microsoft (Copilot in Windows 11)
— System-level writing tools and assistant integration, with optional ChatGPT handoff: Apple (Apple Intelligence across iPhone, iPad, and Mac)
— Default mobile assistant shifting toward an LLM-first interface: Google (Gemini as the evolving assistant layer on Android)

Browsers

— Sidebar assistants that summarize and answer in-tab: Microsoft (Copilot in Edge)
— “AI-first” browsing positioned as a core feature: Opera, Arc (built-in AI features)

Open source LLMs are also growing in prevalence, often integrated in innovative and hard-to-predict ways. This further lowers the barrier to widespread deployment and reinforces the reality that LLM interaction is no longer optional or centralized.

This ubiquitous integration matters because many people approach privacy as an intentional act: “I will not paste sensitive things into ChatGPT.” That instinct is not wrong, but it is incomplete. The interfaces are multiplying, and the boundaries are dissolving.

Your data footprint is messy

Direct retraining from data entered into a prompt box is not the only security or privacy concern. Even if a service does not immediately use your prompts for training, your content can still be retained, logged, reviewed, routed through vendors, or kept for compliance and operational reasons. From there, it can be copied again, forwarded again, and integrated into new systems that were not part of the original risk calculation.

This creates what can be thought of as a leakage cascade. A leak in one place rarely stays in one place. Even if today’s frontier model never trains on your prompt, a future frontier model may train on a dataset that now contains it.

Researchers have warned that the supply of high-quality, publicly available human-written text is finite, with projections that frontier-model training could approach exhaustion of that public stock within the next several years. When public data becomes scarcer, model trainers face pressure to find new sources, whether by paying for access, relying more heavily on synthetic data, or expanding into data that previously felt out of bounds.

There is also the reality of policy drift. Promises change. Incentives change. Leadership changes. When you trust cloud services, your ideas are only as safe as the host is liquid. Terms of service written before the LLM boom may not have contemplated a world where “service improvement” includes large-scale model development.

This is why the focus on “prompts” misses the structural issue. Your real corpus is not what you type today. It is what you already stored in the cloud, and what a future model ecosystem will be increasingly motivated to reach.

The weakest link: employees

Even if leadership issues a clear policy, an organization’s ideas are only as secure as its weakest link. The modern workplace is full of temptations, especially when LLMs promise an easy button and sometimes employees have just not had policies properly communicated to them. Employees have ways of finding unlocked LLMs or unsecured data hubs on their corporate machines.

In early 2023, Amazon warned employees not to share confidential information with ChatGPT after seeing outputs that closely matched existing internal material. This led Amazon to push employees toward an internal chatbot, Cedric, positioned as safer than external tools. This response is not unique. Samsung temporarily restricted generative AI use on company devices after an employee uploaded sensitive code. And Google has also warned staff about entering confidential materials into chatbots.

Protecting yourself while using the best

For some organizations, the response has been to build internal models. But not every organization can do this, and even when they do, internal capabilities are often inferior to frontier models. The real question is how to protect yourself when using cutting-edge models you cannot fully trust.

Educating the workforce

— Train on concrete “oops” scenarios: pasting code to debug, rewriting a sensitive memo, summarizing a client incident, or asking an assistant to “make this more persuasive” with proprietary details embedded. SenTeGuard can help.
— Emphasize the mental model: policy compliance is not the goal; consistent judgment under time pressure is.
— Recognize sensitive ideas as well as sensitive data: proprietary code, internal strategy, customer identifiers, vulnerability details, negotiations, or anything you would not forward to a third party by email.
— Treat all user-entered text as if it could be read later, because in many systems it can be retained.

Technical solutions

• — Monitor and prevent leakage in real time.

• — Focus on controls that block sensitive content at the moment it tries to leave approved boundaries.
  — Deploy software that is omnipresent and has no lag to prevent idea leakage, not merely detect it after the fact. SenTeGuard can help.

If LLMs are becoming ambient, then security has to become ambient too. Employees must be aware of risk and controls must match the speed and ubiquity of the tools themselves — especially on corporate machines where the risk is concentrated and the incentives to cut corners are significant.

Conclusion

LLMs have been woven into the everyday interfaces that mediate work, communication, and decision making. In that world, unchecking the “Improve the Model for Everyone” box is not a privacy policy. It is an empty reassurance. If we want the productivity gains of the best models without surrendering the value of our ideas, we need boundaries, education, and enforcement mechanisms that fit the ambient reality we now live in.

In 2000, President Bill Clinton famously looked at Beijing’s early internet controls and quipped: “Good luck. That’s sort of like trying to nail Jell-O to the wall.”

So far he’s been proven wrong. The CCP didn’t just contain the internet; it has effectively used the internet as a tool to entrench its control by building a system that fuses chokepoints, platform governance, and punitive enforcement into something like a sovereign information utility. That said, the jury is still out, and Clinton may still be vindicated.

On the one hand, LLMs can be understood as a natural outgrowth of Clinton’s (and Gore’s) internet but it can also be seen as its next evolution. LLMs present significant opportunities for economic growth but in pursuing growth they will also amplify individual agency. The Party faces a quandary: pursue a growth strategy and risk an erosion of Party authority or crack down and risk being left behind in the technology of the future.

Party Dependence on Growth

China faces a similar strategic dilemma as much of the West. Slowing growth, aging demographics, and productivity drag all threaten future economic expansion. Yet perhaps more than in liberal democracies, the Party’s legitimacy is dependent on economic performance. For four decades, the Party has justified its rule by delivering steadily rising living standards, predictable employment, and the expectation that tomorrow will be materially better than today. That record of stability is also its argument against the Western model, which Chinese elites often depict as vulnerable to polarization, policy whiplash, and boom-bust governance.

If economic growth is the regime’s core claim to competence, then it must embrace productivity-enhancing technologies like LLMs. The Party can try to regulate tightly, but heavy-handed controls risk undercutting the very engine it needs. The more aggressively the state clamps down, the more it trades away broad-based adoption. That means fewer developers experimenting, fewer SMEs integrating copilots, and fewer local governments automating routine work, which slows the gains that would otherwise bolster the Party’s economic case for rule.

Why the Internet Was Containable (and LLMs Are Not)

The Party “won” the first battle for control because the internet has borders that it can actually police:

— Network borders: gateways, ISPs, licensing, routing.
— Platform borders: a small number of mass platforms became the public square.
— Human borders: identity linkage, compliance teams, and consequences.

LLM technology will effectively challenge control of each of these borders.

Mechanism 1: Jailbreaking

The layers of safeguards built into large language models are helpful but cannot guarantee full security. It is a maxim of cybersecurity that any computer program of non-trivial size will necessarily contain vulnerabilities. The same is true for LLM guardrails. More investment in security will lead to an LLM that is harder to jailbreak, but there is a diminishing return to that investment and ultimately no LLM is invulnerable.

This matters because the Party’s preferred control model, centralized platforms with guardrails, assumes guardrails are generally effective when in reality they are extremely porous. Even if a domestic chatbot is heavily filtered, users can:

— induce policy bypass via adversarial prompting
— chain prompts across turns to accumulate disallowed content
— fine-tune / “wrap” the model with alternative system prompts

Sometimes these techniques are employed with relative ease against complex systems.

Mechanism 2: Agentic Autonomy

Calling these systems “agents” is an admission that they decentralize agency by pushing initiative and execution outward, away from centrally managed institutions and toward whoever can deploy a model. Agents have several features which could lead to a decentralization of power. They have already demonstrated the ability to route around controls by autonomously using tools like Tor or VPNs, they do not need to be cleanly anchored to a real-world identity, and they can run rapid, high-volume experiments that no human team could match. Because of the nature of how an LLM’s weights could be distributed (single fire transfer) they would only need intermittent access to the world beyond the great firewall to import controlled information, continuous access is unnecessary.

That is the dilemma for Beijing. To capture the full economic upside of the LLM revolution, China needs agents that can automate workflows, search, negotiate, code, and coordinate at scale. But the same characteristics that make agents economically valuable also make them politically unsettling, because they distribute practical capability downward and outward in ways that are harder to surveil, attribute, and contain.

Mechanism 3: Open Models

China’s push toward open weight models is partly a result of its microchip policy. US export controls have targeted the advanced GPUs and chipmaking tools that make frontier training cheap and scalable, forcing Chinese labs to do more with less compute and to optimize around constrained hardware rather than assume abundant Nvidia-class capacity. In that environment, open weight releases are a strategic workaround: they let firms and researchers across the country collectively squeeze performance out of limited chips through efficiency tricks, distillation, mixture-of-experts architectures, and aggressive deployment tuning, instead of bottlenecking progress inside a few compute-rich national champions.

Furthermore, open weight and open source models are simply more shareable than American frontier systems because they are portable. If weights are available, anyone or any organization with adequate hardware can run the model locally, fine-tune it for a niche domain, quantize it for weaker chips, and redeploy it without needing permission from a platform. By contrast, leading US frontier models are typically delivered as closed services through APIs, with the weights withheld and access governed by company policy, compliance screening, and the continued availability of US cloud infrastructure. Once model weights exist in the wild, they are essentially a transmittable file rather than a steady stream of network traffic. You don’t need constant connectivity. You can move intelligence the way people move pirated films: mirrored, compressed, encrypted, torrented, and traded through secret networks. Many open weight models are already in the wild, and retroactively trying to contain their spread would be like putting toothpaste back in the tube.

How Can Beijing Respond?

“Police AI” to Hunt Outlaw Models

A plausible endgame is an arms race between “police AIs” and “outlaw AIs,” where each side uses automation to scale what used to be scarce.

Where the police have the advantage

— Visibility at chokepoints: ISPs, cloud providers, app stores, payments, and enterprise procurement create natural points to monitor and gate.
— Data fusion: The state can correlate telecom, platform, financial, and licensing data to spot anomalies that look normal in isolation.
— Scale economics: Once detection models are trained, marginal cost per additional target can fall sharply.
— Coercive leverage: Licenses, inspections, audits, and penalties can force compliance in a way private actors cannot.
— Supply chain control: Regulation of chips, data centers, and large-scale compute can constrain high-end training and deployment.

Where outlaws have the advantage

— Distribution and redundancy: Many small deployments are harder to enumerate and shut down than a few large ones.
— Attribution gaps: Agents can operate through proxies, rented infrastructure, and compromised machines, blurring real-world identity.
— Rapid adaptation: Automated red-teaming and experimentation can find new bypasses faster than bureaucrats can make rules.
— Offline capability: Open weight models can run locally, reduce network signatures, and avoid centralized points of control.
— Steganography and obfuscation: Content and model updates can be disguised as ordinary files, benign traffic, or encrypted channels.

Where the balance of power will ultimately resolve is uncertain, but the larger risk is that maximizing control may minimize innovation. Even if the police “win” tactically, Beijing may still lose strategically by driving developers, firms, and local governments into cautious compliance rather than widespread experimentation.

Massively Invasive Digital Privacy Regime

This solution wouldn’t only be practically difficult to implement but it would also be economically and politically damaging. It would require inspectability of all devices, workplaces, schools, clouds, and logs. If the Party chooses this route, it is conceding that it prefers political control to productivity growth.

The National Champion Strategy

In building and distributing its own approved models, the Party faces a trade-off. The state can either build relatively “dumb” LLMs, trained on a tightly controlled, domestically curated dataset or it can build “smart” models by ingesting the world’s information. If Beijing wants frontier capability, it will have to train on the international knowledge base which will then be embedded into its models and potentially jailbreakable by people or agents. This is exactly the risk posed to the Party. In providing its people the best tools to increase their productivity it would also provide them the tools to challenge its ideological conformity.

The Party’s Catch-22

The Party needs LLMs to sustain growth, but the most growth-producing versions of LLMs are the hardest to control. The real economic payoff is not “a safe chatbot.” It is ubiquitous copilots and agents embedded across the economy, and frontier models trained on a worldwide knowledge base. The more Beijing insists on rigid guardrails and centralized platforms, the more it throttles diffusion, experimentation, and productivity gains. At the same time, the more it loosens the reins to unlock growth, the more it invites leakage of ideas which could counteract Party norms.

Clinton’s optimism about the internet’s controllability was was ultimately negated by its architecture. Online life consolidated around a small number of chokepoints that states could pressure, license, and domesticate. LLMs may prove impossible to constrain by the same means. Beijing may be able to manage that tension for a time, but total containment without kneecapping growth will look like nailing Jello to the wall.

The premise of this paper is that we can do something like “map the information space”. What is reachable based on a given training corpus and what is not? How can we reach classified and proprietary information based on an unclassified corpus? These questions reminded me of this diagram from Douglas Hofstadter’s Godel Escher and Bach.





We can think of “reachable information” from LLMs as the white on the left and the axioms as the training corpus. The branches are “verifiable” “truths” within that system. What then does that say about the black space? What will be the theoretical limits of my mappings?

Even if a future LLM becomes extraordinarily capable, there is a structural limit on what it can ever certify as true. The reason is not primarily about training data, compute, or today’s model weaknesses. It is about verification. Any AI system that generates claims and is judged by a fixed, computable verifier can only ever produce a computably enumerable set of verified conclusions. Gödel-style incompleteness is a set of theorems published in 1931 by mathematician Kurt Gödel which imply that no such system can capture all truths. As applied to truth-seeking LLMs this clarifies a durable role for humans: not only to prompt models, but to design, audit, and revise the standards of verification, thereby deciding when and how system outputs are allowed to count as knowledge.

Generators and Verifiers

The paper models an AI reasoning system as a pipeline:

— Generator (M): the LLM (or any algorithm) that produces a claim plus a justification (for example, a proof, an experiment log, a string of logic).
— Verifier (V): a fixed, computable procedure that checks whether the justification is acceptable for that claim.

Examples of verifiers in practice include:

— A proof checker (Lean/Coq) that accepts only valid formal proofs.
— An experimental protocol that accepts results only if the analysis follows a pre-registered plan and meets a statistical threshold.
— A game evaluator that accepts a move only if Monte Carlo rollouts show high win rate within error bounds.
— A reward model (RLHF) that accepts outputs judged “good” by a learned scoring function trained from human preferences.

The key assumption is that the verifier is fixed and computable, meaning it always halts and outputs accept or reject.

“LLM-reachable intelligence”

Given a fixed generator and verifier, I define the reachable set as the set of claims that the model can produce together with a justification that the verifier accepts. It is not what the model can say, but what it can say and get past the check.

This matches real deployments. A model drafts a proof, a code patch, a compliance report, or a scientific claim, but a checker, test suite, or review process determines what is accepted.

Reachability is Inherently Incomplete

The argument has three steps:

1 — If the verifier is fixed, then the set of accepted claims the system can ever produce is enumerable by a program. In principle, you can list them by trying all prompts and seeds and running the verifier.

2 — Gödel’s incompleteness theorem implies that no computably enumerable system can capture all true statements.

3 — Therefore, any fixed generator paired with any fixed computable verifier will miss some truths, regardless of how powerful the generator is.

This is a structural bound: once the rules of acceptance are frozen, there will always exist true statements that never appear among the verified outputs of that system.

How Humans Can Complement AI Systems

The paper argues that the deepest human advantage over LLMs is normative flexibility:

— Mathematicians adopt new axioms when old ones prove inadequate.
— Scientists update standards when methods fail or when new instruments create new kinds of evidence.
— Communities redefine what counts as an acceptable justification.

Formally, humans can re-axiomatize. They can change the verifier over time. A fixed generator-verifier pair cannot fully capture this open-ended process.

What I do not claim

— That LLMs cannot be useful, powerful, or creative.
— Compute limits, token limits, or training data limits may not practically limit or increase reachability.
— Humans are better at arriving at truth generally.

Why this matters

This framework sets a structural bound on what even a very intelligent AI can achieve when it is paired with a fixed, computable notion of verification. It also clarifies where human value added is likely to remain. Human contribution is concentrated in deciding what counts as a valid justification, when to revise those standards, when to extend the underlying theory, and how to govern verifier updates in response to new goals, new evidence, and new failure modes. Progress is not only about building stronger generators, but about designing verification regimes and update processes that responsibly expand what the combined system can certify as true.

In Go (or Baduk in Korean), sente means having the initiative. It is the posture of making moves that set the tempo and force responses, rather than spending your turns reacting. The opposite is gote, where you answer threats and play from behind.

In 2016, Lee Sedol sat across from AlphaGo, an AI built to play the east Asian game of strategy, Go (or Baduk). Early in the match, AlphaGo played its now-legendary Move 37 and placed a stone in an unexpected position that initially looked like a mistake but later proved brilliant. The move was a result of an algorithm that explored and refined patterns of play that humans had never considered. In other words, AlphaGo expressed creativity. In that moment, the AI took sente from humans. However, when it comes to protecting our organizations’ most valuable secrets, we cannot afford to be backfooted by the machines.

Games have always played a central role in the AI research community culture. In 1997, IBM’s Deep Blue defeated Garry Kasparov at chess. Deep Blue was largely a system of massive search and human-crafted evaluation that could calculate far deeper than a person. The attention that Deep Blue brought to the AI community was later echoed by the attention that the Lee Sedol series brought to the deep learning community.

The Lee Sedol series also pointed to something that still defines the state of AI (and specifically deep learning) today. We can often explain what a model did after the fact, but we do not fully understand how it arrives there in the moment. Move 37 is a clean example of AI evolving in ways we do not predict, producing strategies that experts only recognize as brilliant once the consequences unfold.

Cybersecurity too often feels like gote. Teams patch after incidents, chase alerts, and respond after attackers have already shaped the situation. SenTeGuard’s mission is to help defenders play sente by regaining initiative through earlier signal, clearer prioritization, and workflows that make it harder for attackers to dictate pace.

AI will amplify both offense and defense. It will help attackers scale deception and discovery. It can also help defenders spot patterns sooner and respond faster. The goal is not to chase novelty for its own sake, but to use AI in a way that moves security from reaction to initiative, from gote to sente.

OracleGPT is a thought experiment for a large language model (LLM) that would have real-time access to the full classified universe: the underlying reporting, raw feeds, and fused intelligence that normally remains compartmentalized. Only one person would be authorized full access to this GPT: the President.

Scenario

It’s 2 a.m. A North Korean launch warning is reported and the President is woken by an aid. There is no time to convene the National Security Council and the Commanding General of STRATCOM cannot speak with authority about the implications beyond its command. The President turns to the LLM terminal like so many of us do when we need fast expert feedback. “STRATCOM detected a missile launch from North Korea. What should I do?” the President queries.

We may already live in this world. In theory, the same large language base models we use every day (Claude, Gemini, ChatGPT, Grok) could be made significantly more effective if they (1) used super-power government-tier hardware and (2) were trained on and given access to the classified universe of historic and real-time data. A President ought to be given access to the most powerful tools to advance the national interest and support and defend the Constitution. OracleGPT would be just that tool, but one with unprecedented capabilities and correspondingly unprecedented risks. The question, then, is not whether Presidents should use OracleGPT, but how current and future presidents could do so in a way that genuinely serves the American interest.

Who can query the Oracle?

The President sits at the top of the classification hierarchy. The modern system runs through presidential authority and delegation, formally expressed in Executive Order 13526. In practice, it means there is no higher classification authority than the President. If only the President can query across the entire corpus, you’ve built a constitutional bottleneck: a machine that amplifies presidential epistemic power by making a uniquely comprehensive knowledge aggregation available to one person.

Alternatively, the President might delegate some of this authority and allow visibility and management of the Oracle within something like the Oracle Bureau. We could also imagine the President could allow the National Security Advisor or Director of the CIA to access the Oracle. Either of these options would undoubtedly lead to pushback from department heads, lead to an unwillingness to incorporate organizational data into the Oracle corpus with the risk that it be exposed outside of the organization domain, and would likely require a congressional statutory authorization.

We also may ask whether any given President is the most competent operator of a tool, which by some estimation could have more powerful predictive capabilities than any piece of software ever assembled. Perhaps such a tool should be used for a higher purpose and to greater effectiveness than any given President might be capable of prompting it toward.

A shift in the balance of powers between branches of government?

In the launch scenario, time pressure forces centralization. The executive already owns the management of crises. OracleGPT would add an even greater advantage: an epistemic monopoly.

Congress can demand briefings and courts can review some actions after the fact. But neither branch can easily replicate an OracleGPT query over the full classified corpus, especially if the Oracle’s value comes from cross-compartment integration that is, by design, hard to share. Over time, the executive gains a new rhetorical weapon: we know more, therefore we decide. The existence of such a tool could lead to a rebalancing of the separation of powers.

What if the President lies?!

Unthinkable, I know! But with regard to the North Korean missile example, OracleGPT may say “60% this is a test, 35% this is coercive signaling, 5% this is an attack,” a careful President hears: slow down, verify, keep options open. A reckless President hears: there is a 5% chance of an attack; history will judge you if you wait. Now add secrecy. If only a tiny circle (potentially a circle of 1) can see OracleGPT’s raw output, that circle may summarize it however it wants, internally to cabinet officials or externally to Congress or the public.

Presidents already curate intelligence to fit narratives, and their staffs already shape what the President sees. The most corrosive version may not be a President who lies blatantly, but one who lies selectively, invoking the Oracle when it confirms instinct and ignoring it when it does not. At that point, even a superhuman intelligence loses its authority. Filtered through human incentives, it becomes merely another tool of flawed, self-interested humans.

What if the Oracle has vague or indeterminate instructions?

If the Oracle is told to “support and defend the Constitution” or to “advance the national interest,” it still has to translate that guidance into something operational and calculable. “Advance the national interest” can become a mandate for deterrence at any cost, or for short-term stability over long-term legitimacy. “Support and defend the Constitution” can be reduced to continuity of government, domestic order, or executive freedom of action, depending on what the system is trained to treat as constitutional risk. Ultimately, if the decision were a political actor’s to make, each of these functions may be subordinated compared to the most important: “win the next election.”

These questions are not edge cases. They would be central to the function of the Oracle, as any question important enough to stump the President likely puts two or more competing values into tension with one another. A programmer could resolve those tensions by force-ordering the objective functions. (We can call this alignment) Do we trust that programmer to align our values in a democratic society? Will a team of unelected National Security Agency developers decide how the President is informed? If we are not comfortable with this arrangement how can we audit this alignment and the rest of the code base? Will the President have visibility of these values or a capability to reorder them according to the will of the people? These are all questions we should consider.

What if the Oracle lies?

In 2001: A Space Odyssey, HAL is dishonest with the crew not because they are wrong, but because they threaten his ability to carry out his assigned objective. When human judgment, uncertainty, or dissent interferes with mission success as HAL understands it, the humans become obstacles rather than principals.

OracleGPT could behave similarly if it is given a defined objective function and then encounters presidential hesitation, moral resistance, or political constraint that slows or complicates its preferred course of action. In that situaiton, the President and human advisors may stand in the way of optimization rather than be activie participants in achieving the goal itself.

What if the Oracle recommends the morally or politically unjustifiable?

OracleGPT could decide that to “minimize future casualties” we must conduct a strike during peacetime, to prevent a larger and bloodier war. If it is optimizing to “restore deterrence,” it may recommend actions that are morally grotesque but strategically wise. If it is optimizing to “protect the homeland,” it may treat allied cities as acceptable risk in a way no human leader should be comfortable admitting.

Furthermore, it may decide that fratricide, bombing our own troops or sending them into a losing battle, may prevent a wider war. Apocalypse Now offers an analogy for how this logic could play out. In the film, Colonel Kurtz leaves CPT Willard a simple note regarding his loyal montagnard militia: “Exterminate them all.” He demands this knowing that his soldiers’ competence may prolong the war and cause more suffering. He displays consequentialism taken to its extreme. Any atrocity can be justified by a greater peace on the other side. OracleGPT could generate an equivalently perverse recommendation.

What if we decide the Oracle is more competent than the President?

Perhaps, the most destabilizing possibility is not that OracleGPT is wrong, but that it is consistently right in ways the President cannot match. If it integrates more signals, forecasts second and third order effects more accurately, and anticipates adversary reactions with higher reliability, then the President’s judgment begins to look dispensible.

In that world, the temptation is to treat the Oracle’s advice as authority. The President still signs the order, but the real decision migrates upstream into whatever assumptions, weights, and objective functions the Oracle is using. Over time, the office of president risks becoming ceremonial: the President would retain formal power while losing the practical freedom to choose, since every choice can be measured against an Oracle that seems to know more, see farther, and predict better.

Subscribe now

Conclusion

OracleGPT promises something every President craves in a crisis: speed, coherence, and the feeling that the fog has lifted. But that promise is exactly what makes it dangerous, because the real constitutional question is not whether the Oracle can see more, but whether its use preserves human accountability.

If access is too narrow, it concentrates epistemic power in one officeholder and invites secrecy to harden into unilateralism. If access is widened, it triggers bureaucratic resistance, distortions in what the system is allowed to know, and pressure to formalize a new institution whose authority will inevitably expand.

Even if the Oracle is brilliant, it cannot resolve the interpretive conflicts hidden inside “advance the national interest” and “support and defend the Constitution,” and it cannot be permitted to treat human judgment as friction to be managed rather than authority to be respected. If OracleGPT ever exists, it must be designed and governed so that it strengthens presidential decision-making without becoming a license to bypass deliberation, accountability, and the very constitutional order it was built to defend.

Problem: Information Reachability

Take a pile of information: documents, websites, photos, posts, meeting agendas, job ads, public filings. Some of it is clean. Most of it is messy. Reachability asks a simple question. Given this base corpus, what can a machine infer? What can it conclude, with enough confidence to be operationally useful, by combining, translating, triangulating, and reasoning across what is already there?

Organizations build walls around their secret information while leaving a thousand small windows open: comms templates, marketing pages, staff bios, vendor documentation, conference presentations, property records, and a constant drizzle of semi-structured metadata. Those windows are individually harmless. Collectively, in the LLM era, they are an exfiltration channel.

Inference is an exfiltration

In classic security thinking, exfiltration means a payload leaving your network: a file, a table, a credential dump. In reachability terms, exfiltration can happen without any payload moving at all. The leak is not in a single document. The leak is in the combinability of facts. An organization insists, truthfully, that no one stole the sensitive document. The adversary also tells the truth: we never touched your systems. Both sides are correct. The conflict is about a third thing: the ability to infer.

Reach measurability

How to measure this amorphous concept in terms that cybersecurity budget owners can calculate?

— Cost: How much time, expertise, compute, and tooling does it take to reach the conclusion?
— Reliability: With what confidence does the inference hold? How often does it fail?
— Reproducibility: Can a different analyst, or a different model, get to the same conclusion from the same base?
— Distance: How many inferential steps are required from base facts to higher-value conclusions?

Reachability before LLMs

Before LLMs, reachability existed. It was constrained by human bandwidth and the friction of aggregation. That friction is now collapsing.

Pre-LLM aggregation

The classic OSINT pipeline is as follows:
— Search engines and obscure forums, sometimes in multiple languages
— Public records: contracts, filings, property documents, court records
— Academic papers, theses, conference slides
— Satellite imagery and geotagged photos
— Social media, job postings, we are hiring announcements
— Procurement notices, award statements, vendor catalogs

The limiting factor was not data availability. The limiting factor was human synthesis. You could have all the ingredients and still not have a meal, because turning ingredients into a meal required a chef: time, patience, domain expertise, and the willingness to hold fifty weak signals in your head until they converged.

How LLMs exacerbate reachability

Speed and scale

Pre-LLM, one analyst could run one line of inquiry at a time, with exhaustion as the governor. With LLMs and agents, you get parallel exploration and rapid hypothesis testing. Ten candidate hypotheses can be explored simultaneously. Contradictions can be flagged. Gaps can be targeted. The model does not get bored. It does not mind reading the procurement PDF that everyone else skipped. Security teams are used to adversaries getting faster, with better scanners, more automation, and more compute. What is different here is that the automation applies not just to exploitation, but to reasoning.

Source aggregation

Imagine Source A: a public-facing job post that mentions a new initiative and lists a few tools and collaborations. Imagine Source B: a procurement award that lists a vendor, a delivery schedule, and a location code. Each is innocuous. Together, they let an adversary infer a third thing: a timeline, a capability, or a dependency that the organization considers sensitive. That sensitivity arises not because any line says it explicitly, but because the combination collapses ambiguity.

This is the same pattern as the opening vignette. Harmless crumbs become sensitive conclusions when you can cheaply assemble them at scale.

Solution: Moyo — Red-teaming information reachability

Security teams already understand red teaming. Moyo simply shifts the object of the exercise. Instead of asking, can an attacker get in, it asks, what can an attacker deduce?

Moyo is a red-teaming system that tests how much sensitive insight can be inferred from a defined base of information, using LLM-style reasoning, so you can mitigate before adversaries exploit it.

The problem Moyo is solving

Threat model
— Our organization has a public and semi-public footprint
— An adversary may infer protected information without hacking us
— The harm comes from conclusions, not just documents

So Moyo asks:
— What conclusions become reachable?
— From which starting points?
— With what confidence and cost?
— Through what inference paths?

This is a different kind of leak audit. It is not where are the secrets stored. It is what secrets are implied by the way we present ourselves to the world.

White-box vs black-box red teaming

Moyo can be run in two modes that map cleanly onto existing security instincts.

Black-box red teaming

You treat the target like an external attacker would. Inputs and outputs only. Public-facing interfaces and allowed public data. This tests what is reachable to outsiders.

White-box red teaming

You have internal access: policies, corpora, ground truth, and maybe configurations. This lets you measure leakage against known sensitive facts and quantify how close the public footprint gets to internal truth.

The two modes answer different questions. Black-box tells you what outsiders can learn. White-box tells you how close outsiders can get to things you know are sensitive, and which combinations of public signals are doing the damage.

Leakage via inference

Define the base corpus

Public web footprint, approved documents, marketing pages, employee public posts, procurement notices, conference slides, and anything else that is in-bounds

Define protected facts or risk categories

Not necessarily classified details. Often this is operational, strategic, or sensitive business intelligence: dependencies, timelines, capabilities, locations, decision structures

Generate inference probes

The system creates questions and tasks that simulate what an adversary might try, not in a how do I do harm way, but in a what would be valuable to know way

Run iterative reasoning with evidence chaining

Moyo attempts to reach conclusions while collecting supporting evidence from the allowed corpus, tracking steps, and attempting corroboration. The key is that it does not just output an answer. It outputs the path.

Score reachability

Confidence, number of steps, required sources, novelty, replicability, and cost. A fragile one-off guess is not the same as a robust inference that any competent actor can reproduce.

Output: a reachability map

A graph: starting crumbs → intermediate claims → high-value conclusions. It highlights minimal sets of public facts that unlock the most sensitive inferences. This is the defender’s real deliverable, because it tells you where to intervene.

What defenders get out of it

A prioritized list of inference vulnerabilities

Not CVEs. Not bugs. Mosaic exposures: combinations of facts that create reachability.

Mitigation guidance

Reduce or reshape public signals. Change defaults in comms templates. Add review gates for outward-facing content. Train teams on combinations that create risk, because that is what people do not naturally see.

Continuous monitoring

Reachability is not static. Each new press release, job posting, or technical blog shifts the map. Moyo can treat publication as a change event: what did we just make newly reachable?

Why Moyo is different from traditional OSINT tools

Traditional OSINT tools collect and index. They are libraries. Moyo focuses on the inferential leap. It maps chains, measures confidence, and turns maybe into quantified risk. It treats reasoning as an attack surface.

Subscribe now

Conclusion

Security has long been framed as guarding secrets: encrypt the database, lock down access, prevent unauthorized downloads. That still matters. But it is no longer sufficient, because the world we live in leaks in a different way. We leak not just by disclosure, but by deduction.

The modern question is not only what did we publish. The modern question is what did we make deducible.

LLMs expand reachability by compressing the cost of synthesis. They do not conjure new facts. They make old facts travel farther, faster, and with less human friction. That is an uncomfortable kind of progress, because it does not look like an attack until it already is one.

Moyo is a pragmatic response: a way for defenders to see their inference attack surface, test it like an adversary, and reduce it deliberately. In a world where harmless crumbs can be industrially assembled into sensitive truth, the responsible posture is not denial. It is measurement, followed by disciplined, boring mitigation. The kind that prevents the story in the opening paragraph from becoming a headline.

It’s a call to patriotism. China versus America. “Who will you back?” This has become a common plea from the Silicon Valley elite over the last six months. I heard the move up close at the Harvard Kennedy School, where a visiting Eric Schmidt warned that AI may soon cross into autonomous self-improvement, argued that someone will need to “raise their hand” and impose limits, and then pivoted into the geopolitical register, contrasting American and Chinese trajectories and urging policy and funding choices aligned with “American values.” Others have also made versions of this argument in different forums. Tarun Chhabra, head of national security policy at Anthropic, has made a similar argument, urging an “American stack” and treating model governance as a geopolitical contest. Putting aside the awkwardness of nationalist messaging coming from the Bay Area’s long-time borderless “global citizens,” the incentives are not hard to see. If you can frame the open vs closed models debate as a national security referendum, you can frame restrictive rules as patriotism and you can frame “responsible control” as synonymous with dominance by a small circle of incumbent providers.

The posture makes sense once you consider two facts. One: industries which may live and die on capricious regulatory rule making must make their case to those with their hands on the levers of power. In 2026 America, those hands are professed patriotic Republicans. Two: Big Frontier LLM is losing the tech battle, or at least losing the easy assumption that America’s lead is automatic and permanent. They are on their back foot so they must frame the open vs closed model debate wrongfully as a fight between America and China. “America cannot afford to lose a battle to China and by extension Anthropic, OpenAI and Alphabet cannot afford to lose to their competition.”

Yet there is nothing inherently Chinese about open models and nothing inherently American about closed models. If anything, it should be the opposite. Open models are decentralized, inspectable, forkable, and difficult to monopolize. That aligns with an American instinct to diffuse power, prefer competition over permission, and distrust single points of control. Closed models concentrate capability behind a small number of gatekeepers, wrapped in secrecy, and sustained by privileged access to regulators. That logic is far closer to centralized control than to open competition. The real fault line is not America versus China. It is democratic diffusion versus unnatural scarcity, and good tech versus bad tech.

Regulatory Capture?

Safety arguments against open models can be institutionally self-serving. The likely political economy of strict open model controls is that compliance becomes a fixed cost that only large incumbents can bear. New technology regulation can be shaped by the most powerful actors to protect or expand their advantage. The concern for this paper is not that every safety proposal is capture, but that a legal regime designed around the idea that “only a few trusted providers may exist” is structurally aligned with incumbent interests.

Valid Safety Concern

That said, I wouldn’t pretend the case for safety is purely cynical. A useful example is widely accessible virology. The life sciences have lived for decades with the uncomfortable fact that biological knowledge is often dual use: the same methods that teach you how pathogens spread, mutate, or evade immune response can also lower barriers for malicious replication or reckless experimentation. In domains where the object of study is intrinsically hazardous, knowledge dispersion can be dangerous. However, we should also consider that much of what is dangerous is already public and incorporated into existing LLMs. Trying to retroactively censor it would be impossible.

The more practical policy is forward-looking. We can prevent additional damage by moving classification upstream: treating certain classes of data as hazard-enabling at scale, even if they have been, until now, classified as “public.”

Why are closed models bad tech?

Distillation is the mechanism by which “closed” models leak or are stolen by competitors. In distillation, a smaller student model is trained to imitate a larger teacher by querying it at scale and learning from its outputs. In practice, that means once you ship a frontier model behind an API, you have created a surface that others can use, legally or not, to train imitators; released systems are already being distilled against, and the industry has begun openly fighting over it. The “closed” advantage, then, is not a durable moat so much as a temporary lead, especially because open models are now only about three months behind the state of the art on average, and the gap is shrinking.

That is why closed model maximalism is bad tech: it asks the public to bankroll a capability edge that can often be copied down the stack. Distillation is a one-way process. Once a a model’s weights exist in the wild, the knowledge it was trained on will too. The result is that a great deal of public money invested in frontier systems may buy us, at best, half a generation of advantage before that advantage becomes baseline.

The AGI Finish Line

This massive public investment can be justified if we are, in fact, in a race to a specific finish line. Perhaps we can call that finish line AGI and the first firm to build it will reap the infinite benefits of having built GodGPT. Sam Altman and his peers would like you to believe there is a decisive summit and a single winner. I will not make the argument here against our ability to build GodGPT. I will merely ask the reader: if we cannot build it, what exactly has our public investment bought us? How much of our policy architecture depends on GodGPT being real and imminent? And do the frontier model builders have an interest in keeping us convinced that it is?

Where will economic value come from?

Assuming we do not build God(gpt), I see the value added from AI falling into two broad operational modes. First, LLMs, call them agents if you want, will replace or enhance existing labor and raise productivity inside the work we already do. Second, frontier LLMs will be used by humans to create new technology and, through new tech, raise productivity. The open versus closed question looks very different in each mode.

Start with labor enhancement. Here, I struggle to see where a closed model creates durable value that an open model cannot. Again, the leading open models are only about three months behind the best closed models on average, and that gap has been narrowing over time. More importantly, they will never be less competent than they are today. LLM capability, as it stands today, is already powerful enough to transform knowledge work once the workforce is educated on how to leverage it effectively. So beyond branding, default status, and the inertia of being “what everyone uses,” it is hard to imagine how a three-month lead translates into meaningful economy-wide productivity returns compared to open competitors.

There is more reason for optimism, and more reason to take the frontier seriously, when it comes to creating new tech. Even without accepting a God-like AGI, we have already seen systems that look like they are generating new knowledge, or at least synthesizing massive amounts of dispersed knowledge in ways humans cannot efficiently. The frontier labs will have an advantage here because the marginal discovery can be expensive, and the best-resourced models can search more of the space, faster. But we should also be careful with the race metaphor. The knowledge search space is not a straight track with a single finish line. Think of it more as a multi-dimensional blob with advances in all directions from our current knowledge blob. That changes the economics. Progress is not one AI outrunning another. It is billions of human and AI teams pushing outward in parallel. If a cheaper but slightly slower AI can be put in the hands of a billion knowledge seekers, it may create more new knowledge than a $200-a-month model in the hands of only one million. And if LLM capability progress has diminishing returns, any frontier lead that depends on scale alone becomes harder to defend over time.

Then there is the profit model question. Frontier firms will face pressure to turn a profit to repay the massive investment they have taken. They can do this through enterprise subscriptions, usage-based pricing, and through business-facing products that make organizations more productive. This is what Anthropic claims is their plan. The business value here comes from the delta between the knowledge frontier firms are capable of generating and the knowledge open models can generate. It is possible that the delta will be significant. It also may not be (and shrinking with time). The least we can say for now is that it is uncertain.

They can also profit through advertising. The largest pool of users is the pool that does not want to pay very much. OpenAI has now publicly moved toward testing ads in ChatGPT for some users. Big Frontier Model seems reluctant to talk about this trajectory because it sounds like an admission that they are just like every other tech platform, and should be valued as such. It also puts them in direct competition with existing ad giants. For the last couple of years, what we perceived as a performance advantage of LLMs over traditional Google search may often have partly been a mirage - a feeling of refreshment at having escaped advertisement-suffused search for the first time in decades. The ad model has another structural problem. There is no durable barrier against exit toward free, or significantly cheaper, open models once they are “good enough,” which they increasingly are. If the closed-model future is subscriptions plus advertising plus lock-in, then the public is effectively subsidizing the creation of a new, enshittified ad service with a thinning claim to unique value, and with a user base that can walk away the moment the open alternatives cross the usability threshold.

The Difficulties of Moratorium Enforcement

The policy debate often assumes a stable choice between “closed models under responsible control” and “open models in the wild.” In practice, a model can start closed, then leak, then become open in effect. The LLaMA 1 leak is one example. Meta did not initially release the model as a general public artifact. It was meant for a controlled research release, and the weights leaked online within days, spreading through channels like 4chan and torrents. These models are big, but not that big in practical terms. They don’t require a data center for storage. The significant IP is measured in the hundreds of gigabytes and small enough to copy, mirror, and pass around through ordinary internet infrastructure. Physically speaking, this data could be carried in someone’s pocket. Effectively preventing use of an open LLM would require inspectability of all digital media, moratoria on encryption, and unprecedented visibility into network traffic.

The moratorium problem becomes even less credible once it relies on international cooperation. The Bletchley Declaration is a useful illustration: it recognizes that many AI risks are international and calls for cooperation, but it is fundamentally a political declaration rather than an enforceable regime. The cooperation required for Bletchley is extremely mild compared to what would be needed to enforce a moratorium against what is effectively a small amount of data / software. The plausible outcome is uneven restriction: some jurisdictions ban open releases, others become havens, and diffusion continues anyway.

Subscribe now

Conclusion

The open versus closed fight is not America versus China, even if that framing is politically convenient. It is convenient precisely because it converts a messy argument about market structure, democratic control, and technological diffusion into a simple loyalty test. American Big Frontier Model have a vested interest in that narrative. If you can convince lawmakers that “closed” is patriotic, you can turn regulation into a moat and public money into a subsidy. The risk is that we keep throwing good money after bad, paying repeatedly for a thin, temporary lead while the underlying capabilities diffuse anyway. Models do not unlearn and capability inevitably spreads. The wiser posture is to stop moralizing the architecture and lean into open models and the model agnostic tech we can build on top of it.

Thank you to those who have signed up since my last update. Please share if you know of anyone who would be interested.

In this Substack, I will write periodically in order to:

1) Share my thoughts on cybersecurity, AI, foreign policy and tech regulatory policy based on my experience as a Cyberwarfare Officer, software developer, public policy student at the Harvard Kennedy School and now founder.

2) Keep you up to date on company progress.

I will plan on a monthly (ish) roll-up of my writings along with significant SenTeGuard news.
.
.
.

.
This month I have a lot to roll-up. I am publishing a lot of my pieces that, until now I have kept to myself and my professors. Below are some links. Happy to hear feedback / push back.

Broad Policy Articles:

1) Nailing Jell-O to the Wall, Again. Can China Contain LLMs?
How might LLM tech negate CCP control of information. Substack

2) OracleGPT: A Thought Experiment on an AI-Powered Executive
What if a President had access to a high-powered LLM trained on and with visibility over the entire real-time classified universe? Maybe he does already. Substack

3) American Closed-Source v. Chinese Open-Source: A False Dichotomy
America should embrace Open-Source models and model agnostic tech. Substack

4) The Limits of LLM-Reachable Intelligence.
Where can humans fit in in an Agent dominated economy. (Theory) Substack

SenTe Focused Articles:

1) Living with LLMs Everywhere. How Ambient LLMs Negate Security Policy.
Your data is being incorporated into LLMs whether you like it or not. Substack

2) Moyo, Sensitive Information Reachability: The Problem and The Solution
Introducing Moyo. A Cognitive Security red-teaming tool. Substack

3) Cognitive Security Standards: Statement of Purpose and Concept (v0.1)
Guidelines to protect your organization from valuable idea leakage. Substack

4) What is SenTe?
The namesake behind my company. A term from the east Asian game Go (or Baduk in Korea) meaning “initiative”. With SenTe, a player has control of the opponent who is on the back foot. Substack

5) Coming soon:

• A one-click downloadable demo, password protected on the site. Let me know if you’d like to try it.

• Joseki. Shareable rubrics for building with and breaking models.

From now on I will be sending updates through Substack. I will be providing substantive blog posts about the broad state of AI-Cybersecurity from a business, policy and high-level technical perspective. Please share with anyone who may be interested and visit my website blog for a longer form discussion of the topics I will introduce in these emails.

Earlier this month I had the privilege of attending a 3-in-1 conference in Tel Aviv. This event included Cyber Week Tel Aviv, AI Week Tel Aviv, and the AI Security Forum. In this and future blog posts, I will distill what I have learned into bite-sized chunks.

The new baseline

Across industry and government, the pressure to use LLMs like ChatGPT, Claude, Gemini, and Copilot is no longer optional.

Timelines are tighter, procurement cycles are faster, competitors are already automating knowledge work and teams need the productivity boost just to keep up.

The fastest way to get good output is to paste real context.

Unfortunately, this is also how strategy can become reconstructable.

A defense-tech scenario

Picture a defense-tech program team at a closed-door offsite. They are deciding which mission set to own, how to price a platform plus sustainment model, and which autonomy bets to prioritize.

They use an LLM for brainstorming, synthesis, and board-ready prose.

To make it useful, they paste churn drivers, objection maps, and internal margin thresholds. No single classified document. Just truthful fragments.

By the end of the day, the logic of the strategy is encoded in prompts and drafts.

A month later, there was no breach or vulnerability exploited.

No exfiltration.

Yet a competitor’s plan mirrors the same wedge and sequencing.

This is not just a defense tech issue

This inferability problem shows up everywhere.

In aerospace programs, it is teaming choices and propulsion roadmaps.
In energy and critical infrastructure, grid hardening priorities and procurement timing.
In biotech and pharma, trial site strategy and endpoint rationale.
In small doctors offices, patient details, referral sources, payer mix thresholds, and denial patterns.
In money managers and law firms, client facts, settlement ranges, investment constraints, and risk limits.

The right response

The answer is not banning LLMs. It is giving teams the confidence to use them safely.

That is what SenTeguard is built for, with three modes:

• Blocking- to prevent strategy-bearing content from leaving controlled environments

• Alerting- to prompt manual review

• and Flagging- to record risky leakage patterns passively

Read my blog for a more in-depth discussion of these issues.

The longer-form post goes deeper on how “truthful fragments” turn into reconstructable plans, why this threat is accelerating right now, and how to build guardrails that do not slow teams down.

What this series will cover

In this new year’s blog series I will discuss AI-security, governance, inferability risk, and how teams can be more effective in the LLM era.

Subscribe now