AI governance was designed to control models. The next enterprise architecture must govern what humans and agents are allowed to know, decide, remember, and do together.
Most companies are adding AI to the enterprise. Few are redesigning the enterprise for AI.
We are putting copilots into functions, agents into workflows, and language models between people and the information they use to decide. Yet the organization around them still assumes a familiar world: software executes instructions, people exercise judgment, records preserve what happened, and authority follows the org chart.
That world is disappearing faster than most governance programs can adapt.
An agent can now find evidence, interpret a policy, compare alternatives, recommend a decision, write the rationale, update a system, and leave behind a summary that another human or agent will later treat as context. It does not merely process information. It participates in the production of organizational reality.
This is the shift that matters. The enterprise is becoming a human-agent system.
Organizations are not just collections of people, software, and data. They are systems for deciding what counts as true, who may decide, what may be done, what must be remembered, and who answers when something goes wrong. AI is entering all five questions at once.
I use AI-nterprise to name the institutional architecture this requires: a way to coordinate humans, agents, models, claims, and rules so that an organization can know, decide, act, and learn without allowing authority, accountability, or memory to dissolve into the workflow.
It is not a product category. It is not a model class. And it is not another compliance checklist.
It is a design problem for the enterprise itself.
The enterprise has acquired a new kind of actor
Traditional enterprise software could be complex, but its role was legible. A system stored a record, applied a rule, calculated a result, or routed a task. If the business rule changed, someone changed the software.
Agents blur that boundary. They do not only execute a predefined path. They select which evidence matters, infer what a request means, choose among tools, generate intermediate reasoning, and decide what to do next. Their behavior is partly specified in code and partly produced at runtime.
This does not make an agent a legal person, a colleague, or an autonomous institution. But it does make it an actor in the practical sense: it can alter the premises on which other actors behave.
That distinction is more important than the familiar question of whether AI will replace jobs. A job is a bundle of tasks. An enterprise is a distribution of authority and consequence. Once agents participate in decisions, the central question is no longer only what can they do? It is what are they entitled to make true for the organization?
An agent may be able to draft a policy. That does not mean it may establish one. It may be able to retrieve a customer record. That does not mean it may expose every representation of it. It may be able to recommend a payment, a diagnosis, a hire, or a control exception. That does not mean its recommendation has standing, or that any downstream system should act on it.
Capability is not authority.
The human-agent enterprise begins by refusing to confuse the two.
Governance is looking at the wrong object
Many AI governance programs are organized around models and applications. They inventory systems, classify use cases, evaluate vendors, test outputs, document risks, and assign accountable owners. All of this is necessary.
It is no longer sufficient.
The object that travels through the enterprise is rarely the model. It is the claim the model helped produce: this customer is eligible; this contract contains a material risk; this incident has been resolved; this forecast should guide the plan; this employee should receive an exception.
The claim moves. It enters a presentation, a ticket, a knowledge base, a CRM, a policy, a line of code, or another agent's context. Its origin becomes less visible with every handoff. A provisional answer is copied into an official document. A generated summary becomes the premise for a later decision. Several agents repeat the same unsupported conclusion and create the appearance of consensus.
By the time the consequence arrives, asking which model produced the first sentence may be less useful than asking:
- What exactly was asserted?
- Which version was relied upon?
- For what purpose was it accepted?
- Who or what was allowed to see it?
- What action did it enable?
- What happened when it was corrected?
Provenance can tell us where a claim came from. It cannot, by itself, tell us what the organization is entitled to do with it.
Output is not knowledge.
The difference between the two is institutional standing.
Four decisions we must stop collapsing
A human-agent enterprise needs to make four decisions explicitly. They are related, but they are not interchangeable.
Epistemic standing: For which purpose, in which context, and at what level of consequence is a claim sufficiently warranted?
Persistence: Should the claim be retained, for how long, in which version, and under what correction or disposal rule?
Visibility: Which person or agent may retrieve which representation of the claim and its supporting context?
Action enablement: Who or what may act on the claim now, within which boundary, and with which escalation path?
A claim may be credible without being permanent. It may be retained without being authoritative. It may be visible to an auditor but unavailable to an operational agent. It may be accepted for exploration and forbidden for execution. An actor may be permitted to act on it in one jurisdiction, function, or time window and not in another.
Collapse these distinctions into a confidence score or an access-control list and the enterprise will make category errors at machine speed.
Seeing is not believing. Remembering is not accepting. Accepting is not authorizing action.
The architecture must preserve those differences even when the workflow would prefer to erase them.
“Human in the loop” is not a governance model
The phrase sounds reassuring because it places a person somewhere in the process. It says almost nothing about whether that person still has meaningful power.
A human can be present and still function as a ceremonial approver. If the person sees the model's conclusion before forming an independent view, the system has already framed the question, selected the evidence, and established an anchor. The interface may then record a click called “review” while the organization records no human judgment distinct from the machine's.
Sequence matters.
For consequential decisions, there are cases where the human view should be captured before machine advice is revealed. The point is not to isolate people from useful assistance or to romanticize unaided judgment. Humans bring bias, inconsistency, hierarchy, and self-protection into decisions. The point is to preserve a recoverable difference between human judgment and machine influence.
Only then can the organization ask useful questions: Did the advice change the decision? Why? Was the change an improvement? Did the reviewer defer outside their competence? Did the same pattern recur across teams or agents?
Meaningful human power is also more specific than universal approval. It belongs where people must define purpose, set boundaries, judge consequences, resolve conflicts of interest, authorize exceptions, and order correction or legitimate forgetting.
The goal is not to keep a human hand on every task. It is to keep human authority at the points where society, law, and the organization expect someone to answer for the result.
Human presence is not meaningful human power.
From workflow automation to institutional architecture
The answer is not a larger approval matrix. Nor is it one monolithic platform through which every decision must pass.
The human-agent enterprise needs three distinct planes.
The knowledge plane governs claims, evidence, versions, standing, and correction. The authority plane governs roles, permissions, thresholds, exceptions, and obligations. The assurance plane tests whether the first two work as intended and whether the evidence is strong enough to justify continued reliance.
Those planes must support three loops.
The production loop creates and revises work. The assurance loop challenges claims, tests controls, and detects drift or failure. The learning loop ensures that verified corrections change future retrieval, evaluation, policy, and action.
The learning loop is the one organizations most often neglect. A finding enters an audit report. A reviewer corrects an agent's output. A team discovers that a trusted source was wrong. The incident is closed — but the correction never reaches the prompts, retrieval paths, policies, or decision rules that will shape the next case.
The enterprise observed an error. It did not learn from it.
An assurance process that cannot change future behavior is not a control. It is documentation.
This is why the AI-nterprise is an institutional architecture, not merely a technical stack. The controls may be implemented across identity systems, policy engines, knowledge graphs, audit records, workflow tools, and agent runtimes. What matters is not that one product owns the whole design. What matters is that the distinctions survive across the whole system.
A manifesto for the Human-Agent Enterprise
The principles are simple to state. Their consequences are not.
They are the ten principles of the white paper, stated here for leaders rather than for architects.
1. Draft by default. A generated claim is born without standing, without a place in memory, and without the right to trigger action. Nothing it does on its own changes that.
2. Make promotion explicit. A claim enters reusable memory only through an act someone owns, for a stated purpose, on a stated evidence basis, with a rule for when it leaves. “Accepted” is incomplete without “accepted for what.”
3. Let evidence travel. Qualifiers, uncertainty, counterevidence, transformations, and dependencies stay attached to the version. A paragraph copied into a new system does not become a new truth.
4. Treat provenance as necessary but insufficient. Knowing where a claim came from says nothing about whether it is warranted or who may act on it. A trace is not a warrant.
5. Keep the four dimensions separate. Visibility, standing, persistence, and action rest on distinct policy bases. Access does not confer truth, storage does not confer acceptance, and no system may manufacture its own authority: an agent acts only within a mandate granted elsewhere, and greater capability does not silently enlarge it.
6. Scale scrutiny with consequence and persistence. Expected harm, irreversibility, affected scope, uncertainty, and reuse set the intensity of control. Exploration and clinical action do not deserve the same gate.
7. Preserve independent judgment when warranted. Where consequences justify it, capture the human view before the advice, the change after it, and the reason for the change, so that judgment stays distinguishable from persuasion.
8. Make challenge and removal first-class. Contestation, supersession, retraction, repair, and forgetting are ordinary operations. A memory that cannot propagate corrections preserves error; one that cannot forget becomes liability.
9. Require assurance to change the system. Material findings enter owned remediation and later verification. Otherwise the same failure returns wearing a different sentence.
10. Own the institutional learning loop. Context, evaluations, corrections, policies, and governed memory must survive the substitution of any model. The model may be rented; the learning must be yours.
These commitments are not anti-automation. They are what make deeper automation possible without making the institution unintelligible.
What leaders should do now
The first step is not to buy another tool. It is to change the unit of analysis.
Map decision surfaces, not only AI systems. Identify where generated or machine-mediated claims acquire standing, enter memory, cross visibility boundaries, or trigger action. A low-profile summarizer may create more institutional risk than a famous model if its output quietly becomes precedent.
Define claim and consequence classes. Classify uses as ephemeral, bounded, material, or high-consequence — and prohibit some uses outright. Then specify the evidence, human authority, retention, and escalation each class requires.
Separate the four decisions in policy and architecture. Do not let storage stand in for acceptance, access stand in for authority, or a model score stand in for organizational warrant.
Redesign human review around independent judgment. Reserve decision-before-advice, recorded rationale, and contestability for the points where anchoring or deference could materially change an outcome. Do not waste scarce human attention on ritual clicks.
Close the assurance-learning loop. Test whether a verified correction changes what the next person or agent will retrieve, treat as warranted, and be allowed to do. If it does not, the organization is accumulating reports rather than knowledge.
None of this requires waiting for autonomous agents to become perfect. It requires accepting that imperfect agents are already entering imperfect institutions — and that the combination will not govern itself.
The enterprise after software
The internet made the enterprise networked. Cloud made infrastructure elastic and software increasingly rented. AI is doing something more radical: it is distributing the production of judgment.
That creates extraordinary leverage. It also removes many of the frictions that once made authority visible. The memo had an author. The policy had an owner. The analyst could be questioned. The system executed a rule someone had written. In an agentic workflow, evidence selection, interpretation, recommendation, execution, and memory can happen inside a chain that no participant sees in full.
The answer is not to freeze the enterprise in the human-only past. It is to design an institution capable of carrying distributed intelligence without losing the ability to explain itself.
The defining question is no longer how much AI the enterprise uses. It is whether the enterprise can still tell how something became known, who was allowed to act on it, what changed when it was wrong, and who remained accountable throughout.
An enterprise with agents does not yet have an AI-nterprise.
It has one when intelligence can be distributed without making authority, accountability, and memory disappear.
This article introduces the argument developed in the open white paper AI-nterprise: Governing What the Organization Knows (2026): https://doi.org/10.5281/zenodo.22282125. The white paper credits the research this argument builds on and states what is and is not new here.
I am continuing to explore these themes through research and advisory work. If your leadership team is redesigning how humans and agents share knowledge, judgment, and authority, I am open to those conversations.