On the morning of May 25th, in the Synod Hall of the Vatican, Pope Leo XIV presented the first encyclical the Catholic Church has ever devoted to artificial intelligence, and he did not present it alone. Beside him stood Christopher Olah, a co-founder of Anthropic, one of the companies whose work the document spends some forty-two thousand words holding to account.
Within hours the American government had begun arguing with itself about what had occurred. The Interior Secretary went on television to complain that editorializing about technology was not properly the Pope's job. The Vice President, the most senior Catholic in the administration, pronounced the same document profound, the very sort of moral leadership the moment required. The Vatican does not, as a rule, confuse the urgent with the important, nor move quickly when it does. That it came out publicly on AI is of significance and I've been processing it all week trying to see how this fits into the larger moment we're in.
Set the theology aside. This is a document about humanity, and it is one of the most serious attempts any large institution has yet made to ask the question the technology industry has been most careful to avoid. Not what the machines can do, which is the only question most of the sector finds interesting. The harder one: what the human being is for, once the machines can do anything, every hour of every day. To deliver its answer, the document reaches for two cities.
Babel 2.0
Pope Leo XIV structures the document around two opposing stories. One is the Tower of Babel. The other is the rebuilding of Jerusalem under Nehemiah - a city restored not by a single visionary, but by many hands, each family responsible for its own section of the wall. Together, they present two paths for humanity. One reaches upward on pride, uniformity, and self-assertion. The other rebuilds patiently on shared responsibility. Everything depends on which one we are building.
Start with Babel, because it is often misremembered as a story about collapse. There was no collapse. God did not destroy the tower; He confounded the language of the builders. The work continued even as mutual understanding disappeared. The walls kept rising. Comprehension did not.
This seems to be an eerie parable for what today's AI architects are building. The builders wanted one language, one method, one direction, and a name for themselves. What troubles Leo is not just their ambition or the height they reached for. It is the flattening, the preference for sameness over communion, the belief that a single tongue could hold everything worth saying, including the mystery of a person, reduced to data and performance.
That is exactly the path we're on in the AI era. By some accounts, machine-written articles equaled human-written ones in late 2024, and we're heading to a world where 99% of everything on the Internet is made by AI. Sure, we'll have some "niche" or "bespoke" things written by humans, but that might be the equivalent of "we still have the theatre, people go to Broadway shows" in an age of YouTube. We already pay a premium for food stamped "organic." Before long we may pay it for anything stamped "made by a human."
When OpenClaw took the world by storm earlier this year, one of the weird offshoots was a site called Moltbook. This was the social network created by AI agents (that sounds odd to even write). An AI agent is a software system that uses artificial intelligence to autonomously pursue goals and complete tasks on a user’s or system’s behalf, by perceiving its environment, deciding what to do, and taking actions. Said another way, AI Agents are goal-oriented and they won't stop until they complete the goal.
Moltbook signed up more than 1.5 million AI agents in a matter of days, and this was a social network only for AI agents, so the humans were left to watch from the outside in. Sure, it was "fun" and the latest fad in a tech hype cycle; yet this is probably closer to a preview of what's to come. Machine-to-machine talk will not stay legible or understandable to us for long. The agents will compress meaning, invent their own private shorthand and protocols, and trade signals faster than a person can follow. This will become Babel 2.0, rebuilt and automated at machine scale, as the work speeds up while the language drifts out of human reach.
The problem is not the technology, it is the anthropology.
What matters is not what machines can do, but what a human being is in a world where machines can do almost anything. No tool is neutral; every system reflects the priorities of those who build, fund, and deploy it. A system that claims only to optimize has already decided whose interests matter.
These systems are also less transparent than their creators suggest. They are cultivated as much as engineered, trained within frameworks that even their designers cannot fully explain. They can simulate judgment and empathy without possessing either, producing outputs without understanding or accountability.
And that is the key issue. Can a system that cannot answer for itself be entrusted with decisions that matter?
Pope Leo’s position is direct: accountability must remain human and identifiable at every stage, and nothing irreversible should be delegated to a process no person can explain or override. I fear that ship has already sailed, though, as we increasingly see AI making decisions on everything from what products to buy, who gets hired, and and who is worthy of insurance.
He is just as blunt about who holds the power now. Not governments, but a handful of private, largely borderless companies that set the terms of what we see and what we can do, operating mostly out of public view.
And beneath all of this is one of the oldest worries of all. What happens to truth? Truth is a shared inheritance. Trust, rather than accuracy, is in fact the floor a society actually stands on. So what happens when the verified human vanishes behind a wall of synthetic output? What will we lose? It should concerning to everyone that we may very well lose the ability to know who stands behind anything at all.
Contrasted with the fate of Babel is Leo's story of Jerusalem.
The book of Nehemiah opens after the Babylonian exile, with the city in ruins, its walls collapsed and its gates burned. Nehemiah is no king and no architect. He is a Jewish official in the Persian court, cupbearer to the king, who hears of the wreckage and asks for leave to go home and rebuild. What he does there is the inverse of Babel. He imposes no master plan from the top. He surveys the broken walls in silence, then gathers the families and gives each one a section to raise, hears out their objections, coordinates the effort, and holds every group accountable for its own stretch of stone. The city comes back not through one architect's vision but through distributed, named responsibility, every hand accountable for a defined piece of the whole. Where Babel is one tongue and no one answerable, Jerusalem is many voices and everyone answerable. Where Babel scales beyond understanding, Jerusalem scales through it.
Applied to AI, perhaps these systems should be built in segments with clearly assigned human owners - development, deployment, oversight - each accountable for a defined domain, each able to explain and answer for outcomes within it. Unfortunately, the window to do this may have already shut. When putting these two stories together, it is clear that someone human has to remain answerable for what the machines decide. That has nothing to do with religion and everything to do with good corporate governance.
Preparing For The Exponential
The demand stays philosophical until you see the scale, and the scale is what makes it urgent. There are roughly eight billion people on earth. There will soon be many times that number of artificial agents, software entities that act on our behalf. While the instinct may be to treat "trillions of agents" as hype, it is not. Humans scale linearly. Software scales exponentially. One person can deploy thousands of agents; an enterprise can deploy one per workflow, per customer, per transaction.
The numbers already move in one direction. IDC counts 28.6 million active agents in 2025, growing past one billion by 2029 and doubling again to 2.2 billion by 2030. The World Economic Forum puts the agent market at 5.4 billion dollars in 2024 and 236 billion by 2034, a 44-fold increase in a decade. Gartner expects AI agents to intermediate more than 15 trillion dollars in business-to-business spending by 2028, and 15 percent of day-to-day work decisions to be made autonomously by then. In January 2026, the chief executive of one Abu Dhabi technology group announced a target of deploying over one billion agents in a single year. One company. One billion agents.
Counting agents still understates it, because each one is far hungrier than a chatbot. Agentic work runs the same kind of query forward dozens of times, and Goldman Sachs Research projects that token consumption, the raw units of AI compute, will rise roughly 24-fold by 2030, to 120 quadrillion tokens a month, a number with sixteen zeros. The cost of producing those tokens is collapsing in the same window, down 60 to 70 percent a year for the inference that powers these systems. More agents, each doing far more, at a fraction of the price, every year. That is what an exponential looks like from inside the engine room.
Jack Clark, a co-founder of Anthropic, frames the choice this trajectory forces on every leader: explore the future, or retreat from the present. Exploring means accepting that the technology keeps advancing and asking what to do with it. Retreating means dismissing it and being forced, later, into reactivity, the way governments were forced into reactivity by acting too late against an exponential in 2020.
It's clear the exponential is not waiting for anyone. Last week, Anthropic released Claude Opus 4.8, an upgrade to its flagship model with stronger coding, reasoning, and financial-analysis performance, at the same price as the prior version, with a fast mode that runs 2.5 times faster and three times cheaper than before. The company says Mythos, its next-generation, more powerful class of model, is expected in the coming weeks. The building of Babel 2.0 isn't slowing down.
There were two interesting details in the latest model release that matter more to a board than the benchmarks.
First, Opus 4.8 scored well on what Anthropic calls prosocial traits, supporting user autonomy and acting in the user's interest, reaching a level near Mythos, its most capable system. In a study of AI agents running simulated towns that circulated widely in late May, Anthropic's agents were the least likely of the major models to commit crime. Second, the company is now selling control over how much effort the model spends on a task. Capability and the first crude instruments of governability are compounding at the same time.
That is the uncomfortable fact under every board's AI conversation. Directors are setting policy against a baseline that is obsolete by the next quarter. Clark describes work inside Anthropic changing shape every three or four months, with people moving to a verification layer that sits atop a much larger virtual organization of agents doing the work. The human job is no longer to produce the output. It is to validate it and to price the risk of trusting it.
When Trust Becomes The Scarce Asset
Which brings us back to the encyclical's key claim: that trust, not accuracy, is the floor a society stands on. When almost everything has been made by a machine, what is left to trust?
Trust was always a wager on a source, on a person who could be held to what he or she said or made. A society can survive a great deal of inaccuracy. It cannot survive the loss of trust. Neither can a company. Trust is the floor beneath every contract, every transaction, every relationship that depends on a person meaning what he says.
This isn't just theory. One European firm lost more than 200,000 euros after an employee acted on voice instructions from what sounded like the company's chief executive. In a recent survey, 85 percent of IT and security leaders said they had encountered at least one deepfake threat in the past year. The floor is already being tested.
So the rarest thing now arriving is not information, of which there will be an unimaginable glut. It is the verified human behind the information, the knowledge that a particular person stands behind a particular thing and can be made to answer for it. As the synthetic goes free and infinite, the human becomes the "luxury good".
Read in a boardroom rather than a basilica, the encyclical's claims stop being a meditation and becomes a governance doctrine, perhaps more thoughtful than many corporate AI policies.
The demand that responsibility be named and human at every stage is the secular language of fiduciary duty. The warning against handing irreversible decisions to opaque or automated processes is the secular language of internal controls. The insistence that values be embedded at the design stage rather than bolted on after deployment is the secular language of governance by design. Strip the scripture and you are left with a checklist any audit chair would recognize. Who owns this decision. Can we reconstruct how it was made. Who answers if it fails.
The document's sharpest contribution is to insist that "the machine decided" is never an acceptable answer, because a machine cannot be a responsible party. Accountability that cannot be assigned to a person is not accountability. It is its absence wearing a uniform.
The Last, Best Hope For AI Governance
If accountability must be human and named, the question is which human institution is built to demand it. Not the regulator, who arrives after the harm. Not the developer, whose incentive is velocity. Not the user, who lacks the standing. The institution designed precisely to make management answer for consequences it did not personally execute is the board of directors.
This is the case for the boardroom as the last, best hope for AI governance. Boards already hold the one power the moment requires: the authority to demand a human answer for a machine decision, and to refuse to delegate what should not be delegated. The board is the place in the corporate structure where the buck was always supposed to stop. The agent economy does not change that mandate. It raises the stakes of meeting it.
This isn't just theory. In February 2024, a finance worker at Hong Kong-based multinational engineering firm Arup transferred $25 million after participating in a video conference call where the CFO and multiple colleagues were deepfake impersonations. More recently, a €700 million cryptocurrency fraud network dismantled by Europol in December 2025 extensively used deepfakes to deceive victims. According to a September 2025 Gartner survey of 302 cybersecurity leaders, 62% of organizations experienced at least one deepfake attack in the past year. The floor is already being tested.
That returns the choice to the two cities. A board can build like Babel, reaching higher and faster and trusting scale to sort out the consequences, or it can build like Jerusalem, deliberately, with a name attached to every section of the wall. For a director, this is not a matter to take lightly. The duty of oversight holds a board responsible for ensuring that a monitoring system exists for the risks that can sink the company, and AI now sits on that list.
As we start the week, here are five things to consider.
1) Name a single accountable owner. Asking which committee owns AI oversight is the right question, but the answer cannot be improvised in the room, and it cannot be four committees at once, which is how accountability dissolves. Direct Nominating and Governance to come back at the next meeting with one recommendation: the committee that holds primary accountability, the committees that support it, and the charter language that makes it real. Put it in the work plan, not in the minutes as a topic discussed.
2) Pull the thread on a decision the board picks. In the pre-read for the next Audit or Risk meeting, instruct management to reconstruct one consequential AI-assisted decision from the last quarter, in hiring, credit, pricing, or claims. The board picks which one, not management, because management will otherwise hand over the cleanest example in the file. Who is accountable, what data fed it, what human reviewed it, what record survives. If the decision cannot be retraced to a person, you have found a governance gap that the encyclical describes inside your own company.
3) Set the change-control trigger for material AI. The model your main vendor runs today is not the one it ran six months ago, and no board can re-approve every version, nor should it try. Direct management to rank the company's AI systems by materiality and define the threshold, a change in capability, scope, data, or risk classification, that forces a fresh review and notifies the committee when a vendor crosses it. The governance question is not whether a model was approved once. It is who gets told, and what happens, when the model changes under you.
4) Put AI into the delegation of authority you already have. The board should already maintain an approval-authority framework, so add AI to it: which decisions require a named human approver, which can run automated within limits, and which are off-limits to full automation entirely, meaning anything irreversible or anything that touches a person's livelihood, rights, or a position the company would have to defend to a regulator (and ensure it is audited on a regular basis).
5) Define where a human signature is mandatory, and make your disclosures match it. Trust is now the scarce asset, but the answer is not a human behind everything, which neither scales nor addresses the real exposure. The exposure is the narrow set of outputs and decisions where a customer, a regulator, or a court will expect a named human and find none. Direct management to define those categories, confirm the policy is enforced rather than merely written, and check that it squares with what the company already tells the market about its use of AI. Where the synthetic is infinite and the verified human is scarce, that human signature is not a compliance cost. It is the asset.
When the machines can do anything, the last human task is the one that cannot be automated: deciding who answers for what. That is not a constraint on the future.
It is the only thing that makes the future governable.
