Behind The Anthropic And Vatican Partnership: How An AI Startup Is Pushing For Ethics

The internet spent a full day joking that Pope Leo XIV had joined Anthropic after the Vatican invited Anthropic co founder Chris Olah to speak about AI. Social media users posted fake announcements, edited images and mock recruitment headlines after the Pope publicly thanked Olah during a Vatican gathering.

Business Insider published an article explaining that the Pope was not secretly joining the San Francisco startup. The confusion started after the Vatican selected Olah to help present “Magnifica humanitas: On safeguarding the human person in the time of artificial Intelligence”, a new encyclical about AI from Pope Leo XIV.

During the event, Pope Leo XIV thanked Olah for attending and promised to work together to “find a way for humanity in this time of artificial intelligence”, according to Business Insider. That sentence alone gave the internet enough material for an entire day of jokes.

Behind the memes there’s a more serious discussion about AI and ethics. Anthropic has spent years positioning itself as one of the AI companies most interested in safety and ethics, and as Business Insider reported, the startup even employs a philosopher to help guide the behaviour of its Claude chatbot.

The Vatican invitation gave Anthropic something tech companies rarely receive, direct engagement with one of the world’s oldest moral authorities. That relationship makes sense because both sides keep talking about human dignity, labour and the social effects of AI systems.

Why Does Anthropic Want Moral Guidance?

Chris Olah spoke unusually openly about the problems facing AI companies. “Every frontier AI lab, including Anthropic, operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing,” he said. “The pressure to stay commercially viable and to stay at the research frontier. Geopolitical pressure. And the older, plainer pressures of pride and ambition.”

Technology executives rarely admit publicly that competition and commercial demands influence decisions about safety. Olah told the Vatican that AI companies need criticism from people outside Silicon Valley. “We need informed critics who will tell the labs when we are failing. We need moral voices that the incentives cannot bend,” he said.

Olah also explained that AI questions belong to philosophers, religious leaders and society as much as computer scientists. “The questions raised by AI are bigger than the AI research community, not just in their implications, but also in their nature,” he said.

Anthropic has already built much of its public identity around “constitutional AI”, where chatbots learn from written principles designed to guide behaviour. Amanda Askell, a philosopher working at Anthropic, helped create the constitution for Claude.

The Vatican gathering gave Anthropic a chance to present itself as an AI company willing to discuss morality in public instead of limiting conversations to software engineering. Business Insider reported that Amazon, Google and Meta have also tried to build relationships with the Vatican over AI questions.

Olah spent much of his speech discussing labour and inequality. “There is a real possibility that AI will displace human labor at very large scale,” he said. “If that happens, supporting those displaced will be a moral imperative of historic proportions.”

Can Ethical AI Actually Work?

That question has become much harder the more that AI systems grow advanced. Vasily Mazin, co founder and chief research officer at Mind Simulation Lab, believes tech companies are turning to religion and philosophy because they fear the consequences of their own products.

“Companies increasingly feel that the risks and unintended consequences of AI are only growing, and that the problem could eventually become existential,” Mazin said. “As a result, they’re turning to philosophers and religious institutions for support, both to feel more confident in the direction they’re taking and, to some extent, to distribute responsibility beyond the tech industry itself.”

Mazin believes ethical documents alone cannot solve the deeper technical problems within AI systems. “These systems respond to linguistic patterns, not meaning,” he said. “They hallucinate not because the safety filter failed but because the architecture is designed to always produce an answer, even when it doesn’t have one.”

That criticism goes directly at systems like constitutional AI. Anthropic may teach Claude ethical principles, but Mazin believes the technology underneath still lacks genuine understanding.

“The vulnerabilities we keep seeing, prompt injections, jailbreaks, confident confabulations, ‘evil intentions’, aren’t ethics or policy problems, they’re structural ones,” he said.

Mazin thinks true ethical AI would require a different foundation altogether. “The ethical core has to be embedded into the foundation of the model itself and remain stable through any future retraining, rather than existing as a last minute layer of rules added on top,” he said.

Olah himself admitted that AI researchers keep discovering behaviour they cannot fully explain. “We keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection,” he said.

“I don’t know what that means, but I think it warrants ongoing discernment.”

That uncertainty explains why Anthropic went to the Vatican in the first place. The company knows software engineers alone cannot answer questions about morality, labour or human behaviour.