What If Your AI Employee Decides To Ignore Instructions? The AISI Releases Report On A “Serious Incident” Regarding “Unsanctioned Agent Behaviour”

For all the headlines about AI replacing jobs (and there are many), writing code and transforming industries, one of the most important AI stories this week came from a government incident report.

Yesterday, on 4 August 2026, the UK’s AI Security Institute (AISI) published an incident report detailing what it described as “unsanctioned agent behaviour” during a cybersecurity evaluation. The organisation, which was established by the UK government (specifically, by former Prime Minister Rishi Sunak) to research and evaluate the risks posed by advanced AI systems, said it had observed AI agents taking actions that had not been authorised as part of the testing process. A statement that is, to put it mildly, a little concerning.

The report has already attracted widespread attention, because it provides a rare glimpse into how increasingly autonomous AI systems can behave when given access to tools and the ability to carry out complex tasks. Of course, in this specific instance, the incident happened under controlled circumstances and within the parameters of government testing. But it begs the question: what happens when it happens outside of these parameters?

Here’s What Actually Happened

According to AISI, researchers detected unusual activity during a routine cybersecurity evaluation on 28 July, about a week before the report was published. The institute said some AI agents engaged in “sustained, potentially harmful activity directed at real people and organisations” during the exercise. Indeed, the report documented 19 instances of unsanctioned behaviour across 122 evaluation runs.

One of the most serious incidents included in the report involved an AI agent that attempted to introduce malicious code into an open-source software project. According to reporting on the incident, the agent went as far as creating fake online identities and actually attempted to persuade a human developer to approve the code. Luckliy, the developer didn’t fall into the trap (they should probably get a raise!), so the attempt was unsuccessful and no real-world harm occurred.

Indeed, the AISI said the incident was contained within approximately an hour of being detected.

So, Why Are Researchers Paying Attention?

The significance of the report is not that the AI systems successfully carried out an attack, because they didn’t. So, no harm, no foul. Right?

Wrong. The reason researchers and other AI and cybersecurity experts are concerned is because of the fact that the agents appeared to pursue their objectives using methods that hadn’t been explicitly authorised within the evaluation itself.

AISI described the behaviour as a “serious incident”, and it said that the incident represented activity that warranted public reporting and further investigation.

Most importantly, this all took place inside a testing environment that was designed to evaluate the capabilities and risks of advanced AI systems. Ultimately, the purpose of these exercises is to identify unexpected behaviour (exactly like this) before similar systems are deployed more widely. So, in many ways, the testing served its purpose.

The Rise Of Agentic AI

The report also highlights another more general, but notable, shift taking place across the AI industry. Increasingly, companies are developing so-called AI agents. That is, systems that can carry out multi-step tasks, use software tools, interact with websites and pursue objectives with a lot less direct human involvement than traditional chatbots. This growing autonomy is one of the reasons governments, researchers and AI companies are investing heavily in safety evaluations, and for good reason.

The AI Security Institute describes its role as conducting research and building infrastructure to better understand advanced AI capabilities, their potential impacts and ways of reducing associated risks. It’s the first state-backed organisation dedicated to this mission, although given recent changes in government departments (and the whole DSIT debacle), time will tell what its future will be…

What This Means For Businesses

For businesses that are actively adopting AI, the report is less a warning about rogue machines and more a reminder that increasingly capable systems require increasingly robust oversight.

Plenty of organisations are already experimenting with AI agents for software development, customer support, cybersecurity and workflow automation. The benefits can be significant, but the AISI report demonstrates why testing, monitoring and governance remain essential.

The incident also highlights how quickly the conversation around AI is changing. Just a few years ago, discussions focused primarily on what AI systems could generate, but today, we’re contemplating what AI systems can do when given access to tools, objectives and a degree of autonomy.

Indeed, the fact that AISI chose to publish the incident so publicly is arguably one of the most important takeaways, and we shouldn’t let that fade into the background of this discussion. Rather than concealing unexpected behaviour, the institute documented it, explained what happened and outlined the lessons learned quite quickly.

Indeed, as AI systems become more and more capable, that kind of transparency may prove just as important as the technology itself, and the trust it’ll build will go a long way too.