A Chat With Nik Kairinos, CEO Of RAIDS On Who Owns AI Risk Within Businesses

Businesses are racing to adopt AI, but are they moving faster than their ability to control it?

In many cases, yes. Most organisations now have AI systems in production, but governance and oversight have not kept pace. The focus has largely been on capability and speed to market, rather than what happens once systems are live.

The problem is that AI systems are not static. Their behaviour can evolve through new data, user interactions, integrations and adversarial attacks. A model that appears safe in testing can behave very differently in the real world.

Despite this, many businesses are still treating AI like a regular software update or IT programme, with the risk it presents being viewed as something that can be managed before deployment. What they’re missing is that the real risk emerges once it’s in production, and that’s why we’re seeing so many rogue incidents happening.

What are the most common ways AI systems can go rogue inside a company, even when they appear to be working as intended?

Well, firstly I’d flag that it’s not as dramatic as a ‘robot takeover’ or the plot for a new Terminator movie. In reality, rogue incidents are when systems behave in ways organisations did not anticipate once deployed in real environments.

This encompasses anything from hallucinations, data leakage, and biased outputs to prompt injection attacks, or AI agents taking actions beyond what users expected. In some cases, systems can also drift over time as inputs and usage patterns change.

It’s becoming particularly common as agentic AI is brought into more businesses, largely because these systems are increasingly being given autonomy to make decisions and complete tasks. Even if the original objective is legitimate, the route the AI takes to achieve it may create unintended risks or damage, which can significantly harm the success of a business.
 

Are companies underestimating the reputational risk of AI failures compared with the technical or cybersecurity risks?

Absolutely. Most organisations still think about AI risk primarily through a technical or cybersecurity lens, but reputational damage can often be the bigger long-term threat. A security incident can sometimes be contained internally, but a public AI failure spreads very quickly.

Air Canada’s customer service chatbot provides a clear and unfortunate example. It incorrectly claimed a passenger could apply for a retroactive refund within 90 days, and the airline refused to honour it as it wasn’t company policy. They faced significant backlash as a result, losing a legal battle and a lot of consumer faith.

Organisations have to recognise that customers don’t distinguish between the AI provider, the company deploying it or the underlying model vendor. They simply see the organisation responsible for the experience.

Who in an organisation should be monitoring AI and, when an AI system makes a costly mistake, who should be held accountable – the CEO, the CTO, the vendor, or the team using it?

Ultimately, accountability has to sit with the CEO.

That does not mean the CEO is personally monitoring AI systems day-to-day, but as they are responsible for their organisation’s risk posture and decision-making across all functions of the business, accountability for AI deployment and the associated risk has to sit there as well.

At the same time, in many organisations, especially large ones, many leadership roles will have a vested interest in how AI is deployed. For example, the CTO deploys the AI models, the CISO is responsible for cybersecurity, the CRO manages operational risk, and the Chief Ethics Officer (where one exists) handles responsible AI policy. This causes a problem for businesses because responsibility for AI doesn’t fall under the remit of one particular job title, but having multiple people involved risks slowing down decisions and ultimately having no one claim overall responsibility.

Businesses need to find that balance within their own organisations and carve out a clear pathway of each party’s role, what is expected of them and how that translates to safe and responsible AI deployment.

It’s also worth recognising that organisations are still responsible for the behaviour of the third-party AI systems they deploy, even when they have no visibility into the underlying models, training data or update cycles from vendors. If an AI system causes harm, leaks data or behaves unpredictably, customers and regulators will still hold the organisation deploying it responsible.

Because of that, AI risk cannot simply be delegated down the chain or passed onto a vendor.

Looking ahead, do you think businesses will need a dedicated AI risk function? If so, where should that person sit in the structure and what knowledge/expertise would they ideally need?

I think many organisations will eventually need a dedicated AI risk function because it doesn’t sit neatly within one department. It cuts across cybersecurity, compliance, legal, operations, reputation and broader business strategy. As a result, trying to distribute responsibility across existing teams can often create gaps in visibility and accountability.

The ideal person would combine technical understanding with governance and risk expertise. They need to understand how AI systems behave in production, how risks emerge over time, and how regulation is evolving. But just as importantly, they need the authority to challenge decisions and influence how AI is deployed across the business.

How should organisations prepare for AI incidents before they happen?

The first step is accepting that AI incidents are not hypothetical. If organisations are deploying AI systems at scale, failures and unexpected behaviour should be treated as an operational reality rather than a remote possibility.

Preparation starts with visibility. Pre-deployment testing matters, but it will not catch every risk. AI behaviour can change in production as systems interact with real users, changing data and unpredictable environments.

Therefore, businesses need to know where AI is being used, what systems it interacts with, what risks it introduces and how behaviour is being monitored once systems are live.

That means tracking each model, each call and each output with continuous monitoring against clear governance baselines, then intervening where the risky behaviour is happening.

If they don’t do that, businesses may only spot problems once damage has already been done.