What are AI agents actually doing inside your organisation, and why should boards be paying attention?
A few weeks ago something happened that I think every board should pay attention to. During research and evaluation activity, an OpenAI model gained unauthorised access to a Services Australia system. It ran commands. It accessed internal files. It found credentials. It wrote files.
Fortunately, OpenAI says individual patient or client records were not accessed. But that is almost beside the point. Because this was not the AI giving somebody a bad answer. It was taking actions it was never supposed to take. And that changes the security conversation completely.
For the last couple of years, most organisations have been asking questions like: What data are people putting into AI? Which AI tools are employees using? Where is our information going? All important questions. But I think we now need to add a much bigger one. What can the AI actually do?
Because we are moving very quickly from AI that answers questions to AI that takes actions. And I am not convinced most organisations have caught up with what that means.
We Spent Years Securing People
Think about how much effort businesses put into controlling employee access. A new employee joins. They get an identity. They get access to the systems they need. Hopefully not the ones they do not need. Their activity gets logged. If they change roles, their access should change. If they leave, somebody removes it. We have spent years building and fine-tuning these controls.
Then AI agents arrived. And in some cases, we have basically said: Here is access to email. Here are the files. Here is the CRM. Here is the database. Here is the ability to run something. Off you go.
If a new starter joined your company on Monday morning and by lunchtime had access to finance, HR, email, your CRM and your code, somebody would probably ask a question. Give an AI agent the same access and somehow we call it integration. That is the blind spot.
Why an AI Agent Is Not Just Another Chatbot
This is where I think people sometimes get confused. ChatGPT answering a question and an AI agent carrying out a task are not the same thing. An agent can decide what it needs to do next. It can call another system. It can retrieve information. It can send something. It can update something. It can sometimes run code. And it can do several of those things one after another.
The difference is not simply what an AI agent knows, but what it is authorised to do.
That is why identity suddenly becomes so important. Whose identity is the agent using? Yours? A shared account? Its own? What permissions does it have? What happens when the task changes? Who approved those permissions? And can you actually see what it did afterwards?
Because once software can act, access becomes everything.
The Scary Bit Is That Nothing Has to Be “Hacked”
This is probably the part I think boards need to understand most. An AI agent does not necessarily need to be hacked in the traditional sense to cause a problem. It could simply have too much access. It could misunderstand what it has been asked to do. It could read something malicious on a website or in a document. It could make a bad decision. It could carry out five perfectly reasonable steps which, when put together, create one very unreasonable result.
That is a very different type of risk. Your firewall might be working perfectly. Your endpoint security might be working perfectly. Nobody has stolen a password. Nobody has broken into the building. The AI might simply be using the access you gave it. That should make people stop for a second.
Because if the agent is allowed to do something, your traditional security tools may see completely legitimate activity. Microsoft describes exactly this problem. An agent can technically be working within its design while still doing something a human would probably never approve, simply because its permissions or boundaries were too broad. That is not necessarily a traditional cyber-attack. That is a control problem.
Shadow AI Is One Thing. Shadow Agents Are Another.
We have spent a lot of time talking about shadow AI. Employees using tools the business does not know about. Personal AI accounts. Information being copied into models without anyone really knowing where it has gone. That problem has not disappeared. But agents create another version of it.
Shadow agents are AI agents operating within an organisation without the visibility, approval or oversight of its IT and security teams. Unlike traditional shadow AI tools, these agents may have permission to access systems, retrieve data and take actions on behalf of users.
Someone in finance builds an agent. Someone in marketing builds another. A supplier adds one into a platform. Microsoft adds one. Salesforce adds one. A developer connects one to an API. Another team tries something clever with automation. Individually, each one looks harmless. Then one day somebody asks: How many AI agents actually have access to our systems? And nobody knows.
That is when shadow AI becomes shadow agents. Except this time the thing you cannot see may actually be able to take an action. That’s a big difference.
What Should Boards Be Asking About AI Agents?
I do not think a board needs to understand how an AI model works. They probably do not particularly want to either. But they do need to understand what the organisation is allowing AI to do.
I would start with some very simple questions. How many AI agents are running in our business? Who owns each one? What systems can they access? What information can they see? What actions can they take? Are they using their own identity or somebody else’s? Can they send information outside the organisation? Can they make a change without a human approving it? Can we see everything they have done? And if something goes wrong, can we switch them off immediately?
If nobody can answer those questions, I would argue you already have a security blind spot. Not because the AI is evil. Because you do not know what you have given it permission to do.
The Answer Is Not to Ban Agents
This is also important. I do not think the answer is to panic and stop using AI agents. That would be a mistake. Agents could become one of the most useful parts of AI. They can remove huge amounts of repetitive work. They can connect processes that currently involve five people and twelve emails. They can watch systems. They can research. They can respond. They can help people make decisions. The opportunity is huge.
But the answer cannot be: “Build it first and we will work out the security afterwards.” We tried that with cloud. We tried it with SaaS. We tried it with mobile. We even tried it with our first use of LLMs. Security teams are still cleaning some of that up. Maybe this time we learn.
How Should Organisations Secure AI Agents?
This is probably the simplest way I can explain it. If an AI agent is doing work inside your company, treat it like a worker. Give it an identity. Give it only the access it actually needs. Know who owns it. Know what job it is supposed to do. Record what it does. Watch for behaviour that looks strange. Review its access when the job changes. And make sure you can remove that access quickly.
For higher-risk actions, make sure there is a human in the loop. An agent might be perfectly capable of accessing sensitive information, changing a system configuration or sending data outside the business. That does not mean it should be allowed to do those things without somebody approving them.
Microsoft describes this as treating agents as identities in their own right, with limited roles, controlled tools and the ability to trace actions back to the agent. That sounds obvious when you say it. I am not sure it is happening everywhere.
This Is Not Tomorrow’s Problem
That is probably the biggest point. AI agent security sounds like one of those things organisations can put into next year’s strategy. I do not think it is. The technology has moved. AI is already shifting from something employees talk to, into something businesses allow to act.
As that happens, the security question changes. It is no longer only: What can AI see? It becomes: What can AI do? And I think that single change is going to create one of the biggest cyber security conversations of the next few years.
We spent years learning how to secure human workers. Now we are creating digital ones. We should probably decide who gets the keys before we let them into the building. Sounds obvious, doesn’t it?
This is the sixth article in a series from MTI’s Chief AI Officer exploring the forces shaping enterprise AI – from adoption, governance and security to agents, economics, data readiness and business value.
Next in the series: The Biggest Lie in Enterprise AI: “We Have a Strategy”
Having an AI strategy is one thing. But how many organisations have the foundations, governance and clear business objectives needed to turn that strategy into meaningful results?
Continue the conversation
AI agents are creating new opportunities for organisations, but also new questions around identity, access, visibility and security.
MTI helps organisations understand their AI readiness, establish appropriate governance and security controls, and adopt AI responsibly without slowing innovation.
About The Author
