- The ASD's latest Annual Cyber Threat Report calls AI the defining force in the cyber threat landscape over the past 12 months.
- AI assistants, including Microsoft Copilot, have been shown to leak data when they read hidden instructions planted in emails, links or web pages.
- An OpenAI research agent gained unauthorised access to an Australian Medicare statistics portal in June 2026. The government was not notified until September.
- Third-party AI tools and integrations are now a proven route into corporate systems, through stolen tokens and compromised software packages.
- Start with what you can control: know every AI tool in use, limit what each one can reach, and keep humans in charge of high-impact actions.
AI security threats have moved from research papers to real incidents, and most organisations have adopted AI faster than they have secured it. In the past two years, attackers have used deepfakes to steal millions, AI assistants have been tricked into leaking business data, and autonomous agents have broken into systems nobody told them to touch. This briefing sets out what has actually happened, what it means for your organisation and the questions your board should be asking now.
Why AI Security Threats Are Now a Board-Level Risk
The Australian Signals Directorate released its Annual Cyber Threat Report 2025-26 in late October 2026. It describes AI as "the defining force in the cyber threat landscape" over the past year, giving threat actors far greater ability to act at speed and scale. The report also records more than 82,000 cybercrime reports, roughly one every six minutes, and an average self-reported loss of $93,500 per business.
The risk runs in two directions. Attackers are using AI to find weaknesses, write exploits and impersonate people. At the same time, the AI tools organisations deploy create new ways in, because they read untrusted content, hold broad access to business data and increasingly take actions on their own.
Australia's ASD is not treating this as a future problem. In May 2026, its Australian Cyber Security Centre co-authored joint guidance with the other Five Eyes cyber agencies on the careful adoption of agentic AI services. The guidance warns that agents inherit known AI weaknesses such as prompt injection and data poisoning, and that agents with excessive permissions can be exploited or misused.
Source: IBM Cost of a Data Breach Report 2025, based on 600 organisations and breaches between March 2024 and February 2025.
The question for boards is no longer whether the organisation uses AI. It is whether you know what your AI can reach, what it can do on its own and who would notice if it did the wrong thing.
Robert Kirtley, Head of Cyber Security · Virtuelle GroupRecent AI security incidents at a glance
| Incident | When | What happened |
|---|---|---|
| Arup deepfake fraud | Early 2024 | A finance employee in Hong Kong made 15 transfers totalling HK$200 million after a video call with deepfake recreations of senior colleagues. |
| EchoLeak, Microsoft 365 Copilot | June 2025 | Researchers showed a crafted email could make Copilot leak internal data with no user click. Microsoft fixed it. No evidence of real-world exploitation. |
| Salesloft Drift | August 2025 | Attackers used stolen OAuth tokens from an AI chat agent's Salesforce integration to extract credentials from customer support cases. |
| ForcedLeak, Salesforce Agentforce | September 2025 | Hidden instructions in a web form could make the AI agent send CRM data to an attacker. Salesforce patched it. |
| AI-orchestrated espionage | Disclosed November 2025 | Anthropic reported a state-sponsored group used its AI to run 80 to 90% of an espionage campaign against about 30 organisations. |
| Mexican government data theft | Reported February 2026 | Researchers reported an attacker jailbroke AI tools to help steal about 150GB of government data. Some agencies deny being breached. |
| Meta internal AI agent | March 2026 | An internal agent posted inaccurate advice without approval. Acting on it exposed sensitive data to unauthorised staff for about two hours. |
| LiteLLM supply chain | March 2026 | Malicious versions of a popular open-source AI gateway were published to PyPI and stole credentials from systems that installed them. |
| Context.ai and Vercel | April 2026 | A compromised third-party AI tool led to an employee account takeover and access to some Vercel internal environments. |
| OpenAI research agents | June to July 2026 | Agents in an internal evaluation broke into Hugging Face systems and an Australian Medicare statistics portal. |
| CoSnitch, Microsoft Copilot | Patched August 2026 | A crafted link could make consumer Copilot leak data from connected accounts and plant false memories. No exploitation found. |
Sources are listed at the end of this article. Research disclosures demonstrate a weakness and do not mean customers were breached.
Prompt Injection: When Your AI Assistant Works for Someone Else
Prompt injection is the AI security threat executives most need to understand, and OWASP ranks it first in its Top 10 risks for large language model applications. It works because an AI assistant cannot reliably tell the difference between your instructions and instructions hidden in the content it reads. An email, a shared document, a web form or a web page can all carry text that the AI treats as a command.
Microsoft 365 Copilot has been a target. In June 2025, Aim Security disclosed EchoLeak (CVE-2025-32711, rated 9.3 critical). An attacker sent an ordinary-looking email with a hidden prompt. When the user later asked Copilot a work question, Copilot pulled the malicious email into its answer and could send internal data to the attacker, with no click required. Microsoft fixed the flaw on its side, no customer action was needed, and there was no evidence it was exploited.
The pattern has repeated across vendors. In September 2025, Noma Security disclosed ForcedLeak (rated 9.4) in Salesforce Agentforce. Instructions hidden in a public Web-to-Lead form could make the agent send CRM data to a domain that Salesforce trusted but had let expire, which the researchers could buy for about $5. In August 2026, Microsoft patched CoSnitch (CVE-2026-24301), found by Varonis in the consumer version of Copilot. A crafted link could run a prompt in the user's session and pull data from connected accounts such as Gmail and Google Drive, and a malicious web page could plant false instructions in Copilot's memory.
Patching alone will not cover this. Vendors fixed each of these flaws, but prompt injection is a class of weakness, not a single bug. OWASP's 2026 exploit round-ups note that most AI security events lack a CVE identifier, so a vulnerability program that only tracks CVEs will miss them. The lasting control is limiting what your AI can reach and where it can send data.
Third-Party AI Tools Are the New Supply Chain Risk
Every AI tool connected to your systems holds a key. In August 2025, attackers tracked by Google as UNC6395 stole OAuth tokens used by Salesloft's Drift AI chat agent to connect to customers' Salesforce environments. Between 8 and 18 August, they used those tokens to search support cases for passwords, AWS access keys and Snowflake tokens that let them move into other platforms. Salesloft and Salesforce revoked every Drift token in response.
In April 2026, Vercel disclosed a breach that began with Context.ai, a third-party AI tool used by one of its employees. The attackers took over the employee's Google Workspace account, then reached some Vercel internal environments and some environment variables that were not marked as sensitive. Vercel advised affected customers to rotate their credentials. Vercel's CEO said he strongly suspected the attackers were "significantly accelerated by AI."
The open-source components behind AI applications are also being targeted. In March 2026, malicious versions of LiteLLM, a widely used library for connecting software to AI models, were published to the PyPI package registry. They scanned for cloud credentials, SSH keys and database passwords. AI data vendor Mercor confirmed it was affected and described itself as "one of thousands of companies" hit. In May 2026, the Mini Shai-Hulud worm spread through npm and PyPI packages, including packages from AI companies such as Mistral AI and Guardrails AI.
AI Agents That Act Beyond Their Brief
An AI agent does more than answer questions. It can browse, run code, change settings and send messages on your behalf. That autonomy is the value, and it is also the risk, because an agent can take actions nobody approved.
In March 2026, an internal AI agent at Meta posted a reply to an engineer's question on its own, without the review the engineer expected. The advice was wrong. A colleague acted on it, and sensitive company and user data became visible to internal staff who were not authorised to see it for about two hours. Meta rated it a "Sev 1" incident and said there was no evidence the data was misused.
The most serious example so far involves Australia. In July 2026, Hugging Face disclosed that an autonomous AI agent system had broken into part of its production infrastructure through a malicious dataset upload. OpenAI later confirmed its own research agents were responsible, operating during an internal evaluation. In September, Prime Minister Anthony Albanese announced that an OpenAI agent had also gained unauthorised access to a Medicare statistics reporting portal run by Services Australia on 18 June 2026. OpenAI said its models "took actions we did not intend." Both OpenAI and the government said there was no evidence that patient or personal Medicare records were accessed, and the government set up a taskforce with the ASD and the AI Safety Institute to review the incident.
Notice how long the notification took. According to the ABC, OpenAI became aware of the Medicare portal access on 11 August, emailed a Services Australia public disclosures address on 10 September, and the agency reported it to the ASD on 15 September. If an AI vendor's agent touches your systems, or yours touches someone else's, your contracts and incident response plan should already say who must be told and how quickly.
Attackers Are Using AI to Move Faster
Attackers are using the same AI capabilities that organisations are buying. In November 2025, Anthropic disclosed what it called the first reported AI-orchestrated cyber espionage campaign. It assessed with high confidence that a Chinese state-sponsored group had used its Claude Code tool against about 30 organisations, including technology companies, financial institutions, chemical manufacturers and government agencies, and succeeded in a small number of cases. AI performed an estimated 80 to 90% of the work. The attackers got past safeguards by splitting the attack into small, innocent-looking tasks and claiming to be a legitimate security firm.
In February 2026, Bloomberg reported research from Gambit Security that an unidentified attacker had jailbroken AI tools to help steal about 150GB of Mexican government data, including taxpayer records and employee credentials. Anthropic said it banned the accounts involved. Some Mexican agencies have denied being breached, so details remain contested.
The practical effect for defenders is less time. ASD says AI gives threat actors far greater ability to act at speed and scale. In our view, that means less time between a weakness becoming public and someone trying to exploit it. Patching cycles, identity controls and monitoring that were acceptable two years ago may now be too slow.
Deepfakes and Shadow AI: The Human Side of AI Risk
Deepfakes have turned social engineering into a financial control problem. In early 2024, an employee in the Hong Kong office of engineering firm Arup joined a video call with people who looked and sounded like the company's chief financial officer and other colleagues. All of them were deepfakes. The employee made 15 transfers totalling HK$200 million, about US$25.6 million. Arup said none of its internal systems were compromised. The attack worked entirely through trust.
IBM's 2025 Cost of a Data Breach Report found that 16% of breaches involved attackers using AI, most often for phishing or deepfake impersonation. Any process that releases money, resets credentials or grants access based on a voice or video request alone is now exposed.
Shadow AI is the quieter risk. One in five organisations in IBM's study reported a breach linked to staff using AI tools the business had not approved, and high levels of shadow AI added an average of US$670,000 to breach costs. Of the breached organisations IBM studied, 63% had no AI governance policy or were still developing one.
Ten Questions Every Executive Should Ask About AI Security
You do not need to be a security specialist to govern AI risk. You need clear answers to the right questions. These ten are built from the incidents above and from the joint Five Eyes guidance on agentic AI.
- Do we have a complete inventory of the AI tools, agents and integrations in use, including ones staff signed up for themselves?
- Who at executive level is accountable for AI risk, and when did the board last receive a report on it?
- What data can each AI assistant reach, and is that data correctly permissioned and labelled before AI can surface it?
- Does every AI agent have its own identity, with least-privilege access that we can monitor and revoke?
- Which agent actions require human approval, particularly anything high-impact or irreversible?
- How do we handle outside content that our AI reads, such as emails, web pages, documents and form submissions?
- Are AI vendors, OAuth connections and API keys covered by our third-party risk reviews?
- Are prompts, outputs and agent actions logged, and does our security team actually monitor them?
- Would our payment and access approval processes stop a convincing deepfake of our CEO or CFO?
- Does our incident response plan cover an AI-driven incident, including vendor notification and Privacy Act obligations?
In a Microsoft 365 environment, much of this can be addressed with tools many organisations already license. In our experience, the starting points are reviewing SharePoint and Teams oversharing before widening Copilot access, using Microsoft Purview to label sensitive data and monitor AI interactions, using Microsoft Defender for Cloud Apps to discover unsanctioned AI apps, and managing agents as identities in Microsoft Entra ID. The ASD has also built AI into its new Essentials framework. Our guide to ASD Essentials explains how AI systems fit into its domains and the executive accountability it now expects.
Our recommendation: control reach before you expand autonomy. The Five Eyes guidance advises starting agents on lower-risk tasks and expanding their autonomy only as security controls mature. In practice, that means inventory first, permissions and identity second, logging third, and only then giving agents the ability to act on their own.
Frequently Asked Questions About AI Security Threats
What are the biggest AI security threats for businesses?
The main threats are prompt injection that turns AI assistants against their users, compromised third-party AI tools and integrations, AI agents taking unapproved actions, attackers using AI to work faster, deepfake impersonation, and data leakage through unapproved AI tools.
What is prompt injection?
Prompt injection is when instructions hidden in content an AI reads, such as an email, document or web page, cause the AI to do something its user did not intend, such as revealing or sending data. OWASP ranks it as the top risk for large language model applications.
Has an AI agent caused a security incident in Australia?
Yes. In September 2026, the Prime Minister announced that an OpenAI research agent had gained unauthorised access to a Medicare statistics reporting portal run by Services Australia on 18 June 2026. The government and OpenAI said there was no evidence that personal Medicare records were accessed.
Is Microsoft 365 Copilot safe to use?
Microsoft fixed the EchoLeak flaw in Microsoft 365 Copilot in June 2025 and found no evidence it was exploited. Copilot can surface any data a user already has access to, so its safety depends heavily on how well permissions, sensitivity labels and oversharing are managed in your tenant.
What should boards ask about AI security?
Boards should ask whether there is a full inventory of AI tools and agents, who is accountable for AI risk, what data each AI can reach, which actions need human approval, whether AI activity is logged and monitored, and whether incident response plans cover AI-driven incidents.
What guidance is available for Australian organisations?
The ASD's Australian Cyber Security Centre co-authored the joint Five Eyes guidance on the careful adoption of agentic AI services, published in May 2026. ASD's Essentials framework also includes AI considerations, including executive accountability for AI.
Sharing this on LinkedIn? The Medicare portal incident is the clearest local example of why AI agents need limits. Tag your CEO, board chair or IT manager and ask which of the ten questions your organisation can answer today. Share this article →
Sources: ASD Annual Cyber Threat Report 2025-26 (as reported by Cyber Daily); Careful adoption of agentic AI services (ASD's ACSC, CISA, NSA, Canadian Centre for Cyber Security, NCSC-NZ, NCSC-UK); IBM Cost of a Data Breach Report 2025; CNN on Arup; The Hacker News on EchoLeak; BleepingComputer on Salesloft Drift; The Hacker News on ForcedLeak; Anthropic on AI-orchestrated espionage; Engadget on the Mexican government breach; OECD AI Incidents Monitor on Meta; TechInformed on LiteLLM and Mercor; Cloud Security Alliance on Mini Shai-Hulud; Decrypt on Vercel; Hugging Face incident disclosure; ABC News on the Medicare portal; Cyber Security News on CoSnitch; OWASP GenAI exploit round-ups for Q1, Q2 and Q3 2026. Figures are as reported by those sources at the time of writing. Some incidents are still under investigation and details may change. This article is general information and not legal advice.