AI Didn’t Break Your Security. It Found What Was Already Broken.
The UK government’s evaluation wasn’t a warning about the future. It was a verdict on the present.
On Tuesday morning, the UK Secretary of State for Science, Innovation and Technology wrote an open letter to every business leader in the country. Not a press release, not a policy consultation, but a letter. That kind of thing does not happen unless something crossed a threshold.
The trigger was a published evaluation by the UK’s AI Security Institute, a government body, of Anthropic’s latest AI model. Their finding- in controlled testing, the model autonomously completed a 32-step corporate network attack from initial reconnaissance to full takeover. Tasks that a skilled human professional would need around 20 hours to complete. Done autonomously without a human in the loop.
The headlines ran hard with it. “Unprecedented attack capability.” “An alarm bell.” “The window is closing.”
Here is what the evaluation actually showed, stripped of the hype. The test environment had no active defenders, no endpoint detection, no real-time incident response. The model completed the full attack chain in three of its ten attempts. The AISI was explicit, they cannot conclude the model would perform as well against a hardened, well-monitored network. These are the honest numbers, and the honest numbers are still significant.
More significant than the single finding is what sits underneath it. Two years ago, the best available AI models could barely complete beginner-level cyber tasks. Now one has completed 32 sequential steps of a professional attack simulation. AISI reports that frontier AI capabilities in cyber offence are doubling every four months, twice the pace recorded just months ago. The finding is not the scary part, the trajectory is.
Recently I was at a cyber security conference. The room was full of security leaders, experienced, capable, serious professionals. The conversation that emerged in networking breaks and informal moments was not about AI capability. It was about something quieter and more uncomfortable. Most of them felt structurally unsupported, not underqualified, not unaware of the threat, but unsupported. Responsible for outcomes they could not fully control, in organisations that had not genuinely reckoned with what that means.
That conversation did not start this week. The AI evaluation did not create it. But the two things belong together, and most of the coverage since Tuesday has not connected them.
For years, the security community has watched opportunistic attacks accelerate. Bad guys begin scanning the internet for vulnerable systems within minutes of a new vulnerability being publicly announced. Attack times have compressed, and ransomware deployment that once took weeks now takes hours. That acceleration is not new, and anyone who has been paying attention is not surprised by it.
What has remained expensive, until now, is something different. Targeted, multi-stage intrusions, the kind that begin with reconnaissance, move through a network, escalate privileges, and end with full system compromise, have required two things that could not easily be automated or outsourced- judgement and adaptability. The ability to make contingent decisions across dozens of sequential steps, each one shaped by what the previous step revealed. That is what skilled attackers brought to the table. That is what made them scarce, and scarcity made them more expensive.
The AISI evaluation is significant precisely because of what it tested. Not whether an AI model could scan for known vulnerabilities. Whether it could complete 32 sequential steps of a professional network intrusion, from initial reconnaissance to full takeover, making adaptive decisions throughout. That is the category of attack that previously required a capable human. Now AI successfully exploited a system in three of ten attempts end to end, in an undefended environment.
AI is not making opportunistic attacks faster, they already are. It is lowering the skill floor for the attacks that were never fast, the targeted, adaptive, multi-step campaigns that organisations have quietly relied on being difficult to execute. That reliance was never a strategy. It was a structural feature of how scarce genuine attacker expertise was. That scarcity is now in question.
There is a philosophy in security that has existed for years now, passed through enough strategy documents and vendor presentations to have been bleached of almost all meaning. It goes by the name assume breach.
In its genuine form it is a serious and demanding idea. It means accepting, structurally, not performatively, that the attacker will get in. That the question is not whether a breach happens but how quickly you detect it, how contained the damage is, and how effectively you recover. It means orienting investment toward detection, resilience and recovery, not just prevention. It means building governance structures that treat a breach as a systemic risk event rather than an individual failure.
Very few organisations have actually done this.
What most organisations have done is put assume breach in the deck and leave everything else unchanged. Security leaders still carry personal accountability for preventing breaches. The board still treats a breach as evidence of individual failure. The investment profile still skews heavily toward keeping attackers out rather than assuming they are already in. Gartner’s 2024 Board Survey found that while 93% of directors recognise cyber risk as a threat to stakeholder value, two thirds rate their own oversight practices as inadequate to manage it. They know it matters. That is not the same as having genuinely reckoned with it.
The conference room I described earlier is what that gap looks like from the inside. Those security professionals are not failing at their jobs. They are operating inside a structural contradiction that most organisations have never acknowledged. Assume breach as a philosophy and holding a single individual accountable when the attacker gets in as a practice cannot both be true simultaneously. One says a breach is systemic and inevitable. The other says it is individual and preventable. Most organisations hold both positions without noticing the conflict.
That contradiction was always there. What AI has done is remove the margin for error that allowed organisations to sustain it without immediate consequence.
The government’s letter this week is not wrong to call this a wake-up call. But wake-up calls only work if the response is structural rather than reactive. Buying a new tool, commissioning a review, issuing a memo about cyber hygiene, none of that addresses the underlying problem, which is not technical. It is a governance problem dressed in technical clothing.
The questions worth asking are not questions for your security team. They are questions for your board.
Has your organisation formally accepted, in writing, in your risk register, that a breach is a question of when rather than if? Not in a presentation. In the governance framework that shapes how you invest and how you respond.
If a significant breach occurred tomorrow, would a single individual be held responsible? If the honest answer is yes, your organisation does not have an assume breach posture, it only has the words.
Does your investment in security focus mainly on keeping attackers out, or toward detecting and containing them once they are in, and recovering afterwards? Prevention-first is not wrong, absolutely always try to prevent. But prevention-only, in a threat environment where the cost of a capable attack is falling rapidly is not sustainable.
The AI finding matters. The acceleration is real. But the organisations most exposed this week are not the ones who failed to predict it. They are the ones who had already been told, by their own security leadership, in conference rooms and board papers and risk registers, and built a culture that made it impossible to hear.
The breach was always coming. AI just made it cheaper to deliver.
I write about AI, cybersecurity, and technology every Friday. Subscribe to get it in your inbox.
Sources & Further Reading
UK AI Security Institute (2026), Our evaluation of Claude Mythos Preview’s cyber capabilities
aisi.gov.uk/blog/our-evaluation-of-claude-mythos-previews-cyber-capabilities
UK Government (2026), AI cyber threats: open letter to business leaders (15 April 2026)
gov.uk/government/publications/ai-cyber-threats-open-letter-to-business-leaders
Gartner (2024), Board of Directors Survey: Cybersecurity as Business Risk
gartner.com/en/newsroom/press-releases/2024-11-13-gartner-says-80-percent-of-non-executive-directors-believe-current-board-practices-and-structures-are-inadequate-to-oversee-ai
Help Net Security (2024), CISOs in 2025: Balancing security, compliance, and accountability
helpnetsecurity.com/2024/11/13/daniel-schwalbe-domaintools-cisos-2025/