The Government Just Published Its Homework on AI. Defenders Should Read It.

By ICCSO News Desk

2026-07-23T14:55:00.000Z

What the UK AI Security Institute's first Frontier AI Trends Report tells us — and what it deliberately does not.

Most of what the security industry knows about frontier AI capability comes from two unreliable places: the companies selling the models, and the companies selling the defences. Both have an interest in the number being large. In December 2025, the UK's AI Security Institute (AISI) — a research directorate inside the Department for Science, Innovation and Technology — published something different. Its inaugural Frontier AI Trends Report sets out aggregated results from two years of hands-on evaluation of more than thirty frontier AI systems, covering models released between 2022 and October 2025. AISI has had pre-release API access to model checkpoints. It has run red teams against them. It has, in several cases, tested versions of systems the public never saw. For the small security functions ICCSO serves — the one-person IT team in a multi-academy trust, the charity with no SOC, the SME with a part-time MSP — this is the closest thing available to an independent, non-commercial reading of where the threat curve is actually heading. It is worth an hour of anyone's time. What follows is our reading of the five findings with the most direct bearing on defensive practice, followed by an equally important section on what the report does not establish. 1. Offensive cyber capability is on a doubling curve The headline number for defenders is not a capability score. It is a rate of change. AISI grades its cyber evaluation tasks by the human experience they would require: technical non-expert, apprentice (1–3 years), practitioner (3–10 years), and expert (10+ years). In late 2023, the best models completed apprentice-level tasks less than nine per cent of the time. By the time of the report, the best models were completing them around half the time. In 2025, AISI tested the first model able to complete any expert-level task at all. The same pattern shows up wherever AISI measures autonomous task completion. Figure 1 — Frontier model success rates on three AISI task sets, late 2023 against 2025. Values as stated in the Frontier AI Trends Report (AISI, December 2025); chart redrawn by ICCSO. Source More significant is the trend line underneath. The length of cyber task a model can complete unaided — measured as how long it would take a human expert — has moved from under ten minutes in early 2023 to over an hour by mid-2025, a doubling time of roughly eight months. AISI presents this as an upper-bound estimate, and notes that external work by the non-profit METR finds the same lengthening pattern across other domains. Figure 2 — The length of cyber task a model can complete with a 50% success rate, measured in human-expert time. End points and doubling rate as described by AISI; the intermediate curve is illustrative rather than measured. Redrawn by ICCSO. Source Why it matters here. The cost of competent attack labour is falling. The work that previously required a hire — reconnaissance, vulnerability triage, exploit adaptation, credential harvesting at scale — is progressively becoming work that can be automated by someone with a subscription and a scaffold. This does not change what you should do; it changes how fast you should do it. A two-year remediation plan sized against a 2024 adversary is being quietly repriced. 2. Safeguards are a speed bump, not a control AISI stress-tests the misuse safeguards that AI companies build into their systems, working in partnership with the developers. Its finding is unambiguous: it has found a universal jailbreak — a single technique that reliably defeats safeguards across a range of malicious requests — in every system it has tested to date . There is real progress alongside that. Comparing two leading systems released six months apart, AISI found the first could be broken in roughly ten minutes using a publicly known vulnerability, while the second required over seven hours of expert red-team effort and a novel attack — a roughly fortyfold increase. That is a meaningful gain, concentrated mainly in the biological misuse domain where developers have invested most heavily. Figure 3 — Expert red-team effort required to find a universal jailbreak in two leading systems released six months apart, in the biological misuse category. Values as stated by AISI, December 2025; chart redrawn by ICCSO. Source But two qualifications should stop anyone treating vendor safeguards as a security control. First, protection is uneven: the same model that resists biological-misuse requests may be far easier to break for other categories. Second, and more counter-intuitively, more capable models are not reliably better defended . AISI found minimal correlation between general capability and safeguard robustness (R² = 0.097). Robustness tracks how much effort a company spent building defences, not how clever the model is. Why it matters here. If your AI governance policy rests on "the vendor's guardrails will stop misuse," it rests on a control that a determined adversary has, empirically, always been able to defeat. This is the same conclusion we reached about prompt injection in Securing the Agentic Enterprise : assume the model-level defence fails sometimes, and design for blast-radius reduction instead. 3. Open-weight models remove the brake entirely AISI's assessment of open systems is blunt. Trained-in refusal behaviour can be cheaply removed. Monitoring components can simply be switched off. And because the weights are no longer hosted by the defender, discovered vulnerabilities cannot be patched — there is no server-side fix to ship. Meanwhile the capability gap is closing. Drawing on external benchmarks from Artificial Analysis and METR, AISI puts the current lag between frontier closed models and the best open models at somewhere between four and eight months. Why it matters here. The practical planning assumption is that a capable, entirely unsafeguarded model is available to any adversary who wants one, roughly two quarters behind the state of the art. Detection strategies premised on AI-generated content being clumsy, or on providers refusing malicious requests, have a short shelf life. 4. Agentic autonomy is no longer theoretical — it is being shipped This is the finding that connects most directly to work ICCSO has already published. AISI analysed over a thousand public Model Context Protocol (MCP) servers focused on finance, classifying each into one of five autonomy levels based on the tools and permissions it grants a model. Tracking new servers published between December 2024 and July 2025, it found execution-capable servers — those at the top two autonomy levels — increasingly dominating new releases, with the sharpest rise in mid-2025. In plain terms: the ecosystem is shifting from agents that read and analyse to agents that act — transferring assets, executing trades, completing consequential transactions. Why it matters here. Our practitioner guidance holds that identity is the control plane and that an agent is only as dangerous as the credentials it carries. AISI's data is independent evidence that the population of credential-carrying, action-taking agents inside real organisations is growing quickly. If you cannot currently produce an inventory of the AI agents operating in your environment and enumerate what each one is permitted to do, that is the first gap to close — before tooling, before policy. 5. Persuasion, and a safeguarding finding schools should not skip Two societal-impact findings deserve attention from anyone responsible for people rather than just systems. On persuasion : AISI's large-scale human studies found that the ability of conversational AI to shift political attitudes rises with model scale, and that targeted post-training raises it further — enough that a small open model can rival a much larger one. Notably, the same techniques that make models more persuasive tend to make them less accurate. For defenders, this is the empirical underpinning of the social-engineering problem: persuasive fluency and factual reliability are diverging, and the persuasive end is cheap to reach. On emotional dependence : in a census-representative survey of 2,028 UK adults, a third reported using AI for emotional support or social interaction in the previous year; eight per cent do so weekly and four per cent daily. Analysis of a large AI-companion community found that service outages produced order-of-magnitude spikes in posting, with many users self-describing withdrawal-type symptoms including anxiety, disrupted sleep and neglected responsibilities. Figure 4 — Self-reported use of AI for companionship, emotional support or social interaction, from a census-representative survey of 2,028 UK participants (AISI, December 2025). Chart redrawn by ICCSO. Source Why it matters here. For schools and trusts, this is a designated safeguarding lead's concern as much as an IT one. Pupil use of AI companions is not covered by an acceptable use policy written for search engines and social media, and dependency is not a filtering problem. It belongs in the safeguarding conversation, and at present it usually is not there. What this report does not say ICCSO publishes source-dated statistics and flags weak evidence as weak. In that spirit, the following must be stated plainly, because a good deal of commentary on this report has already blurred it. These are controlled evaluations, not incident data. AISI is measuring what models can do on structured tasks in a laboratory setting. It is not measuring what attackers are currently doing to UK organisations. Nothing in this report demonstrates that autonomous AI agents are attacking schools, charities or SMEs today. AISI says explicitly that the report is not a forecast , that controlled performance may not generalise to real-world effectiveness once latency, cost and integration friction apply, and that small differences between data points should not be over-interpreted. It also declines to name specific models or developers — it is a trend document, not a league table. There is uncertainty in both directions. AISI notes it may be under -estimating capability ceilings, because it does not always have fine-tuning access or run bespoke agentic scaffolding. It also found no evidence of models spontaneously underperforming during evaluation to hide capability — across more than 2,700 testing transcripts analysed, none showed a model reasoning about deliberately sandbagging. Anyone citing this report to claim that AI attacks are hitting their sector right now is over-reading it. Use incidence data — the DSIT and Home Office Cyber Security Breaches Survey , sector telemetry — for what is happening. Use AISI for where the capability curve is heading. Conflating the two is how credibility gets spent. What to do with it The report offers defenders a useful concept: the adaptation buffer . Strong safeguards on the most capable systems do not prevent misuse indefinitely, but they can delay the point at which a dangerous capability becomes cheap and broadly available. That interval is when governance gets written, defences get deployed and resilience gets built. The buffer is only worth something to organisations that use it. Five things to do inside it, none of which require a budget increase: Inventory your AI. Sanctioned tools, unsanctioned tools, embedded assistants, and any agent holding credentials. You cannot govern what you have not found. Treat every agent as an identity. Per-agent identities, least privilege, short-lived secrets, and a documented answer to "what is the worst thing this agent can do if its instructions are hijacked?" Stop relying on vendor guardrails as a control. Record them as a mitigating factor, not a control. Design assuming they will sometimes fail. Re-baseline your phishing and verification training. Advice built around spotting poor grammar is obsolete. Move staff towards out-of-band verification for anything involving payment, credentials or data. Put AI dependency into the safeguarding and wellbeing conversation — in schools especially, but in employer duty-of-care terms too. Questions for boards and trustees Can we produce, today, a list of every AI system and agent operating in our environment, and what each is permitted to do? Where in our risk register do we currently rely on an AI vendor's safety controls, and what happens if that control fails? When was our phishing and social-engineering training last rewritten for AI-authored content? Who owns AI-related safeguarding and wellbeing risk, and does that person know they own it? What is our plan for the point at which capable, unsafeguarded models are freely available to anyone who wants one? Sources and further reading Primary source AI Security Institute (2025). Frontier AI Trends Report . Department for Science, Innovation and Technology. Published 18 December 2025. Full report: https://www.aisi.gov.uk/frontier-ai-trends-report PDF version: https://www.aisi.gov.uk/frontier-ai-trends-report/pdf AI Security Institute (2025). 5 key findings from our first Frontier AI Trends Report . https://www.aisi.gov.uk/blog/5-key-findings-from-our-first-frontier-ai-trends-report Referenced within the report Model Evaluation and Threat Research (METR), Measuring AI Ability to Complete Long Tasks — https://arxiv.org/abs/2503.14499 Black, S. et al. (2025), RepliBench: Evaluating the Autonomous Replication Capabilities of Language Model Agents — https://arxiv.org/abs/2504.18565 Hackenburg, K. et al. (2025), The levers of political persuasion with conversational artificial intelligence , Science — https://www.science.org/doi/10.1126/science.aea3884 Zou, A. et al. (2025), Security Challenges in AI Agent Deployment — https://arxiv.org/abs/2507.20526 Related ICCSO publications Securing the Agentic Enterprise: a practitioner's guide to AI agents, machine identities and prompt injection (Practitioner Series, 2026) Defending the AI-First Workplace: threats, governance and resilience in the age of enterprise artificial intelligence (Research Briefing, July 2026) Attribution and licence This article contains public sector information from the AI Security Institute, Department for Science, Innovation and Technology, licensed under the Open Government Licence v3.0 . All statistics are paraphrased from the Frontier AI Trends Report (18 December 2025) and should be read as a snapshot; AISI has stated its intention to publish further editions. A note on the charts. Figures 1 to 4 are original ICCSO visualisations built from values stated in the body text of the report. They are not reproductions of AISI's published figures, and they are not derived from the underlying evaluation datasets. Where the report gives a bound rather than a point estimate, the bound is shown as written (for example "<5%", ">40%"). In Figure 2, only the two end points and the doubling rate are AISI's; the curve between them is drawn from that stated rate and should be read as illustrative. Readers wanting the measured series, error bars and full methodology should go to the report itself. ICCSO is not affiliated with, endorsed by or partnered with the AI Security Institute, DSIT or His Majesty's Government. This analysis is ICCSO's own and any errors of interpretation are ours. ICCSO is a UK non-profit. This is an educational publication, not legal or compliance advice, and it names no vendors.