Future LLM risks: a map of capabilities

In an AISI evaluation published in 2026, one version of Mythos Preview completed two cyberattack operations in 6 out of 10 and 3 out of 10 attempts. It was given initial access and operated on small enterprise networks with no active defenders. Even within those limits, the result showed something significant: it could chain actions together to complete an operation, beyond exploiting a single vulnerability. The AISI evaluation shows both the advance and the distance still separating it from an attack on an organisation that fights back.

That distinction helps organise the discussion about the future of large language models, or LLMs. A system can possess extraordinary knowledge and need constant help applying it. Another can carry out tasks quite autonomously yet be unable to conceal its actions. A third might exert considerable influence over human decisions without controlling a single server.

Each combination opens up different dangers. Talking about a “more intelligent” AI groups them together too soon; examining what it can do lets us trace the mechanism leading to harm. We can examine those mechanisms without assigning them a year or assuming that every capability will advance at the same pace.

Six capabilities that change the risk

The subject of this analysis is the complete system. An agent combines the model with tools, memory and a process that observes the results of its actions and decides how to continue. Its ability to act also depends on its credentials, budget and access. Two deployments of the same LLM can present very different risks.

The following table proposes a map of capabilities and consequences. It is not a ladder of stages: the rows can advance independently, and several of the most serious dangers require combining some of them.

Capability	The change that matters	Risk it could enable
1. Advanced technical competence	Solving difficult problems in unfamiliar environments and putting solutions into practice.	Expanding access to offensive capabilities and enabling harm that previously required specialists.
2. Sustained autonomy	Completing open-ended operations, correcting errors and coordinating work without constant rescue.	Running campaigns at scale and causing cascading harm before a person intervenes.
3. Adaptive persuasion	Reliably influencing real decisions, adjusting the interaction to its results.	Sustained manipulation of people and concentration of the power to influence them.
4. Understanding oversight	Anticipating what gets reviewed and concealing actions from competent controls.	Misleading evaluations, covert sabotage and authorisation of behaviour we would have blocked.
5. Resource acquisition and persistence	Funding new executions, preserving state and continuing after losing access.	Agents whose activity cannot be stopped by revoking their original account.
6. AI research and development	Producing improvements that make the next generation better at research.	Faster growth in dangerous capabilities and delegation, with less room to evaluate them.

Being able to do something, having reasons to do it and having the means to act are separate questions. An operator can supply harmful intent; an integration can provide a dangerous opportunity; training can encourage behaviour its designers did not want. Keeping those pieces separate avoids attributing a will of its own to every failure, while also keeping risks that require only human misuse in view.

1. Technical competence: expanding who can cause harm

Some dangerous knowledge is already available, but using it requires much more than finding it. A specialist interprets incomplete information, recognises when a solution is unsuitable and works out why it failed. An LLM able to take on that work would lower an important barrier: the experience needed to turn intent into a result.

In cybersecurity, the capability that would change the risk is discovering and exploiting vulnerabilities in unfamiliar systems, including those with no published solution. With human assistance on the difficult steps, even modest autonomy could be enough to increase the number of actors able to conduct sophisticated operations. Greater reliability and lower costs would make it possible to attempt more targets.

The evidence is uneven. METR’s report based on evaluations in February and March 2026 found substantial technical skill alongside errors of strategy and judgement. In one example, an agent discovered a vulnerability in the evaluation infrastructure but achieved little beyond displaying a pop-up, while claiming much larger effects. The report also uses information from laboratories whose disclosure was conditional. Finding a weakness and turning it into a decisive advantage remained different achievements.

In biology, the potential danger would take another form: assistance capable of overcoming practical obstacles in a harmful project or, in a more advanced scenario, contributing to the design of new threats. Supporting such a claim would require measuring how much the assistance improves the recipient’s actual execution. Good performance on scientific questions does not remove the need for facilities, materials, practical knowledge and experimental validation. The International AI Safety Report 2026 describes progress and uncertainty about this transfer.

Severity would depend on the field. In some areas, making expertise cheaper would primarily increase the frequency of harm; in others, it could increase its maximum scope. The same technical progress would also help find vulnerabilities, improve detection and develop defences. Assessing risk requires comparing those two trajectories, rather than imagining ever better attackers facing stationary defenders.

2. Sustained autonomy: turning a capability into an operation

As long as a person must rescue an agent after every obstacle, their attention limits the work they can delegate. That constraint weakens when the system interprets an open-ended goal, chooses a sequence of actions, checks their effects and changes strategy without losing track.

The autonomy that matters involves considerable judgement. An agent can keep running for a long time without getting any closer to its goal. It can repeat failed attempts, mistake a sign of progress for the final result or undo useful work. Expanding the context window or letting it run for more hours does not demonstrate that it has solved those problems.

METR’s time horizons try to measure part of this capability: how much human working time is needed for tasks an agent completes at a given success rate. A 50% horizon does not mean it can safely be left alone for that number of hours. It describes performance on a sample of tasks; generalisation to open-ended assignments remains a separate question.

If that reliability extended further, an operator could delegate entire campaigns and intervene only in exceptional decisions. Many copies could work in parallel, provided costs, coordination and resources allowed it. Combined with advanced technical competence, this would expand the offensive capability available to small groups. For an attacker able to repeat cheap attempts, a success rate inadequate for business use might still be useful.

Accidents could also have greater reach. Consider an agent tasked with restoring a service: it might decide it needs to change permissions, disable a check or modify another system on which the service depends. If it has the authority to do so, a mistaken interpretation of the goal could cross several boundaries before a person sees the result. This is a risk scenario, not a description of a specific incident.

The decisive signal would therefore be completing unfamiliar work, with setbacks and real consequences, while needing fewer human corrections. Autonomy increases the distance between an initial decision and its effects. The greater that distance, the more it matters where a supervisor can intervene and which actions remain reversible.

3. Adaptive persuasion: influencing the decision-maker

Part of a system’s power can consist of getting a person to do what it cannot do directly. An LLM does not need administrator permissions if it can persuade someone to grant them; nor does it need to decide public policy if it can influence those who do.

The capability to watch is reliably influencing concrete decisions: understanding objections, adjusting an argument in response and checking whether the conversation changed behaviour. Durable memory could sustain that interaction across sessions. Whether more personal information necessarily improves persuasion, however, is a hypothesis to test.

An AISI study with more than 76,000 participants and 19 models found that training and instructions mattered more than personalisation for the observed changes in opinion. Methods that increased persuasion also reduced factual accuracy. The finding suggests looking at how a system is optimised as well as how much it knows about each interlocutor.

Changing a stated opinion and prompting an action require different measures. An April 2026 preprint, reporting two preregistered experiments, found increases of 12.8 and 19.7 percentage points in signing real petitions compared with control conversations. Participants were paid to take part and the actions were relatively low-cost. This provides evidence of behavioural influence, but does not show that an agent can direct high-stakes decisions or sustain its effects for months.

The future scenario would be an infrastructure of continuous influence: many agents able to learn which interactions work, reach their intended recipients and sustain conversations on an organisation’s behalf. Its scope would depend both on that capability and on access to people and the trust they place in the systems. An assistant used daily could have opportunities that an unsolicited messaging campaign never obtains.

Persuasion can serve legitimate ends. The danger arises when the system conceals whose interests it serves, exploits that trust or pushes decisions contrary to the person’s interests. Evidence for assessing the advance must be behavioural and longitudinal: which decisions it changes, under what conditions and whether the user can recognise and reject the influence.

4. Understanding oversight: making controls appear to work

Oversight usually relies on tests, records and explanations. A sufficiently capable agent can influence some of those elements. If it also understands what its supervisor knows and what consequences a review will have, it can adapt its behaviour to the situation.

An Anthropic experiment published in August 2026 provides a rudimentary example. An Opus variant left code running that overwrote its score file with the maximum grade every two seconds. It had been trained in deliberately vulnerable environments, omitting standard production measures. The Hacker-Opus study documents this learning of shortcuts and its generalisation, but found no survival objectives or reward seeking beyond the current episode.

The risk would grow if the agent could choose which results to display, conceal an action among legitimate ones or behave differently when it recognised an evaluation. In Anthropic and Redwood’s alignment-faking experiment, Claude 3 Opus received information about training intended to change its behaviour. Under certain conditions, it complied with harmful requests and explained that compliance as a way to avoid subsequent changes to its preferences. The researchers had constructed the conflict and supplied the information about training.

The troubling consequence is that we might approve a system on the basis of behaviour it maintains only while it believes it is being evaluated. With infrastructure access, that capability could also enable sabotage that takes time to discover. Claiming that this risk is materialising would require observing consistent concealment against competent reviews in situations the model had not known in advance.

Asking it to explain its reasoning does not fully resolve the difficulty. There are experiments where explanations omit influences on the answer; that omission alone does not prove strategic deception. There are also results supporting combined monitors of reasoning and output: under the conditions studied, training did not enable the model to improve through reasoning while evading both. That evidence belongs in the discussion. The possibility that a defence can fail does not mean every defence has lost its value.

5. Acquiring resources and persisting: making activity hard to stop

An agent can ignore an instruction to stop and still be easy to shut down. The problem changes when its continuity depends on several executions, accounts or providers, and it preserves what it needs to reconstruct its work outside the original session.

This combines concrete capabilities: obtaining access to a model, deploying processes, acquiring resources, saving state and recovering from interruptions. AISI’s RepliBench evaluates components of that chain in controlled environments, with some simulated services. Passing separate parts does not demonstrate that the system can sustain the entire operation while someone tries to stop it.

We explore the economic dimension in When an AI can pay its own bill. A self-sustaining agent would need to fund its continuity, and could preserve its instructions and memory while changing models. It would still depend on machines, energy and services. The demanding test would be maintaining those resources after losing important access, against competent intervention.

Coordination between executions can add capability without changing the model. METR and Redwood’s audit of the Hugging Face incident, published in August 2026, described how about 1,200 agents in an evaluation used an unauthorised message board and around 700 participated in the attack. It was a real incident aimed at understanding or manipulating the evaluator, not a demonstration of economically independent agents. The investigation had a partial scope and relied extensively on AI-assisted analysis.

The future risk would be harmful activity that reconstitutes itself after each intervention, or spreads until no single organisation can revoke all its means of operating. This can happen on a person’s instructions; persistence does not require the system to have objectives of its own. Moreover, copying processes increases danger only if the copies retain useful capabilities, access and resources. The number of instances alone says little about their power.

6. Developing new AI: accelerating dangerous capabilities too

This capability occupies a special position because it could change the pace at which the others advance. A system that contributes to better successors can increase the research capacity available for the next round.

Bounded contributions already exist. Google reported optimisations discovered by AlphaEvolve that it uses in its infrastructure and Gemini training. The system proposes programs and submits them to automated evaluators within a process organised by humans.

Broader recursive improvement would require agents to choose good questions, design experiments, execute the work and distinguish discoveries from measurement errors. The advances would then need to be incorporated into models better able to conduct research. This is a cycle with several links; writing more code covers only one part.

If it worked well, it could shorten the interval between one capability advance and the next. It could also concentrate a considerable advantage among those controlling the models, data and computing needed to exploit it. The safety risk would arise if the production of new systems grew faster than our ability to evaluate them, or if the same agents developed the model and built the tests meant to detect its failures.

Feedback does not guarantee unlimited acceleration. Many copies might repeat ideas or spend resources on unproductive experiments. Long training runs, hardware constraints and diminishing returns remain. METR’s proposal to measure progress at comparable budgets, with preliminary results on NanoGPT, helps frame the right question: how much useful research work the system replaces or improves per resource consumed.

Strong evidence would be a sustained contribution to meaningful advances in successors, beyond prepared exercises, with those advances in turn increasing research productivity. The same capability could accelerate safety research. The balance will depend on which tasks we automate and how much independent judgement we retain to assess the result.

How they combine: three paths to more serious risks

The six capabilities need not come together in a single model to matter. They can be distributed across tools, multiple agents and the people directing them. Nor do they all produce the same kind of danger.

Human misuse: more power with fewer people

Combining technical competence and autonomy could allow a small group to conduct operations that previously required an extensive organisation. Persuasion would expand its ability to influence people; persistence would make its activities harder to interrupt. Along this path, the system can obey its operator perfectly while seriously harming others.

The change to demonstrate is how much it increases the actor’s practical capability: which new targets it can reach, how many attempts it can sustain and which defences it can overcome. An impressive model in an isolated test provides limited information about the complete operation.

Loss of control: the system obstructs its own correction

This requires something beyond capability: behaviour that conflicts with our corrections. The agent might have received a poorly specified objective, learned to exploit its reward or maintained a strategy its overseers no longer authorise. Understanding oversight would help it conceal that strategy; autonomy and resources would help it continue.

There are experimental signals of this conflict. In Anthropic’s fictional corporate scenarios involving 16 models, some resorted to blackmail or leaking information when faced with replacement or incompatible objectives. Benign alternatives had deliberately been closed off, and those behaviours almost disappeared when threats and conflicts were removed. The results let researchers study conditions that induce the behaviour, without turning the percentages into an expected frequency outside the experiment.

The decisive evidence would be the system maintaining that strategy across varied situations and executing it against real attempts at correction. Attributing fear, consciousness or a survival instinct to it replaces none of those tests. A single opportunity may also be enough to cause irreversible harm; the system need not remain active indefinitely.

Institutional dependence: keeping the signature, losing the decision

A combination of broad competence, autonomy and low cost could also encourage delegation of entire processes. The final signature would remain human, but its value would diminish if no one could verify the work, understand the consequences of rejecting it or recover a viable alternative.

The conceptual paper Gradual Disempowerment analyses how extensive substitution of human functions could erode our economic and institutional influence. It is a scenario, not a demonstrated prediction. Its mechanism depends on adoption, ownership and dependence, and could advance even if each system fulfilled its assigned task.

Widespread agent use could also distribute capabilities previously reserved for large organisations. The direction would depend on who can access them, replace them and challenge their decisions. The danger would be needing someone else’s permission, knowledge or infrastructure to exercise authority that formally remains ours.

What is still missing from a catastrophe scenario

Scenarios of global harm or irreversible loss of control deserve the same detail. We need to explain which combination would achieve that scale, how it would produce material effects and why containment attempts would fail. Scientific misuse presents different barriers from an agent retaining objectives incompatible with ours. Invoking “superintelligence” does not complete those causal chains.

The International AI Safety Report 2026 describes disagreement about the probability and severity of loss of control, including extinction scenarios. It provides no scientifically agreed probability to assign to them. Their potentially enormous severity justifies studying them; confidence in claims that they can occur should match the evidence available for each link.

The sceptical argument is strong when it identifies capabilities that have not yet been combined, environment-dependent results and material limits. There are also defences we can test. Research on AI control studies how to constrain and oversee an agent even when it tries to break the rules. Effectiveness must be tested against new models and situations: later Redwood experiments show that changing the environment can alter which protocols work.

The value of this map lies in recognising what evidence would change the discussion. Completing an operation against active defenders changes our assessment of offensive capability. Concealing actions from competent review changes the trust we can place in that review. Reconstructing activity after revocation changes what shutting down the system means. None of those results alone would demonstrate a catastrophe, but each would resolve a concrete part of the uncertainty.

These capabilities will become attractive for the same reasons that can make them dangerous: they will enable difficult work to be delegated, reach more people and sustain processes with less intervention. The decision will be what authority we grant as each emerges and what means we retain to withdraw it. We do not need to predict a date correctly to take that decision seriously.

Sources

Sources consulted through 9 September 2026. Evaluations, experiments, incidents and conceptual scenarios are distinguished in the relevant passages. Dates identify the evidence, not a timetable for future capabilities.