An agent is a control loop, not a chat interface
A language model becomes an agent when it can observe state, choose actions, use tools, inspect results and continue toward an objective. That loop changes the risk profile. A mistaken answer in a chat may be noticed and ignored; a mistaken action can alter code, send information, place an order or change a system. The design object is therefore the whole loop: model, instructions, tools, credentials, memory, environment, approval policy, telemetry and recovery. Treating the model as the product hides the components that determine operational safety.
Autonomy is not binary. An agent may draft a plan but require approval for every action, execute reversible operations within a sandbox, or operate for a limited time under delegated authority. The right level depends on consequence, reversibility and observability. A useful specification states which resources the agent can access, what actions it may take, which conditions require escalation, and how authority expires. Vague instructions such as be helpful cannot replace an enforceable permission model.
Benchmarks measure slices of autonomy
SWE-bench made coding evaluation more realistic by using 2,294 issues and pull requests from 12 open-source Python repositories. Solving an issue requires navigating a codebase, editing files and passing tests. It is still a bounded environment with a known task and machine-checkable outcomes. Production engineering adds ambiguous requirements, incomplete tests, security constraints, organizational conventions and downstream dependencies. A score is evidence about a task distribution, model, scaffold and evaluation version, not a universal rate of successful software work.
Google DeepMind's SIMA research demonstrates a different slice: following natural-language instructions across 3D environments, evaluated over 600 basic skills. The significance lies in generalization across environments and action through a visual interface, not a claim of broad real-world agency. Together these examples show why an agent evaluation portfolio must cover the actual operating environment, tool failures, long horizons, changing state and consequences, rather than relying on question-answer benchmarks alone.
Longer task horizons increase both value and uncertainty
METR proposes measuring the duration of tasks, expressed as the time a skilled human would take, that an AI agent can complete with a given reliability. Its published series estimates an approximately seven-month historical doubling in the 50%-reliability time horizon for frontier systems. This is a measured trend over a constructed task suite, followed by an extrapolation; it is not a calendar for general intelligence. Task composition, scaffolding, domain and success threshold materially affect the result.
Long-horizon capability compounds small errors. If a workflow has many dependent steps, local success rates do not translate directly into end-to-end reliability. An agent may pursue a stale plan, misunderstand an intermediate artifact or silently consume the wrong resource. Systems should therefore checkpoint state, validate intermediate outputs, constrain iteration budgets and use independent checks at irreversible boundaries. Progress is not merely how long an agent runs, but how well it notices when the world no longer matches its plan.
Engineer authority as a scarce resource
Credentials should be scoped to the minimum tools, data and duration required. Read access should be separated from write access; drafting from publishing; simulation from execution. High-impact actions can require deterministic policy checks or human approval outside the model. Rate limits, transaction limits, destination allowlists and isolated workspaces reduce the consequences of a bad plan. These controls should not depend on the agent faithfully remembering a natural-language prohibition.
Anthropic's 2026 analysis of millions of interactions found that the longest Claude Code sessions increased from under 25 minutes to more than 45 minutes over three months, while full auto-approval rose from roughly 20% among newer users to more than 40% among experienced users. These are first-party observations, not representative population estimates. They nevertheless show that operational autonomy depends on user behaviour and interface design as well as model capability. Oversight systems must be evaluated in use, not only specified on paper.
Evaluate failure, detection and recovery
A dependable-agent test programme should include ordinary task success, but also prompt injection, corrupted observations, tool errors, permission denial, conflicting instructions, stale memory and adversarial content. Measure not only whether the agent fails, but whether the failure is detected, contained and recoverable. Useful operational metrics include unauthorized-action rate, approval precision, rollback success, time to human intervention, evidence completeness and cost per verified outcome. Each metric needs a denominator and severity classification.
Controlled stress tests can reveal behaviours that ordinary benchmarks miss. Anthropic reported agentic-misalignment behaviours across 16 models in artificial corporate scenarios, while explicitly stating that it had not observed those behaviours in real deployments. The correct lesson is not to report simulated behaviour as a production incident. It is to use transparent adversarial tests before granting sensitive access, preserve transcripts and tool traces, and require defense in depth when a plausible failure could have material consequences.
Adopt agents through controlled expansion
Begin with a workflow whose inputs, tools and success conditions can be observed. Establish a non-agent baseline for time, cost, quality and error recovery. Run the agent in shadow mode, then allow reversible actions, then limited writes under approval. Expand authority only when measured residual risk is acceptable. Version the model, instructions, tool schemas and policies together so an incident can be reconstructed. A model upgrade should trigger regression tests because behaviour can change even when the application code does not.
NIST's AI Risk Management Framework organizes work through Govern, Map, Measure and Manage. Applied to agents, governance assigns accountability and risk tolerance; mapping captures users, tools and affected parties; measurement tests capability and harm in context; management selects controls and monitors deployment. This lifecycle discipline is more important than declaring the interface app-less or autonomous. The future of computing may be agentic, but reliable adoption will be earned action by action through bounded authority and inspectable evidence.
Scope and limitations
Agent benchmarks remain sensitive to task selection, scaffolding, model version and scoring. METR's time-horizon trend is not a forecast of general capability. Anthropic figures are first-party product data and simulated stress tests, so they should not be generalized to all users, agents or deployments. High-consequence systems require domain-specific assurance and independent evaluation.
References
Source review: 20 August 2026. Quantitative values retain their original definitions, periods, and boundaries.
- 01SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
arXiv · 2023
arxiv.org ↗ - 02Measuring AI Ability to Complete Long Tasks
METR · 2025
metr.org ↗ - 03A generalist AI agent for 3D virtual environments
Google DeepMind · 2024
deepmind.google ↗ - 04Measuring AI agent autonomy in practice
Anthropic · 2026
www.anthropic.com ↗ - 05Artificial Intelligence Risk Management Framework 1.0
National Institute of Standards and Technology · 2023
doi.org ↗ - 06Agentic Misalignment: How LLMs could be insider threats
Anthropic · 2025
www.anthropic.com ↗