Anthropic's Agentic Coding Study: What 400K Sessions Tell Us About Expertise and AI

In June 2026, Anthropic published Agentic coding and persistent returns to expertise, a privacy-preserving analysis of approximately 400,000 interactive Claude Code sessions from approximately 235,000 people, spanning October 2025 through April 2026. The study examines task composition, human-AI collaboration, and session success. Its central result is that returns to domain expertise persist: more experienced users get more work done per instruction, and sessions from people with more domain expertise are more likely to end successfully.

The result needs its qualifier: Anthropic describes the success-rate gap between intermediate and expert users as modest, and reports that coding sessions across major occupations succeed at nearly the same rate as software engineers on average. That does not show that expertise stops mattering, but it also does not support treating the gap as enormous. The findings concern observed Claude Code sessions; they do not establish that senior developers always gain more productivity, or that the same pattern holds for every editor, CLI agent, or model.

How to apply the finding to other tools—as a hypothesis

The report itself is Claude Code-specific. For Cursor, Copilot, Windsurf, Codex CLI, Aider, and Cline, treat the expertise gradient as a question to test—not as a result already established by this dataset. Their interaction models differ, and the supplied research does not compare them. A team evaluating another tool should measure accepted changes and review effort on its own repositories before generalizing.

The transferable takeaway is methodological: do not infer a tool's value from generated code or session activity alone. Check whether the work was completed with verifiable evidence, and count the supervision and review needed to get there. That makes expertise a plausible factor to measure without pretending this study proves the same effect across products.

What the study measured

The research builds on prior Anthropic work on agent autonomy and uses a privacy-preserving analysis of sessions to study task composition, collaboration, and success. The key findings supplied here do not specify every measure used to estimate expertise, nor do they establish that an observed association is caused by expertise itself.

This is observational usage data, not a randomized trial or a benchmark with a control group. The scale provides a broad view of sessions, but it does not by itself establish that Claude Code caused the outcomes or that the results generalize to developers who do not use it. Anthropic says session success is assessed using verifiable evidence such as passing tests or committed work; the supplied summary does not make the dataset a cross-product quality comparison.

Anthropic also reports that the share of sessions spent debugging fell by nearly half over the seven-month window, use shifted toward more end-to-end work, and estimated typical task value rose about 25% on average. The task-value figure is an estimate based on comparison with freelance job postings—not measured pay, saved time, or a direct productivity gain.

The expertise gradient: what it looks like in practice

The paper's core finding is that returns to expertise are persistent during the observed period. It does not follow that the gap widened as models improved: Anthropic calls the success-rate gap between intermediate and expert users modest. Keep those two points together when summarizing the result.

The study does not compare model generations or measure how the expertise gap changes after a capability upgrade. What it does report is a relationship between domain expertise, work completed per instruction, and the likelihood that a session ends successfully. It is evidence against assuming expertise becomes irrelevant, not evidence that better models necessarily widen the gap.

For teams, these are sensible pilot questions—not separate effects quantified by this study:

• Task selection: Does the agent save time on this task compared with implementing it directly, after review and correction?
• Scoping: Do bounded requests reduce clarification and rework for developers at different experience levels?
• Output evaluation: Can reviewers reliably identify incorrect approaches and missed edge cases in the generated changes?
• Recovery: How much time do people with different levels of domain expertise spend diagnosing and restarting an unproductive run?

What this means for hiring and team composition

The findings do not establish a staffing formula or show that companies can replace senior engineers with AI-assisted junior developers. Nor do they show that senior developers invariably produce more value with an agent. They suggest that domain expertise remains relevant to how much work Claude completes per instruction and to session success, while the reported success gap between intermediate and expert users is modest.

For engineering managers, treat expertise as an evaluation variable, not a verdict about headcount. A team pilot can check whether the association appears in its own tasks and whether review burden differs across experience levels.

Practical implications to test:

1. Do not make staffing decisions from the study alone. It covers one product and does not estimate the effect of replacing one experience level with another.
2. Keep investing in onboarding and review. Give developers ways to learn the domain and verify agent output; measure whether those practices improve accepted outcomes in your workflow.

What the study does—and does not—say about autonomy

The findings supplied here describe planning, execution, expertise, and success; they do not identify an optimal autonomy setting or rank task types by how safely they can be delegated. Do not turn a session-level association into a universal autonomy policy. Teams still need acceptance criteria, tests, and review appropriate to the change.

A practical calibration question remains: can the person running the agent tell whether its result meets the task's acceptance criteria? That is a useful policy and training question, but the study summary does not claim to measure a specific failure mode called misplaced autonomy.

Counter-considerations: the paper is not pessimistic

None of this means agentic coding tools are only for senior developers. The reported near-parity in coding-session success across major occupations is a reason not to gate access by job title. But this session analysis is not a controlled estimate of productivity gains versus a no-agent baseline, so it cannot establish how much faster each group became.

Read the persistence finding as a prompt for evaluation and enablement, not as a staffing verdict. Help developers learn to verify agent output, then assess quality and review burden across experience levels in your own codebase.

What the study does not cover

The paper uses Claude Code sessions specifically. It is plausible that similar patterns apply to Cursor, Copilot, Codex CLI, and other tools, but that comparison is not in the dataset. Different tools have different interaction models — Copilot operates at a more granular completion level than Claude Code's session-oriented model — which may shift the expertise gradient.

The study also covers a specific time window (October 2025 to April 2026) and does not model how the distribution of outcomes evolves as model capability improves further. “Persistent” describes the returns observed in the study; it is not a forecast that the same gap will remain indefinitely.

How to use this research in your organisation

A few concrete applications:

• Before deploying agentic coding tools broadly: run a structured pilot that measures accepted outcomes, review time, and correction effort across experience levels, rather than relying on completion speed alone.
• When designing code review workflows for AI-generated code: match reviewer seniority to task complexity, not just to diff size. A large AI-generated diff in an unfamiliar domain is a high-risk review even if it looks clean on the surface.
• When evaluating AI coding tool ROI claims: check what “success” means, how it was measured, and whether results from one experience mix transfer to yours.
• When onboarding developers to agentic workflows: teach both tool operation and how to verify changes against the relevant codebase and tests.

What to change this week if you run multiple tools

1. Keep editor and CLI lanes separate: use Cursor/Copilot/Windsurf for local iteration, then require a stricter Claude Code or Codex CLI pass for bounded execution with explicit test targets.
2. Track operator metrics, not just output metrics: intervention count, review minutes per accepted PR, and rollback rate should sit beside token or subscription spend.
3. Pair junior developers with expert reviewers on agent-heavy work: not to police them, but to accelerate judgment transfer on where agents fail silently.
4. Gate autonomous runs hard: if you use Devin or OpenHands, require pre-written acceptance criteria and rollback notes before execution.

The takeaway

Anthropic's study is a large, useful observational account of Claude Code sessions—not a causal estimate of productivity and not a benchmark of the wider coding-agent market. Its useful combination is that returns to expertise persist while the reported intermediate-to-expert success gap remains modest. Teams should preserve both parts of that finding when making tooling or training decisions.

The right conclusion is not that agents erase the value of expertise, nor that they inevitably widen a skills gap. Anthropic's evidence supports a narrower claim: within its observed Claude Code sessions, expertise remained useful, and the success difference between intermediate and expert users was modest. For other tools and teams, measure outcomes directly rather than borrowing a result this study did not test.

Sources: Anthropic: Agentic coding and persistent returns to expertise (June 16, 2026), HN: AI agent coding skeptic tries agent coding, HN discussion on balanced AI-assisted coding take, botspot.dev: Claude Code vs Codex CLI, botspot.dev: Cursor vs Copilot vs Windsurf.