Copilot's new agent metrics: what “leverage” measures, and what it doesn't

An industry leaderboard for merge rates is useful context and a dangerous target.

Published October 23, 2026. On October 22, GitHub shipped two changes to how organizations measure Copilot's agent features. The Copilot impact dashboard gained an opt-in industry comparison that ranks your agent adoption and a “leverage” score against peer companies. Separately, the usage metrics API now reports custom skills, agents, and MCP servers by name instead of lumping them into other or custom.

Both changes give engineering leaders something new to look at. One is a solid inventory tool. The other is a benchmark that is very easy to misread. Here's how to tell them apart before a number ends up in a quarterly review.

What actually shipped

Industry comparison. The dashboard section is off by default. When an enterprise owner turns it on, or the owner of an organization not owned by an enterprise, it shows two things:

The percentage of your users who are “agent-first or multi-agent.”
How the pull request merge rate of agentic users compares with that of non-agentic users. GitHub calls this “leverage.”

Your scores are ranked against the peer median for a 28-day window, and against the top decile once enough companies participate. Peers are grouped by company size (1–999, 1,000–4,999, or 5,000+ employees) and by one of ten industries. If your industry group is too small, the comparison widens to all industries at your size. Opting in to view comparisons also means contributing your own data, through the same policy. GitHub says the aggregated benchmarks contain no identifying data and are deleted after 30 days. The comparisons appear only in the dashboard, not in the API or in NDJSON exports.

Named custom tooling. In the usage metrics API, the totals_by_skill, totals_by_custom_agent, totals_by_mcp, totals_by_slash_cmd, and totals_by_plugin arrays now show each item's configured name, with up to 25 entries per record instead of five. Names come from Copilot CLI 1.0.97 or later, the Copilot app, ACP integrations such as JetBrains, and Copilot SDK v1.0.21 or later. They don't come from an IDE's built-in chat telemetry, and days processed before the release still show other.

The problem with “leverage”

A higher merge rate for agentic users than for non-agentic users sounds like proof that agents help. It isn't, for at least three reasons.

Selection bias. The people who adopt agent workflows first aren't a random sample. They tend to be the engineers who work on well-tested services, pick tractable tickets, and already merge a lot. If your agent-first group was your strongest group before Copilot, a leverage gap shows that, not what the agent added. The dashboard compares two groups at one point in time. It doesn't compare the same people before and after.

Merge rate isn't value. A merged PR can be a one-line config bump or a migration that took a week. It can also be a change that gets reverted two days later. Agents are especially good at producing many small, easily approved PRs, which raises merge rate whether or not they move the work forward. Our ROI breakdown and technical debt piece cover why review time and rework belong in the denominator.

It's gameable the moment it's a target. Rank a team against an industry top decile and someone will eventually optimize for the rank. Splitting work into smaller PRs, approving agent PRs faster, or steering agents toward easy tickets all raise leverage without raising output. That's Goodhart's law, and a leaderboard invites it.

None of this makes the number useless. It's a reasonable adoption signal and a prompt for questions. The trouble starts when it's treated as a productivity result.

Read the benchmark's own caveats

GitHub's changelog is candid about the early limits, and they're worth repeating to anyone who sees the dashboard:

Thin peer groups at launch. Rankings appear only once enough businesses opt in. Early on, you may see only a median, a broader peer group than you chose, or no ranking at all.
No baseline, no score. If all your users are already agent-first, there's no non-agentic group to compare against and no leverage score.
The peers chose to be there. Companies that opt in to compare their agent adoption are probably more agent-forward than average. A median among volunteers isn't an industry median.
Regional isolation. On GitHub Enterprise Cloud with data residency, you're compared only with participants in the same region.

The more useful change: seeing which custom tooling gets used

The quieter change is the one most platform teams should act on. If you've written custom skills, agents, or MCP servers for Copilot, you can now see activity for each one by name, per day. Be clear about what the counts mean, though. For skills, slash commands, and plugins, interaction_count counts invocations, and for custom agents it counts starts. For MCP servers it counts something else entirely. GitHub's definition says it “increases only when Copilot CLI attempts to connect or reconnect to the server,” and failed attempts count too. Tool calls on a server that's already connected don't add to it. So an MCP server that's configured but unreachable can still show activity without a single tool call. With that caveat, the data helps with questions that used to require guesswork:

Which of the twelve internal MCP servers you maintain are even configured in developers' sessions? A server with zero connection attempts is a strong retirement candidate. One with attempts may still never be called.
Did the new code-review skill get picked up after you announced it, or did usage spike for a day and fade?
Is one team relying on an unofficial agent nobody reviewed?

Two practical notes. First, names are reported exactly as configured (trimmed, and capped at 128 characters), so jira-mcp-prod and Jira MCP count as two entries. Standardize naming before you start trending this. Second, GitHub warns that the data comes from client telemetry and can undercount, for example with old clients or blocked telemetry, and recommends using it “for trends and adoption, not exact accounting.” Upgrading everyone to CLI 1.0.97 or later and SDK v1.0.21 or later will make the numbers more complete. It won't make them exact.

Also review the names before sharing reports outside your organization. GitHub calls this out explicitly, since an internal skill name can reveal more than you'd expect.

What to do with this

Opt in to the comparison if you want context, not a goal. Look at where you sit, then ask why. Don't put the percentile in an OKR.
Pair leverage with outcomes that are harder to game. Track revert rate, time to merge, and change failure rate for agentic PRs next to merge rate. If leverage rises while reverts rise too, the agent is shipping churn.
Compare the same people over time. To estimate what agents add, track one team's metrics before and after adoption instead of comparing adopters with holdouts.
Use the named-tooling data to shortlist, then confirm. Every custom MCP server is code you maintain and attack surface you own. Zero connection attempts makes a server a candidate for retirement. Before you remove one that does show attempts, check its own request logs for real tool calls.

We haven't seen the dashboard with live peer data. This analysis is based on GitHub's changelog descriptions, and early rankings may look different once more companies participate.