Developer relations metrics when half the audience is a machine
Talks, stars and impressions measured a human funnel. What to measure when adoption decisions no longer involve a person reading.
Developer relations has always had a measurement problem. It got harder in a specific way, and also more tractable in one.
What broke
The classic metrics measured a human funnel: talk attendance, workshop signups, stars, impressions, referral traffic.
Those metrics assumed the person who heard you is the person who adopts. A growing share of adoption now happens without that person reading anything at all. Somebody types one line into a coding agent, the agent reads the repository, picks a product and writes the integration.
Two consequences.
Attribution mostly stops working. The session happens inside Claude Code or Codex. Your analytics see an install with no referrer and no campaign parameter. You cannot attribute it to the talk because there is nothing to attribute with.
Reach is measured in the wrong unit. A talk reaches the people in the room and whoever watches the recording. The repository, the example and the written post it produced reach every agent that later reads them, for years.
The shift: measure artefacts, not audiences
A conference talk is now valuable mostly for what it leaves behind. The repository, the example and the written post are readable by agents indefinitely. The talk itself reaches a room.
That is not a demotion of developer relations. It is a change in what the job optimises for, and it makes the work more durable rather than less.
| Old metric | What to measure instead |
|---|---|
| Talk attendance | Whether the talk produced a maintained template or example |
| GitHub stars | Whether the quickstart in the README runs when pasted |
| Impressions | Whether your product is in the frameworks your users actually run |
| Referral traffic | Install share, measured by running agents |
| Community size | Whether questions in the community become findable pages |
| Ticket volume | Which error messages appear, and whether each has a page |
The metrics worth keeping
Not everything changed. These still measure something real.
Time to first successful call. Still the best predictor of adoption. The reader changed, which changes how you measure it: give a coding agent a repository and a one-line request and watch where it stalls. See time to first successful call.
Issue quality. A community that files good issues is a community that is using the product seriously. That has not changed.
Contributions to your examples. Somebody who fixes your example has read it carefully. That is a stronger signal than a star.
The metrics that are new
Install share. Of the sessions where an agent had to solve a problem your product solves, the percentage where it chose you and wrote working code. Split by agent, language and persona, because the blended number averages markets that behave differently: the three agents we measured named different category leaders in 10 of 18 categories, and the leader changed with the persona in 14 of 18.
Template coverage. How many of your users' top stacks have an official, maintained, current template. This is the highest-leverage number a developer relations team can own now, because in our tests what is already installed in a repository beat an explicit written instruction to use something else, two times out of three.
Example freshness. How many of your published examples run against the current release today. Put them in continuous integration and this becomes a number rather than a hope.
Documented error coverage. What share of your real error messages have a page with a cause and a fix. Agents search the exact error string.
Consideration rate. How often agents raise your product without choosing it. That number separates a presence problem from an integration problem, and they need completely different work. See consideration rate.
What this means for how the team spends its week
Fewer stages, more maintained artefacts. That is the whole recommendation and it is unwelcome to say to a team that likes conferences.
The honest version: the talk still matters for the relationships, the recruiting and the credibility, and those are real. It reaches fewer adoption decisions than it did. The template it inspired reaches more of them every month it stays current.
A developer advocate who knows their install share drops thirty points on TypeScript repositories has a much better idea of what to build next than one who knows their talk had four hundred attendees.
How to start measuring the new ones
Three repositories that look like your users' projects. Ten requests written the way your users describe the problem rather than the way you name the category. Five runs each with two agents. Record what got installed, then read the losses.
300 sessions, a weekend, a few hundred dollars in tokens. It is the cheapest useful measurement in this list and nobody is doing it yet.
Common questions
What should developer relations measure now?
The artefacts rather than the audience. A talk reaches the people in the room; the repository, example and written post it produces reach every agent that later reads them. Measure whether those artefacts exist, are current, and get used.
Are GitHub stars a useful metric?
As a rough popularity signal, weakly. They do not measure whether an agent can integrate the library correctly, which is what decides adoption when the agent is doing the integrating.
How do you attribute adoption that happens inside an agent?
You largely cannot, by referrer. The session happens inside Claude or Codex and your analytics see an install with no campaign parameter. Measure install share directly by running agents instead of trying to attribute after the fact.
Does that make developer relations unmeasurable?
No, it moves the measurement earlier. Instead of attributing installs to a talk, measure whether the talk produced a maintained template, whether that template is current, and whether agents working in that stack now reach for your product.
Where this comes from
Armature ran 5,292 judged sessions with Claude Code, Codex and Cursor inside 51 realistic codebases, and published every run. The numbers on this page come from that work.