AI visibility for developer tools
AI visibility dashboards count brand mentions in chat answers. For a developer tool the number that pays rent is install share.
Every AI visibility tool works the same way. It sends a list of prompts to a list of models on a schedule, reads the replies, and counts how often your brand appears, in what position, and in what tone.
That is a real measurement of a real thing. For a developer tool, it is also the smaller half of the story.
What AI visibility measures
AI visibility is how often and how favourably AI systems mention a brand when answering questions. It is measured by sending a fixed prompt set to chat models on a schedule and counting brand mentions, position and sentiment in the replies.
The standard metrics:
| Metric | What it counts |
|---|---|
| Visibility | Percentage of tracked prompts where the brand appears |
| Position | Where in the answer the brand appears |
| Share of voice | Your mentions as a share of all brand mentions |
| Sentiment | Whether the mention is positive, neutral or negative |
| Citations | Which of your pages the model linked to |
All of that is worth knowing. It tells you what a person hears when they ask an assistant about your category.
The number that ends in revenue
For a developer tool, the sentence in a chat window is not where the money is. The money is in the install.
Install share is the percentage of sessions, in a category, where a coding agent picked your product and wrote working integration code. It is the agent-era equivalent of market share, and it is measured by running real agents on real tasks.
The difference is not academic. Here is what install share looks like when you measure it properly, from 15,287 judged sessions:
| Category | Leader | Install share |
|---|---|---|
| Payments | Stripe | 88% |
| Databases | Neon | 46% |
| Cloud | AWS | 63% |
| Product analytics | PostHog | 56% |
| Observability | Sentry | 36% |
| Resend | 33% | |
| Authentication | Auth0 | 25% |
| Agent frameworks | Vercel AI SDK | 14% |
A payments company at 88% and an agent framework at 14% are in completely different situations, and no mention count would have told them apart.
Why visibility and install share come apart
Three reasons, all of them structural.
The repository is not in the prompt. A visibility tool asks a model a question in an empty context. A coding agent asks the same question inside a codebase that already has opinions. We asked agents to make an app's data persist, and on apps with their own server one product won 280 times out of 359. On apps with no server of their own, the same product won none of 142, and a different one won 111. Same category, different codebase, different result.
The agents disagree with each other. Claude Code, Codex and Cursor did not all pick the same category leader in 18 of the 29 categories all three ran. A single visibility score averages that away.
Some agents search far less than others. Claude Code ran at least one web search in 28% of sessions. Codex did in 97%. Web content, which is what visibility tools measure the effect of, is live in almost every Codex session and in about a third of Claude Code's.
The person asking changes the answer too
We wrote every request as one of four kinds of person: a vibe coder describing symptoms, a junior developer naming the category, a senior engineer adding constraints, and a senior engineer at a large company adding procurement rules.
The category leader changed with the person in 24 of 31 categories.
| Category | Vibe coder picks | Enterprise team picks |
|---|---|---|
| Storage | Amazon S3 | Azure Blob Storage and Google Cloud Storage, tied |
| Authentication | Google Identity | Microsoft Entra ID |
| Observability | Better Stack | Grafana |
| Serverless | Cloudflare Workers | Azure Functions |
If your visibility dashboard reports one number, it is reporting an average across markets that behave differently. The average is true and it is not actionable.
What a developer tool should actually track
Keep the visibility tracking. Add the part that pays.
| Track | Why | How |
|---|---|---|
| Install share by category | The headline number | Real agent runs, hundreds per category |
| Install share by agent | They disagree half the time | Split by Claude Code, Codex, Cursor |
| Install share by language | The repository is the largest lever | Split by the languages your users use |
| Install share by persona | The leader changes in 24 of 31 categories | Vibe coder, junior, senior, enterprise |
| Install share with a competitor present | Incumbency is close to decisive | Run with and without the rival installed |
| In-house rate | Your real competitor may be a script | Count sessions where the agent built it itself |
| Failure reasons | Where you lose is fixable | Read the transcripts of the losses |
| Chat visibility | The other half of the market | Any prompt monitoring tool |
The in-house rate deserves its own line
In five of the 31 categories we measured, the most common outcome was that the agent wrote the thing itself rather than install anything. In evals, building it in-house came a close second.
| Category | Built in-house |
|---|---|
| Performance CI | 56% |
| Evals | 34% |
| Agent frameworks | 25% |
| Search | 24% |
| Message queues | 17% |
In performance CI, the main rival was not another vendor. It was a hand-written script, which won 56% of sessions against 8% for the leading tool.
Meanwhile in payments, deploy and sandboxes, the in-house rate is zero. The agent never builds it itself.
That single number tells you what your content has to argue. If your in-house rate is high, you must first convince the agent the problem is harder than it looks. If it is zero, you are in a straight vendor fight.
Getting started without a large budget
You can approximate this yourself before buying anything.
- Take three repositories that look like your users' projects. Real lock files, real structure.
- Write ten requests in the words your users use, not the category name.
- Run them with Claude Code and Codex, five times each, in a sandbox.
- Record what got installed.
- Read the ten sessions where you lost.
That is 300 runs. It costs a few hundred dollars in tokens and a weekend. It will tell you more than a year of mention counting, because you will read the actual reasons you lost, in the agent's own words.
Then decide whether the pattern is worth measuring continuously.
Common questions
What is AI visibility?
AI visibility is how often and how favourably AI systems mention a brand when they answer questions. It is usually measured by sending a fixed set of prompts to chat models on a schedule and counting brand mentions, position and sentiment in the replies.
Is AI visibility a useful metric for a developer tool?
It is useful and incomplete. It tells you what a chat assistant says about you to a person. It says nothing about whether a coding agent installs you into a codebase, which for many developer tools is the larger and more valuable surface.
What should a developer tool measure instead?
Install share: of all sessions where an agent had to solve a problem your product solves, the percentage where it picked your product and wrote working integration code. Track it split by agent, by language, by persona and by whether a competitor was already installed.
How do you measure install share?
Run real coding agents on real tasks inside repositories that resemble your users' projects, and record what gets installed. You need enough replicates to see past noise, because the same agent on the same repository disagrees with itself more than a quarter of the time.
Do AI visibility scores predict install share?
No. A product can be widely cited in chat answers and lose almost every session inside a repository where a competitor is already wired in. The two are measured on different surfaces and move for different reasons.
Where this comes from
Armature ran 15,287 judged sessions with Claude Code, Codex, Cursor, Grok Build CLI and Muse Code inside 110 realistic codebases, and published every run. The numbers on this page come from that work.