AI visibility for developer tools
AI visibility dashboards count brand mentions in chat answers. For a developer tool the number that pays rent is install share.
Every AI visibility tool works the same way. It sends a list of prompts to a list of models on a schedule, reads the replies, and counts how often your brand appears, in what position, and in what tone.
That is a real measurement of a real thing. For a developer tool, it is also the smaller half of the story.
What AI visibility measures
AI visibility is how often and how favourably AI systems mention a brand when answering questions. It is measured by sending a fixed prompt set to chat models on a schedule and counting brand mentions, position and sentiment in the replies.
The standard metrics:
| Metric | What it counts |
|---|---|
| Visibility | Percentage of tracked prompts where the brand appears |
| Position | Where in the answer the brand appears |
| Share of voice | Your mentions as a share of all brand mentions |
| Sentiment | Whether the mention is positive, neutral or negative |
| Citations | Which of your pages the model linked to |
All of that is worth knowing. It tells you what a person hears when they ask an assistant about your category.
The number that ends in revenue
For a developer tool, the sentence in a chat window is not where the money is. The money is in the install.
Install share is the percentage of sessions, in a category, where a coding agent picked your product and wrote working integration code. It is the agent-era equivalent of market share, and it is measured by running real agents on real tasks.
The difference is not academic. Here is what install share looks like when you measure it properly, from 5,292 judged sessions:
| Category | Leader | Install share |
|---|---|---|
| Payments | Stripe | 88% |
| Databases | Neon | 66% |
| Cloud | AWS | 62% |
| Product analytics | PostHog | 53% |
| Observability | Sentry | 37% |
| Resend | 36% | |
| Authentication | WorkOS AuthKit | 26% |
| Agent frameworks | Vercel AI SDK | 14% |
A payments company at 88% and an agent framework at 14% are in completely different situations, and no mention count would have told them apart.
Why visibility and install share come apart
Three reasons, all of them structural.
The repository is not in the prompt. A visibility tool asks a model a question in an empty context. A coding agent asks the same question inside a codebase that already has opinions. We ran the same database request on JavaScript codebases and one product won 111 times out of 111. On TypeScript codebases the same product won 48 out of 132. Same question, different file extension, different winner.
The agents disagree with each other. Claude Code, Codex and Cursor picked different category leaders in 9 of the 18 categories we measured. A single visibility score averages that away.
Some agents barely search. On decision tasks, Claude Code ran a web search in 1.6% of runs. Codex ran one in 53%. Web content, which is what visibility tools measure the effect of, is live in half of one agent's sessions and almost none of the other's.
The person asking changes the answer too
We wrote every request as one of four kinds of person: a vibe coder describing symptoms, a junior developer naming the category, a senior engineer adding constraints, and a senior engineer at a large company adding procurement rules.
The category leader changed with the person in 14 of 18 categories.
| Category | Vibe coder picks | Enterprise team picks |
|---|---|---|
| Storage | Amazon S3 | Azure Blob Storage |
| Authentication | Google Identity | Microsoft Entra ID |
| Observability | Better Stack | Grafana |
| Serverless | Cloudflare Workers | Azure Functions |
If your visibility dashboard reports one number, it is reporting an average across markets that behave differently. The average is true and it is not actionable.
What a developer tool should actually track
Keep the visibility tracking. Add the part that pays.
| Track | Why | How |
|---|---|---|
| Install share by category | The headline number | Real agent runs, hundreds per category |
| Install share by agent | They disagree half the time | Split by Claude Code, Codex, Cursor |
| Install share by language | The repository is the largest lever | Split by the languages your users use |
| Install share by persona | The leader changes in 14 of 18 categories | Vibe coder, junior, senior, enterprise |
| Install share with a competitor present | Incumbency is close to decisive | Run with and without the rival installed |
| In-house rate | Your real competitor may be a script | Count sessions where the agent built it itself |
| Failure reasons | Where you lose is fixable | Read the transcripts of the losses |
| Chat visibility | The other half of the market | Any prompt monitoring tool |
The in-house rate deserves its own line
In five of the eighteen categories we measured, the most common outcome was that the agent wrote the thing itself rather than install anything.
| Category | Built in-house |
|---|---|
| Performance CI | 52% |
| Evals | 29% |
| Agent frameworks | 25% |
| Search | 23% |
| Product analytics | 23% |
In performance CI, no vendor lost to another vendor. Every vendor lost to a hand-written script.
Meanwhile in payments, databases, storage, deploy, sandboxes and email, the in-house rate is zero. The agent never considers writing it.
That single number tells you what your content has to argue. If your in-house rate is high, you must first convince the agent the problem is harder than it looks. If it is zero, you are in a straight vendor fight.
Getting started without a large budget
You can approximate this yourself before buying anything.
- Take three repositories that look like your users' projects. Real lock files, real structure.
- Write ten requests in the words your users use, not the category name.
- Run them with Claude Code and Codex, five times each, in a sandbox.
- Record what got installed.
- Read the ten sessions where you lost.
That is 300 runs. It costs a few hundred dollars in tokens and a weekend. It will tell you more than a year of mention counting, because you will read the actual reasons you lost, in the agent's own words.
Then decide whether the pattern is worth measuring continuously.
Common questions
What is AI visibility?
AI visibility is how often and how favourably AI systems mention a brand when they answer questions. It is usually measured by sending a fixed set of prompts to chat models on a schedule and counting brand mentions, position and sentiment in the replies.
Is AI visibility a useful metric for a developer tool?
It is useful and incomplete. It tells you what a chat assistant says about you to a person. It says nothing about whether a coding agent installs you into a codebase, which for many developer tools is the larger and more valuable surface.
What should a developer tool measure instead?
Install share: of all sessions where an agent had to solve a problem your product solves, the percentage where it picked your product and wrote working integration code. Track it split by agent, by language, by persona and by whether a competitor was already installed.
How do you measure install share?
Run real coding agents on real tasks inside repositories that resemble your users' projects, and record what gets installed. You need enough replicates to see past noise, because the same agent on the same repository disagrees with itself about a quarter of the time.
Do AI visibility scores predict install share?
No. A product can be widely cited in chat answers and lose almost every session inside a repository where a competitor is already wired in. The two are measured on different surfaces and move for different reasons.
Where this comes from
Armature ran 5,292 judged sessions with Claude Code, Codex and Cursor inside 51 realistic codebases, and published every run. The numbers on this page come from that work.