Prompt monitoring
Sending a fixed list of prompts to AI assistants on a schedule and counting brand mentions. How every AI visibility tool works, and what the method cannot see.
Prompt monitoring is sending a fixed list of prompts to AI assistants at a regular interval and parsing the replies for brand mentions, position, sentiment and cited sources. It is the method behind almost every AI visibility product.
How it works
- Pick a prompt set. "Best hosted Postgres." "Cheapest transactional email API."
- Send them to several assistants, usually daily.
- Parse each reply for brand names, position, tone and links.
- Chart the result over time.
Some products supplement this with real user session data, which is a stronger signal because it reflects what people actually asked rather than what you guessed they would ask.
The strengths
Cheap. It is API calls. That is why these tools can run continuously across many engines for a low monthly price.
Fast to set up. An hour, and you have a baseline.
Directly actionable on citations. Knowing which of your pages a model drew on tells you what to write more of.
The limits
The score is relative to your prompt set. Change the prompts and the number changes. It is useful for tracking yourself over time and comparing named competitors inside the same set. It is meaningless as an absolute figure, and it cannot be compared with a number from another tool.
Prompt sets are usually too small. Entry tiers often cap at 50 prompts. A category with four distinct use cases and four kinds of buyer needs more than that to be representative.
Symptom wording gets missed. Most tracked prompts name the category. Real users describe symptoms. "People keep signing up with fake emails" and "recommend a bot protection service" do not produce the same answer, and only one of them is usually in the prompt set.
The test contains no repository. This is the structural limit, and for a developer tool it is the important one.
What the method structurally cannot see
A prompt sent to a chat model is missing the things that decide a coding agent's choice.
| Missing from the test | Measured effect |
|---|---|
| The repository | Same request: one winner in 111 of 111 JavaScript sessions, 48 of 132 in TypeScript |
| Installed software | A written instruction lost 2 times out of 3 once a rival library was present |
| The agent's own habits | The three coding agents disagreed on the leader in 10 of 18 categories |
| Running the code | Products lose at the integration step for reasons a chat answer never surfaces |
None of that is a flaw in the tools. It is a property of the test.
The reasonable use
Run prompt monitoring for the chat surface. It is cheap and it tells you what a person hears.
Measure install share separately for the surface where agents install software. They do not predict each other.
Common questions
What is prompt monitoring?
Sending a fixed list of prompts to AI assistants at a regular interval and parsing the replies for brand mentions, position, sentiment and cited sources. It is the method behind almost every AI visibility product.
What are the limits of prompt monitoring?
The score depends entirely on the prompts you chose, so it cannot be compared across tools or companies. And the test contains no repository, no installed software and no code, so it cannot measure what a coding agent installs.
How many prompts should be tracked?
Enough to cover the real ways your category gets described, including symptom wording rather than only category names. Entry tiers of these tools often cap at 50 prompts, which is thin for a category with several distinct use cases.
Is prompt monitoring worth doing?
Yes, for what it measures. It is inexpensive, quick to set up, and knowing what an assistant says about you is worth knowing. It is not a proxy for adoption.
Where this comes from
Armature ran 5,292 judged sessions with Claude Code, Codex and Cursor inside 51 realistic codebases, and published every run. The numbers on this page come from that work.