Armature vs the Semrush AI Toolkit
The Semrush AI Toolkit tracks AI visibility inside a suite you may already have. Armature measures installs by coding agents. What each one is for.
Semrush and Ahrefs arrived at the same place from the same direction: a large SEO suite with an AI visibility module attached. If either is already in your budget, that module is your cheapest starting point.
The short answer
The Semrush AI Toolkit tracks brand presence and sentiment in AI answers across the major assistants, inside the wider Semrush suite. Armature measures whether a coding agent installs your product, by running Claude Code, Codex and Cursor on real tasks inside realistic repositories.
What the AI Toolkit does well
- It is bundled. If you pay for Semrush, this is a tab, not a purchase.
- Competitor framing is native. Semrush has always been built around comparing yourself with a named set of rivals, and that framing transfers well to share of voice in AI answers.
- It connects to the rest of the funnel data. Keyword positions, traffic estimates and AI mentions in one place.
- Sentiment as well as presence. Knowing you are mentioned dismissively is more useful than knowing you are mentioned.
Where it fits loosely
It is a marketing tool. For a developer tool, the pages that decide adoption are the configuration reference, the limits page and the quickstart, and the queries that reach them look nothing like head terms:
site:example.com/docs index state management snapshots security plugin official documentation
site:example.com pricing records requests grow plan officialA suite built around keyword volume does not naturally surface that. Again, not a defect. A different job.
Side by side
| Semrush AI Toolkit | Armature | |
|---|---|---|
| Surface measured | Chat assistants answering people | Coding agents installing software |
| The test | Tracked prompts sent to chat models | A real agent in a real repository on a real task |
| Output | Presence, share of voice, sentiment | Install share, and the reason for each loss |
| Includes a codebase | No | Yes |
| Ends in running code | No | Yes |
| Built for | Content and brand marketing | Developer tools |
| Delivery | A module in an SEO suite | A service with a growth engineer, plus self-serve analytics and evals |
| Price | Included in the suite | From $5,000 per month |
The structural gap
Prompt monitoring, which is what every tool in this category does, sends a question to a chat model and counts brand names. The test is missing everything that decides a coding agent's choice.
| Missing | Measured effect |
|---|---|
| The repository | One winner in 111 of 111 JavaScript sessions; 48 of 132 in TypeScript |
| Installed software | A written instruction lost 2 of 3 once a rival was present |
| The specific agent | The three agents disagreed on the leader in 10 of 18 categories |
| Running the code | Products lose at the integration step for reasons no chat answer surfaces |
And one number that undercuts the model entirely for some vendors: Claude Code ran a web search in 1.6% of decision runs. Whatever you publish is absent from 98% of its decisions.
Which to use
The AI Toolkit if Semrush is already paid for and you want a chat-surface baseline today.
Armature if agents install products in your category and you need to know your install share and why you lose.
Both if your category is researched by people and implemented by agents. Authentication, databases and observability all behave that way.
Before you buy anything
Take three repositories that look like your users' projects. Write ten requests the way your users describe the problem. Run each five times with Claude Code and Codex. Record what got installed, then read the losses.
That is 300 runs, a weekend, and a few hundred dollars. It tells you which of these two purchases is the one you actually need.
Common questions
What is the Semrush AI Toolkit?
An AI visibility module inside the Semrush suite. It tracks brand presence and sentiment in AI answers across the major assistants, alongside the keyword, competitor and site audit tooling, on the suite subscription.
Is the Semrush AI Toolkit good for developer tools?
It covers the chat surface competently and it is the cheapest option if Semrush is already in your budget. It was built for content and brand marketing, so it fits a documentation-led product more loosely.
What does Armature measure that the AI Toolkit cannot?
Install share: whether a coding agent chose your product and wrote working integration code inside a real codebase. The AI Toolkit's test is a prompt to a chat model, which contains no repository and produces no code.
Do I need both?
If people research your category in a chat window and agents implement it, yes. They measure different surfaces and neither predicts the other.
Where this comes from
Armature ran 5,292 judged sessions with Claude Code, Codex and Cursor inside 51 realistic codebases, and published every run. The numbers on this page come from that work.