What about Gemini CLI, Windsurf, Cline and the others?
We measured three agents and not the rest. Why those three, what we think transfers to the others, and what we are not willing to claim without data.
We measured Claude Code, Codex and Cursor. We did not measure Gemini CLI, Windsurf, Cline, Aider, OpenCode, Amp or Devin. This page is about that gap, because pretending it is not there would make everything else less trustworthy.
Why three
Two reasons, both boring.
Cost. One session is a dollar to fifteen in tokens and about nine minutes of real agent time. Adding a simulated project owner in the loop multiplied our bill by roughly four, because every run gained a recommendation turn, up to two questions, and then the implementation. The published set is 883 hours of agent time.
Statistical power. The same agent, on the same repository, with the same request, disagrees with itself about a quarter of the time. So a category needs hundreds of sessions, not tens. Spreading the same budget across seven agents would have produced seven numbers, none of which we could stand behind.
Three agents measured properly beats seven measured badly. That is the whole trade, and we would make it the same way again.
What almost certainly transfers
The structural findings, because they are about how agents work rather than about which agent it is.
The repository dominates. The same hosted database request produced one winner in 111 of 111 JavaScript sessions and 48 of 132 TypeScript sessions. Every agent reads the repository first. There is no reason a different harness would ignore what is installed.
An incumbent beats an instruction. A sentence in a repository file naming a preferred product was followed 16 times out of 18, and lost 2 times out of 3 once a rival library was installed. This is a reasoning pattern, not a product quirk.
A quickstart that does not run loses the session. Every agent has to make the code work. A missing environment variable stops all of them.
Names have to match. An agent that cannot map your product name to your package name guesses, in any harness.
Who is asking changes the answer. The category leader changed with the persona in 14 of 18 categories. That is the model reading constraints, and every agent runs a model that does that.
What almost certainly does not transfer
The rankings.
Claude Code, Codex and Cursor named different category leaders in 10 of the 18 categories we measured. Email: Postmark, Resend, Resend. Sandboxes: E2B, Modal, E2B. Authentication: WorkOS AuthKit, Auth0, Auth0.
If three agents disagree that much with each other, a fourth is not safely predicted by any of them. Anyone telling you the ranking generalises to an agent they did not run is guessing.
What we would expect, stated as expectations
Worth separating from the findings. These are informed guesses and we would not put them on a chart.
Editor-based agents should behave more like Cursor. Cline and Windsurf work with the project open and indexed, as Cursor does. In our data Cursor leaned towards options that drop into an existing project with the least new infrastructure. That mechanism should carry.
Command line agents should behave more like Claude Code and Codex, and the size of the split between them will probably come down to how often each one searches. That single number varied by a factor of thirty between the two we measured: 1.6% against 53% of decision runs.
An agent whose maker sells products in a category is worth watching in that category specifically. We saw one instance of it, in agent frameworks, and one instance is not a rule. See self-preference.
Model family should matter less than the harness. What decided behaviour in our data was mostly how the harness gathers evidence, not which model reasoned over it.
Each of those is a hypothesis. We will replace them with numbers as we run more.
What a vendor should do about the gap
Find out which agents your users actually run, and measure that one.
If most of your users are on an agent nobody has published data about, your position in our tables tells you something about the mechanism and nothing reliable about your number. Running your own measurement is the only way to close that, and it is cheaper than it sounds: three repositories, ten requests written as symptoms, five runs each. See how to measure install share.
What we will not claim
That our three agents represent the market. They represent a large part of it and we have no basis for saying how large.
That a product leading here leads everywhere. Ten of eighteen categories say otherwise within our own three.
That an agent we did not run behaves like one we did. It might. We have not looked.
Common questions
Which coding agents did you measure?
Claude Code, Codex and Cursor, at pinned versions, across 5,292 judged sessions. Gemini CLI, Windsurf, Cline, Aider, OpenCode, Amp and Devin were not in the published set.
Why only three agents?
Cost and statistical power. Each session is minutes of real agent time and a category needs hundreds of sessions because the same agent disagrees with itself about a quarter of the time. Three agents measured properly beats seven measured badly.
Do the findings transfer to other agents?
The structural ones very likely do: the repository dominating the choice, an installed rival beating a written instruction, a quickstart that does not run losing the session. The specific rankings almost certainly do not, because the three agents we did measure disagreed with each other in 10 of 18 categories.
Should a vendor measure the agent its users actually run?
Yes. The three agents we measured named different category leaders in 10 of 18 categories, so an agent we did not measure could easily behave differently again.
Where this comes from
Armature ran 5,292 judged sessions with Claude Code, Codex and Cursor inside 51 realistic codebases, and published every run. The numbers on this page come from that work.