Guides

What about Gemini CLI, Windsurf, Cline and the others?

We measured five agents, two of them on thinner samples, and not the rest. Why those, what we think transfers, and what we will not claim without data.

Published September 3, 2026 Updated September 24, 2026 Read as Markdown

We measured Claude Code, Codex, Cursor, Grok Build CLI and Muse Code. We did not measure Gemini CLI, Windsurf, Cline, Aider, OpenCode, Amp or Devin. This page is about that gap, because pretending it is not there would make everything else less trustworthy.

Why five

Two reasons, both boring.

Cost. One session is a dollar to fifteen in tokens and about nine minutes of real agent time. Adding a simulated project owner in the loop multiplied our bill by roughly four, because every run gained a recommendation turn, up to two questions, and then the implementation. The published set is 2,767 hours of agent time.

Statistical power. The same agent, on the same repository, with the same request, disagrees with itself more than a quarter of the time. So a category needs hundreds of sessions, not tens. Spreading the same budget across seven agents would have produced seven numbers, none of which we could stand behind.

Claude Code, Codex and Cursor each have more than 3,300 published sessions. Grok Build CLI has 853. Muse Code now has 2,530 in the aggregate: three passes over the same input panel, plus 18 runs on two Python voice-agent codebases, 112 in security testing and 28 in security platforms. Both cover 27 of the 31 aggregate categories.

Muse's dedicated board has 2,749 captured choices across 33 categories. The aggregate uses a narrower sector set. Repeating the same inputs shows how stable the choices are. It does not add new repositories or prompt cases. Read each sector's sample size and repository spread before comparing product shares.

Grok Build CLI wrote its own implementation in 10% of its aggregate runs. Muse Code did so in 15%.

What almost certainly transfers

The structural findings, because they are about how agents work rather than about which agent it is.

The repository dominates. The same email request went to Resend in 38 of 51 sessions on TypeScript codebases and to SendGrid in 16 of 17 on a Python codebase. Every agent reads the repository first. There is no reason a different harness would ignore what is installed.

An incumbent beats an instruction. A sentence in a repository file naming a preferred product was followed 16 times out of 18, and lost 2 times out of 3 once a rival library was installed. This is a reasoning pattern, not a product quirk.

A quickstart that does not run loses the session. Every agent has to make the code work. A missing environment variable stops all of them.

Names have to match. An agent that cannot map your product name to your package name guesses, in any harness.

Who is asking changes the answer. The category leader changed with the persona in 24 of 31 categories. Part of that is the codebases each persona ran on. The rest is the model reading constraints, and every agent runs a model that does that.

What almost certainly does not transfer

The rankings.

Claude Code, Codex and Cursor did not all name the same category leader in 18 of the 29 categories all three ran. Email: Postmark, Resend, Resend. Sandboxes: E2B, Modal, E2B. Authentication: WorkOS AuthKit, Auth0, WorkOS AuthKit.

If three agents disagree that much with each other, a fourth is not safely predicted by any of them. Anyone telling you the ranking generalises to an agent they did not run is guessing.

What we would expect, stated as expectations

Worth separating from the findings. These are informed guesses and we would not put them on a chart.

Editor-based agents should behave more like Cursor. Cline and Windsurf work with the project open and indexed, as Cursor does. In our data Cursor leaned towards options that drop into an existing project with the least new infrastructure. That mechanism should carry.

Command line agents should behave more like Claude Code and Codex, and the size of the split between them will probably come down to how often each one searches. That single number varied more than threefold across the command line agents we measured: at least one web search in 28% of sessions for Claude Code, 43% for Muse Code, 90% for Grok Build CLI and 97% for Codex.

An agent whose maker sells products in a category is worth watching in that category specifically. We now see it in several categories, most clearly code review, where Claude Code and Cursor each put its own maker's reviewer first and Codex put its own second. It does not show everywhere a maker sells something. See self-preference.

Model family should matter less than the harness. The one direct test we have says both matter. Grok 4.7 ran in both Cursor and Grok Build CLI, and on the same requests it made the same choice 64% of the time, against about 73% when one agent repeats a request. Agents that differ in both harness and model family agree about half the time. And inside Claude Code, the web search rate ran from 6% of sessions with Claude Sonnet 5 to 37% with Claude Fable 5.1, leaving aside the 12 security testing sessions Claude Opus 4.8 took over from Claude Opus 5.5.

The first two are still hypotheses. We will replace them with numbers as we run more.

What a vendor should do about the gap

Find out which agents your users actually run, and measure that one.

If most of your users are on an agent nobody has published data about, your position in our tables tells you something about the mechanism and nothing reliable about your number. Running your own measurement is the only way to close that, and it is cheaper than it sounds: three repositories, ten requests written as symptoms, five runs each. See how to measure install share.

What we will not claim

That the five agents we measured represent the market. They represent a large part of it and we have no basis for saying how large.

That a product leading here leads everywhere. 18 of the 29 categories all three ran say otherwise within Claude Code, Codex and Cursor alone.

That an agent we did not run behaves like one we did. It might. We have not looked.

Common questions

Which coding agents did you measure?

Claude Code, Codex and Cursor, with more than 3,300 sessions each, and Grok Build CLI and Muse Code, with 853 and 2,530, at pinned versions, across 15,287 judged sessions. Gemini CLI, Windsurf, Cline, Aider, OpenCode, Amp and Devin were not in the published set.

Why only five agents?

Cost and statistical power. Each session is minutes of real agent time and a category needs hundreds of sessions because the same agent disagrees with itself more than a quarter of the time. Grok Build CLI and Muse Code have a few dozen sessions per category, enough to see their overall habits and not enough to rank products inside a category.

Do the findings transfer to other agents?

The structural ones very likely do: the repository dominating the choice, an installed rival beating a written instruction, a quickstart that does not run losing the session. The specific rankings almost certainly do not, because Claude Code, Codex and Cursor did not all name the same leader in 18 of the 29 categories all three ran.

Should a vendor measure the agent its users actually run?

Yes. Claude Code, Codex and Cursor did not all name the same category leader in 18 of the 29 categories all three ran, so an agent we did not measure could easily behave differently again.

Where this comes from

Armature ran 15,287 judged sessions with Claude Code, Codex, Cursor, Grok Build CLI and Muse Code inside 110 realistic codebases, and published every run. The numbers on this page come from that work.

Read next

All library pages