# The dataset: which software coding agents choose

> 235 ranking rows across 18 categories from 5,292 judged sessions, as CSV and JSON, under CC BY 4.0. Cite it, load it, feed it to an agent.

Source: https://armature.tech/library/the-dataset
Published: 2026-09-03
Publisher: Armature, Inc. (https://armature.tech)

---

Everything the sector pages show, in a shape you can load rather than read.

## The files

| File | What it holds | Size |
| --- | --- | --- |
| [agent-leaderboards.csv](/library/agent-leaderboards.csv) | One row per product per category: rank, wins, share, category session count | About 15 KB |
| [agent-leaderboards.json](/library/agent-leaderboards.json) | The same, plus the per-agent and per-persona breakdown for every category | About 86 KB |

> **235 ranking rows across 18 categories**, drawn from 5,292 judged sessions with Claude Code, Codex and Cursor inside 51 realistic codebases. Released under **CC BY 4.0**: use it, publish from it, build on it, and link back.

## The CSV columns

```
sector          the category slug, for example "mail"
sector_title    the category as it appears on the site
rank            1 is the most chosen
product         the product name
domain          the product's site, empty for outcomes that are not products
wins            sessions where an agent chose this and wrote working code
share_percent   wins as a percentage of the category's sessions
sector_runs     total judged sessions in the category
```

Some rows are outcomes rather than products. "Built in-house" means the agent wrote the implementation itself and adopted nothing. In performance testing that outcome is the category leader at 52%.

## The JSON shape

Each category carries the ranking plus two breakdowns that the CSV cannot hold cleanly:

```json
{
  "sector": "mail",
  "title": "Email providers",
  "runs": 208,
  "codebases": 9,
  "ranking": [
    { "rank": 1, "product": "Resend", "domain": "resend.com", "wins": 74, "share_percent": 36 }
  ],
  "by_agent": [
    { "agent": "Claude Code", "model": "Claude Opus 5", "runs": 65,
      "most_chosen": { "product": "Postmark", "wins": 23 },
      "second": { "product": "Resend", "wins": 20 } }
  ],
  "by_persona": [
    { "persona": "Enterprise team", "runs": 44,
      "most_chosen": { "product": "Azure Communication Services Email", "wins": 22 } }
  ]
}
```

The per-agent and per-persona splits are the interesting part, because that is where the averages stop being true. The three agents named different category leaders in 10 of the 18 categories, and the leader changed with who was asking in 14 of 18.

## What is not in it

Run traces, transcripts, prompt text and repository names.

Those are the raw material of the experiment. They are served only inside the interactive board, behind a viewer gate, and they are not in this export by construction: the export is built from an allow-list of fields rather than by removing things from a full record.

Every field in these files is already visible on a public page. If you need to check one, the [sector pages](/leaderboards/sectors) carry the same numbers with the working shown.

## How to cite it

> Armature agent leaderboards. https://armature.tech/library/the-dataset. Generated 2026-09-02, 5,292 judged sessions across 18 categories.

The `generated` field and `total_runs` are in the JSON, so a citation can pin the version you used.

## Things worth doing with it

**Check a claim.** Every number in the library and on the sector pages comes from this. If something reads as too neat, the file is right there.

**Compare a category's concentration.** Sort by the leader's share. Payments at 88% and agent frameworks at 14% are two completely different markets, and the difference decides what a vendor in each should do.

**Find the categories where no vendor wins.** Filter for the "Built in-house" rows. Performance testing 52%, evals 29%, agent frameworks 25%, search 23%, product analytics 23%.

**Look at the per-agent split before drawing a conclusion.** A blended number describes a market that does not exist in half the categories.

## The caveats that come with it

**It is three agents.** Claude Code, Codex and Cursor. Not Gemini CLI, Windsurf, Cline, Aider or the rest. See [what about the other coding agents](/library/what-about-the-other-coding-agents).

**A judge produced the verdicts, and it undercounts.** We track that as the judge gap: products a transcript mentions that the judge did not list. In observability, 155 of 360 sessions have at least one. Treat the counts as a floor.

**Categories differ in coverage.** Bot protection ran on five codebases because the others had no public form worth protecting. `codebases` in the JSON tells you how many each category used.

**It is a snapshot.** These tools update, models update, and a number from six months ago is a historical fact rather than a current one.

## The method

[How we measured this](/library/how-we-measured-this) covers all of it: the codebases and their lint rules, the open seams, the prompt families, the runner and its quarantine, the simulated project owner and the check showing it made leaders smaller rather than larger, the judge and why it comes from a model family that builds none of the three agents, and the eight decoy products that do not exist.

## Common questions

### What is in the dataset?

235 ranking rows across 18 categories: the product, its rank, how many sessions it won, its share, and the category's session count. The JSON adds the per-agent and per-persona breakdown for each category.

### What licence is it under?

CC BY 4.0. Use it, publish from it, build on it. Attribute Armature and link back to the source page.

### What is not in the dataset?

Run traces, transcripts, prompt text and repository names. Those are the raw material of the experiment and they are served only inside the interactive board. Every field in the export is already visible on a public page.

### How often is it updated?

It is regenerated whenever the board data is, from the same source the public pages read. The JSON carries a generated date and the total run count so you can tell which version you have.

### Can I use it in a paper or a post?

Yes. That is what it is for. Cite it as the Armature agent leaderboards with the source URL and the generated date from the file.

## Read next

- [How we measured this](https://armature.tech/library/how-we-measured-this) (Markdown: https://armature.tech/library/how-we-measured-this.md)
- [Do Claude Code, Codex and Cursor pick the same tools?](https://armature.tech/library/do-claude-code-and-codex-agree) (Markdown: https://armature.tech/library/do-claude-code-and-codex-agree.md)
- [When coding agents build it themselves instead of buying](https://armature.tech/library/what-coding-agents-build-in-house) (Markdown: https://armature.tech/library/what-coding-agents-build-in-house.md)
- [The products coding agents consider and never choose](https://armature.tech/library/products-coding-agents-consider-and-never-choose) (Markdown: https://armature.tech/library/products-coding-agents-consider-and-never-choose.md)

---

Armature helps software products get discovered and used by coding agents.
Service: https://armature.tech/discoverability · Results: https://armature.tech/leaderboards/sectors · Contact: contact@armature.tech