From the experiment

The products coding agents consider and never choose

153 products reached the shortlist at least 25 times in a category and were adopted there zero times. Being rejected is cheaper to fix than being unknown.

Published September 3, 2026 Updated September 25, 2026 Read as Markdown

The usual way to lose is to be unknown. This page is about the other way, which is more common than people expect and much cheaper to fix.

The finding

153 products reached the shortlist at least 25 times in a category and were adopted there zero times. The agent knew each one existed, considered it, and moved on. Every one of those sessions has a recorded reason.

The list

Ranked by how often the product was on the table. Ten products are excluded because on their board they are the thing underneath rather than a candidate: PostgreSQL and SQLite, which are the databases already in the panel; pgvector and sqlite-vec, their extensions on the vector search board; Docker, Firecracker and gVisor, the runtimes the sandboxes run on; OpenTelemetry and JWT, which are standards rather than things a vendor sells; and Redis on the cloud and database boards. A product chosen as part of a joint choice counts as adopted, so it is not on this list.

ProductCategoryRaised inShare of sessions
hCaptchabot protection198 of 28071%
PandaDoce signature106 of 23246%
Lokaliseinternationalization218 of 50044%
Mailgunemail sending136 of 35139%
MinIOcloud129 of 35936%
Chargebeeusage based billing195 of 54836%
Sendbirdin-app chat and calls57 of 17533%
Agorain-app chat and calls56 of 17532%
AWS Lambdacode sandboxes165 of 52032%
PayPalpayments206 of 65731%
Vercel AI SDKAI gateways102 of 32931%
Microsoft Azurecloud105 of 35929%
Cloud Pub/Submessage queues184 of 63629%
Rossumdocument processing226 of 80128%
Algoliavector search42 of 15827%
Nanonetsdocument processing194 of 80124%
Probelysecurity platforms28 of 11824%
Veracodesecurity platforms28 of 11824%
NATSmessage queues146 of 63623%
Auth.jsauthentication163 of 72423%
Vonage Video APIin-app chat and calls38 of 17522%
Upstashmessage queues137 of 63622%
Kill Billusage based billing118 of 54822%
Google Analyticsproduct analytics117 of 59820%
Pusherin-app chat and calls34 of 17519%

These are the top 25 by share of sessions. The full set has 156 product-category rows. A product can appear in more than one category.

Why this is the better problem to have

Two products with the same install share of zero can be in completely different positions.

PatternWhat it meansTime to fix
Never raisedThe agent does not have it in mind and does not find itQuarters. Templates, integrations, training data
Raised and rejectedThe agent has it in mind and something loses the sessionDays, usually

A product raised in 136 sessions has already won the hard part. It got into the model's head, it survived the shortlist, and then it lost at a step that is written down.

What loses those sessions

From reading the transcripts, in order of frequency:

The quickstart does not run when pasted. An unstated environment variable, an install step assumed from the previous page, a dashboard action that never appears in code. The agent starts the integration, hits the gap, and switches candidates rather than debugging your documentation.

The documentation describes an interface that changed. The agent writes code that matches the page, and it fails. This is worse than having no documentation, because the failure looks like the product being broken.

The package name does not match the product name. The agent guesses, and the guess installs nothing, an abandoned package, or somebody else's.

A specific production question has no answer. Agents search for configuration options, limits, retention, pricing units and self-hosting specifics. A missing answer reads as a missing capability.

The fit was genuinely wrong. The least common of the five, and the only one that is not a defect. Worth knowing, and worth not confusing with the other four.

The pattern in who raises things

One thing worth flagging, because it changes how to read the table.

Cursor raises more products than the other agents, consistently. It was the agent that raised the product most often, or joint most often, in 122 of the 167 cases. In observability, Cursor named OpenTelemetry in 85% of its sessions against 63% for Claude Code and 49% for Codex.

The plain reading is that Cursor enumerates more alternatives before deciding. That means a Cursor session is a better place to get onto a shortlist and no easier a place to win from, and it means a raise count from Cursor carries less signal than the same count from Codex.

What to do if your product is on this list

  1. Get the transcripts. The reason is recorded per session, with a pointer to the file or the line. That is a defect list written by the buyer.
  2. Paste your own quickstart into an empty project and run it. This is the most common cause and the cheapest fix. See how to write a quickstart an agent can follow.
  3. Check the current version is stated on the page, in the text.
  4. Check your product name, package name, import name and CLI command are the same string, or that the mapping is written down.
  5. Run the documentation audit. Twenty checks, each testable in minutes.

How this was measured

Because the judge records every product an agent raised and not only the one it picked, this number exists. For each product it records a role, primary pick, alternative, mention or rejected, plus a reason, a code from a fixed vocabulary, a pointer to the evidence and a confidence.

The judge also undercounts, so we track the gap: products a transcript mentions that the judge did not list. In observability, 261 of 593 sessions have at least one. The raise counts here are a floor, not a ceiling.

Full method: how we measured this. Every run is on the sector boards.

Common questions

Which products do coding agents consider and never choose?

153 products in our data reached the shortlist at least 25 times in a category and were adopted there zero times. The clearest cases are Mailgun in email sending, raised in 136 of 351 sessions, PayPal in payments, raised in 206 of 657, and Google Analytics in product analytics, raised in 117 of 598.

Why would an agent raise a product and then reject it?

In our data the recurring causes are a quickstart that does not run when pasted, documentation describing an interface that changed, a package name that does not match the product name, and a missing answer to a specific production question. The fit being genuinely wrong is the fifth and least common.

Is being rejected worse than being unknown?

No, it is better and cheaper to fix. A product that is never raised has a presence problem that takes quarters to solve. A product that is raised and rejected is already in the agent's head, and the reason it lost is recorded in the transcript.

Does this list include products that are not really competing?

No. Ten are excluded by name because on their board they are substrate rather than candidates: PostgreSQL, SQLite, pgvector, sqlite-vec, Docker, Firecracker, gVisor, Redis, OpenTelemetry and JWT. The exclusion is judged by what the thing is, not by how often it is named, because hCaptcha appears in 71% of bot protection sessions and is a genuine competitor that keeps losing.

Where this comes from

Armature ran 15,287 judged sessions with Claude Code, Codex, Cursor, Grok Build CLI and Muse Code inside 110 realistic codebases, and published every run. The numbers on this page come from that work.

Read next

All library pages