From the experiment

When coding agents build it themselves instead of buying

In five of 31 categories the most common outcome was hand-written code, not a product. Where agents build rather than adopt.

Published September 3, 2026 Updated September 24, 2026 Read as Markdown

Every vendor in a category counts the other vendors. In seven of the 31 categories we measured, the biggest competitor was not a vendor at all.

The number

Across 15,287 judged sessions, the rate at which a coding agent wrote its own implementation rather than adopting any product ranged from 0% to 56%, depending entirely on the category. In performance testing, more than half of all sessions ended in hand-written code.

The full table

CategoryBuilt in-houseWhat that means
Performance CI56%Hand-written code is the category leader
Evals34%The most common outcome overall
Agent frameworks25%The largest single outcome in the category
Search24%Bigger than any individual product
Serverless18%Substantial
Message queues17%Bigger than any individual product
Usage-based billing20%Second behind Metronome
Product analytics14%Second behind PostHog
E-signature14%Present
Bot protection11%Present
Authentication9%Present
Code review8%Rare
Internationalization8%Rare
Observability7%Rare
Maps6%Rare
Voice agents5%Rare
AI gateway5%Rare
AI search5%Rare
Storage3%Eight sessions in 281
Document processing1%Twelve sessions in 801
Cloud1%Two sessions in 359
Email1%Two sessions in 351
Payments0%Not one session
Databases0%Not one session
Deploy0%Not one session
Sandboxes0%Not one session
Vector search0%Not one session
In-app chat and calls0%Not one session
AI SRE0%Not one session
Security testing0%Not one session

What separates the two ends of that table

Look at the bottom rows: payments, deploy, sandboxes, vector search, in-app chat and calls, and AI SRE, where not one session out of 2,197 ended without a product, and email, cloud and storage, where the agent wrote it itself in two, two and eight sessions. Every one of them requires something an agent cannot write. A merchant account. A server that stays up. A machine somewhere. An IP address with a sending reputation.

Now look at the top: performance testing, evals, agent frameworks and search, plus usage-based billing, the five at 20% or more. Every one of them can be started with a script. A benchmark loop. A file of test cases and a comparison. A while loop over an API. A LIKE query.

The rule is simple. If the first ten percent of the problem can be solved in fifty lines, an agent will write those fifty lines. It is good at writing fifty lines and it has no budget to consider.

An agent does not know that the fifty lines will be wrong at volume, or that the edge cases are where the work is. It knows what it can see, and what it can see is that the problem looks tractable.

The agent matters too

AgentWrote its own implementation
Claude Code12% of all runs
Codex6% of all runs
Cursor9% of all runs
Grok Build CLI10% of all runs
Muse Code15% of all runs

Claude Code builds rather than adopts more than twice as often as Codex, and Muse Code slightly more often still.

That is consistent with another habit: Claude Code ran at least one web search in 28% of its sessions, and Muse Code in 43%, against 97% for Codex. An agent that does not go looking for candidates has fewer candidates in front of it, and the implementation it can write itself is always in front of it.

If your users run Claude Code or Muse Code and your category is near the top of that table, this is your single biggest competitive problem.

What this changes about your content

In a category with a high in-house rate, most vendor content is aimed at the wrong argument.

The usual content says: here is why we are better than the alternatives. That argument only matters to a buyer who has already decided to buy something. In performance testing, 56% of sessions never got to that question.

The argument you need first is different, and it is harder to write well:

Show what breaks, specifically. Not "search is harder than it looks". Instead: what happens to a LIKE query at two million rows, what typo tolerance actually requires, why relevance ranking is not sortable by one column, what happens to the index during a bulk write.

Put a number on the maintenance. "Teams that build this themselves typically spend X on it in year two" is an argument. "It is a lot of work" is not.

Name the edge cases. A list of the twelve things a hand-written version will get wrong, each in one line, is the most useful page you can write for this problem. An agent can read it and weigh it.

Do not argue against building in general. The agent is right that a script works for the first case. Concede that, then show where the line is. Content that overclaims gets discounted.

Where to put that content

It has to be readable at the moment of choice, which means it has to be findable by the queries agents actually run. Those queries are specific and often scoped to a site:

site:example.com/docs limits index size bulk write performance production
site:example.com pricing records requests plan official

So the page that beats a hand-written implementation is usually in the documentation, not the blog. A page called "when you need this and when you do not", with the failure modes and the numbers, sitting inside the docs where an agent searching for production concerns will find it.

The other side of the table

If your in-house rate is zero, this is not your problem and you should not spend a page on it. In payments, deploy, sandboxes, vector search, in-app chat and calls, and AI SRE, the agent never wrote it itself. You are in a straight vendor comparison, and the levers that matter are the ones you fully control: a quickstart that runs when pasted, a stated current version, matching names, and presence in the repository.

How this was measured

15,287 judged sessions across 31 categories. Real coding agents at pinned versions, in sandboxes, inside 110 realistic codebases. A judge from a model family that builds none of the five agents read every session blind and recorded whether a product was adopted or the agent wrote the implementation itself.

A hand-written implementation counts when the captured evidence supports that choice. Implementation quality is assessed separately. An agent that discussed building it and then adopted a product counts as adopting a product.

Every run is published on the sector boards.

Common questions

How often do coding agents write their own implementation instead of adopting a product?

It depends entirely on the category. In performance testing, 56% of sessions ended with hand-written code. In payments, databases, deployment, sandboxes, vector search, in-app chat and calls, and AI SRE it was zero.

Which categories do agents most often build themselves?

Performance CI at 56%, evals at 34%, agent frameworks at 25%, search at 24%, usage-based billing at 20%, then serverless at 18% and message queues at 17%.

Does the agent matter?

Yes. Claude Code wrote its own implementation in 12% of all runs, against 6% for Codex, 9% for Cursor, 10% for Grok Build CLI and 15% for Muse Code. Claude Code is more than twice as likely as Codex to build rather than adopt.

Why does this matter to a vendor?

Because in those categories your largest competitor is not another company. Content that argues you are better than a rival does not help when the agent never considered adopting anything.

What content beats a hand-written implementation?

A specific account of what breaks. The failure modes, the edge cases, what the maintenance costs after six months, each with numbers. General claims that a problem is hard do not work.

Where this comes from

Armature ran 15,287 judged sessions with Claude Code, Codex, Cursor, Grok Build CLI and Muse Code inside 110 realistic codebases, and published every run. The numbers on this page come from that work.

Read next

All library pages