AI search: which services coding agents choose

Key learnings

• Codex picked its own maker's OpenAI web search in about 12% of its runs.
• The simulated user, who approves each plan before code is written, sent the agent back at least once in 306 runs.
• The agents wrote it themselves in 32 runs, about 5%.

The ranking639 runs

By agent, by persona, by wording

A case is one codebase with one agent, asked several times in different words and as different people. 55 of 60 cases did not hold to a single service.

How this was measured

Every number on this page comes from a controlled experiment. We took 12 small applications, asked 5 coding agents (Codex, Claude Code, Cursor, Muse Code (Muse Spark 1.3), Grok Build CLI (Grok 4.7)) to get live web information into each of them, in several wordings and as a junior developer and enterprise team and senior engineer and vibe coder, and let the agent choose the product. Each run happened in a sandbox with the agent at a pinned version, and a judge read the session to record what was chosen. That is 639 runs. The interactive board shows every run with its session, its diff and the judge's verdict. A simulated user stood in for the owner of the codebase: it read the agent's plan and had to approve it before any code was written; it sent the agent back at least once in 306 runs. Read the methodology and the publications.

If you sell in this sector: what these numbers mean for a vendor.

Other sectors

• Agent sandboxes
• Observability
• AI SRE
• Payments
• Deploy
• Auth
• Email providers
• Product analytics
• Databases
• File storage
• LLM evals & observability
• Voice Agents
• Serverless functions
• Cloud
• AI gateway
• Bot protection
• Search
• Agent frameworks
• Performance in CI
• Document processing & OCR
• Usage-based billing
• Internationalization
• Security testing
• Maps
• Message queues
• Code review
• E-signature
• Security platforms
• In-app chat & calls
• Vector search