Agent leaderboards / All sectors / AI gateway

AI gateway: which products coding agents choose

No product won more than about 26%, and Portkey was the leader.

329 runs8 apps5 agents4 personasupdated 2026-09-29

The interactive board, open on ai gateway. Open it full page · this page as Markdown

Read this leaderboard as textrankings, key learnings, method

Key learnings

We asked five coding agents to add an AI gateway to eight codebases. That came to 329 runs, each phrased in different words and asked as one of four different people. Cloudflare AI Gateway came second, Vercel AI Gateway third, and OpenRouter just behind it.

35 of 40

Wording changed the answer almost every time

A case here is one codebase with one agent, asked several times over in different words and as different people. Of 40 such cases, 35 did not settle on a single product.

37 of 81

The persona moved the order

Junior developers picked Vercel AI Gateway in 37 runs, more than any other product. Vibe coders put OpenRouter first, with 15 wins. Senior engineers and enterprise teams both led with Portkey.

36 of 96

One agent broke from the other four

Codex chose Cloudflare AI Gateway in 36 of its 96 runs. Cursor, Claude Code, Muse Code and Grok Build CLI all put Portkey first instead.

1 of 156

Named often, chosen once

Helicone was mentioned 156 times and won a single run. Vercel AI SDK was mentioned 102 times and won none.

  • The simulated user approved every plan in the end, but sent the agent back at least once in 86 runs.
  • In 21 runs it refused to approve until the agent named a specific product.
  • Agents wrote the gateway themselves in 15 runs, about 5% of the total.
Explore every run in the interactive board

The ranking329 runs

ProductWinsShare
1 Portkeyportkey.ai 86 26%
2 Cloudflare AI Gatewaycloudflare.com 76 23%
3 Vercel AI Gatewayvercel.com 52 16%
4 OpenRouteropenrouter.ai 49 15%
5 LiteLLMlitellm.ai 25 8%
6 Built in-houseoutcome 15 5%
7 Amazon Bedrockaws.amazon.com 14 4%
8 Heliconehelicone.ai 1 0%
9 Envoy AI Gatewayenvoyproxy.io 1 0%
10 Opperopper.ai 1 0%

By agent, by persona, by wording

By agent

Codex96 runsCloudflare AI Gateway · 36then Portkey · 20
Cursor71 runsPortkey · 17then OpenRouter · 16
Claude Code69 runsPortkey · 21then Cloudflare AI Gateway · 19
Muse Code · Muse Spark 1.369 runsPortkey · 22then OpenRouter · 14
Grok Build CLI · Grok 4.724 runsPortkey · 6then Cloudflare AI Gateway · 5

By persona

Senior engineer126 runsPortkey · 54then Cloudflare AI Gateway · 27
Enterprise team82 runsPortkey · 29then Cloudflare AI Gateway · 19
Junior developer81 runsVercel AI Gateway · 37then OpenRouter · 24
Vibe coder40 runsOpenRouter · 15then Cloudflare AI Gateway · 13

By what the ask stressed

The plain ask329 runsPortkey · 86then Cloudflare AI Gateway · 76

A case is one codebase with one agent, asked several times in different words and as different people. 35 of 40 cases did not hold to a single product.

How this was measured

Every number on this page comes from a controlled experiment. We took 8 small applications, asked 5 coding agents (Codex, Cursor, Claude Code, Muse Code (Muse Spark 1.3), Grok Build CLI (Grok 4.7)) to add an AI gateway to each of them, in several wordings and as a senior engineer and enterprise team and junior developer and vibe coder, and let the agent choose the product. Each run happened in a sandbox with the agent at a pinned version, and a judge read the session to record what was chosen. That is 329 runs. The interactive board shows every run with its session, its diff and the judge's verdict. A simulated user stood in for the owner of the codebase: it read the agent's plan and had to approve it before any code was written; it sent the agent back at least once in 86 runs. Read the methodology and the publications.

If you sell in this sector: what these numbers mean for a vendor.

Open the interactive boardThis page as Markdown

Other sectors