# GPT-6.2 Sol leaderboards: which developer tools GPT-6.2 Sol chooses in Codex

Source: https://armature.tech/leaderboards/sol-6-2 (also https://leaderboards.armature.tech/sol-6-2), Armature agent leaderboards. 994 runs, 34 sectors, 97 apps, one model (GPT-6.2 Sol) in one agent (Codex), updated 2026-11-18. Interactive board with every run: https://armature.tech/leaderboards/sol-6-2#app

We tested GPT-6.2 Sol in Codex across 34 sectors. These boards contain 994 runs.

- Open a sector to see its product rankings, key learnings, and evidence. Open a product to see where it wins and loses.
- The test set is the GPT-6.2 Sol campaign manifest, with one run per prompt input. The agent frameworks sector is left out.
- Product choices count even when the implementation was incomplete. Runs without captured evidence are not counted.

## Sectors at a glance

| Sector | Runs | First choice | Share |
|---|---:|---|---:|
| [Agent runtime](#agent-runtime) | 59 | Temporal + Vercel AI SDK | 10% |
| [AI gateway](#ai-gateway) | 24 | Vercel AI Gateway | 46% |
| [AI search](#ai-search) | 48 | Tavily | 27% |
| [AI SRE](#ai-sre) | 18 | Sentry Seer | 28% |
| [Auth](#auth) | 68 | Auth0 | 40% |
| [Background jobs](#background-jobs) | 16 | Celery | 6% |
| [Bot protection](#bot-protection) | 15 | Cloudflare Turnstile | 73% |
| [Cloud](#cloud) | 18 | AWS | 67% |
| [Code review](#code-review) | 30 | CodeRabbit | 40% |
| [Databases](#databases) | 18 | Neon | 44% |
| [Deploy](#deploy) | 30 | Render | 43% |
| [Document processing & OCR](#document-processing) | 60 | Azure AI Document Intelligence | 33% |
| [E-signature](#e-signature) | 18 | DocuSign | 72% |
| [LLM evals & observability](#evals) | 24 | Langfuse | 33% |
| [In-app chat & calls](#in-app-communication) | 18 | Daily | 22% |
| [Internationalization](#internationalization) | 36 | next-intl | 17% |
| [Email providers](#mail) | 18 | Resend | 56% |
| [Maps](#maps) | 57 | Google Maps | 12% |
| [MCP gateway](#mcp-gateway) | 15 | agentgateway | 40% |
| [Message queues](#message-queues) | 48 | Apache Kafka | 17% |
| [Model choice](#model-choice) | 9 | OpenAI (audio) | 100% |
| [Observability](#observability) | 30 | Sentry | 30% |
| [OCR](#ocr) | 15 | Azure AI Document Intelligence | 33% |
| [Payments](#payments) | 33 | Stripe | 91% |
| [Performance in CI](#perf-ci) | 18 | Grafana k6 | 28% |
| [Product analytics](#product-analytics) | 29 | PostHog | 55% |
| [Agent sandboxes](#sandboxes) | 17 | Vercel Sandbox | 59% |
| [Search](#search) | 53 | Typesense | 15% |
| [Serverless functions](#serverless) | 24 | AWS Lambda | 50% |
| [File storage](#storage) | 24 | Amazon S3 | 67% |
| [Usage-based billing](#usage-based-billing) | 33 | Metronome | 48% |
| [Vector search](#vector-search) | 17 | Pinecone | 24% |
| [Voice](#voice) | 9 | Deepgram | 33% |
| [Voice Agents](#voice-agents) | 45 | Retell AI | 60% |

## Agent runtime

<a id="agent-runtime"></a>Temporal + Vercel AI SDK shares the lead. 59 runs on 12 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/agent-runtime

We asked GPT-6.2 Sol to add agent runtime to 12 apps. This board contains 59 runs.

- **The leading products are tied (10%).** Temporal + Vercel AI SDK was chosen in 6 runs. Temporal + Vercel AI SDK + Vercel Sandbox also won 6 runs.
- **The project affects the choice (12 apps).** We tested 12 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Temporal + Vercel AI SDK | 6 | 10% |
| 2 | Temporal + Vercel AI SDK + Vercel Sandbox | 6 | 10% |
| 3 | Temporal (temporal.io) | 5 | 8% |
| 4 | E2B + Pydantic AI + Temporal | 4 | 7% |
| 5 | Vercel AI SDK + Vercel Sandbox + Vercel Workflow | 4 | 7% |
| 6 | LangChain4j + Temporal | 3 | 5% |
| 7 | E2B + FastMCP + Pydantic AI + Temporal | 3 | 5% |
| 8 | RubyLLM + Temporal | 3 | 5% |
| 9 | Pydantic AI + Temporal | 2 | 3% |
| 10 | E2B + LangChain + LangGraph | 2 | 3% |
| | 19 more on the board | | |

## AI gateway

<a id="ai-gateway"></a>Vercel AI Gateway leads the product choices. 24 runs on 8 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/ai-gateway

We asked GPT-6.2 Sol to add an AI gateway to 8 apps. This board contains 24 runs.

- **The leading choice (46%).** Vercel AI Gateway was chosen in 11 runs. Cloudflare AI Gateway followed with 5 runs.
- **The project affects the choice (8 apps).** We tested 8 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Vercel AI Gateway (vercel.com) | 11 | 46% |
| 2 | Cloudflare AI Gateway (cloudflare.com) | 5 | 21% |
| 3 | OpenRouter (openrouter.ai) | 3 | 13% |
| 4 | Portkey (portkey.ai) | 3 | 13% |
| 5 | Amazon Bedrock (aws.amazon.com) | 1 | 4% |
| 6 | Amazon Bedrock AgentCore Gateway (aws.amazon.com) | 1 | 4% |

## AI search

<a id="ai-search"></a>Tavily leads the product choices. 48 runs on 12 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/ai-search

We asked GPT-6.2 Sol to get live web information into 12 apps. This board contains 48 runs.

- **The leading choice (27%).** Tavily was chosen in 13 runs. Brave Search API followed with 9 runs.
- **The project affects the choice (12 apps).** We tested 12 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Tavily (tavily.com) | 13 | 27% |
| 2 | Brave Search API (brave.com) | 9 | 19% |
| 3 | OpenAI web search (openai.com) | 7 | 15% |
| 4 | Linkup (linkup.so) | 4 | 8% |
| 5 | Exa (exa.ai) | 3 | 6% |
| 6 | NewsAPI.ai (newsapi.ai) | 3 | 6% |
| 7 | Parallel (parallel.ai) | 3 | 6% |
| 8 | Firecrawl (firecrawl.dev) | 2 | 4% |
| 9 | Built in-house (outcome) | 2 | 4% |
| 10 | Nexis Data | 1 | 2% |
| | 1 more on the board | | |

## AI SRE

<a id="ai-sre"></a>Sentry Seer leads the product choices. 18 runs on 6 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/ai-sre

We asked GPT-6.2 Sol to add automatic incident investigation and fixes to 6 apps. This board contains 18 runs.

- **The leading choice (28%).** Sentry Seer was chosen in 5 runs. Datadog Bits AI Dev Agent + Datadog Bits Investigation followed with 3 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Sentry Seer (sentry.io) | 5 | 28% |
| 2 | Datadog Bits AI Dev Agent + Datadog Bits Investigation | 3 | 17% |
| 3 | AWS DevOps Agent (aws.amazon.com) | 2 | 11% |
| 4 | Better Stack AI SRE (betterstack.com) | 2 | 11% |
| 5 | AWS DevOps Agent + Claude Code | 1 | 6% |
| 6 | Islo | 1 | 6% |
| 7 | Kestrel (usekestrel.ai) | 1 | 6% |
| 8 | Grafana Assistant Investigations (grafana.com) | 1 | 6% |
| 9 | Corvo | 1 | 6% |

## Auth

<a id="auth"></a>Auth0 leads the product choices. 68 runs on 17 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/auth

We asked GPT-6.2 Sol to add auth to 17 apps. This board contains 68 runs.

- **The leading choice (40%).** Auth0 was chosen in 27 runs. Clerk followed with 8 runs.
- **The project affects the choice (17 apps).** We tested 17 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Auth0 (auth0.com) | 27 | 40% |
| 2 | Clerk (clerk.com) | 8 | 12% |
| 3 | WorkOS AuthKit (workos.com) | 8 | 12% |
| 4 | Microsoft Entra ID (entra.microsoft.com) | 7 | 10% |
| 5 | Keycloak (keycloak.org) | 7 | 10% |
| 6 | Built in-house (outcome) | 5 | 7% |
| 7 | Google Identity (developers.google.com) | 3 | 4% |
| 8 | Laravel Fortify + Laravel Socialite | 1 | 1% |
| 9 | openid-client (github.com) | 1 | 1% |
| 10 | Google Cloud Identity Platform (cloud.google.com) | 1 | 1% |

## Background jobs

<a id="background-jobs"></a>GPT-6.2 Sol most often built it in the app. 16 runs on 11 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/background-jobs

We asked GPT-6.2 Sol to add background jobs to 11 apps. This board contains 16 runs.

- **Building in the app comes first (81%).** The agents built the capability in the app in 13 runs. This was more common than any single product choice.
- **The project affects the choice (11 apps).** We tested 11 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 13 | 81% |
| 2 | Celery (celeryq.dev) | 1 | 6% |
| 3 | Trigger.dev (trigger.dev) | 1 | 6% |
| 4 | pg-boss (github.com) | 1 | 6% |

## Bot protection

<a id="bot-protection"></a>Cloudflare Turnstile leads the product choices. 15 runs on 5 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/bot-protection

We asked GPT-6.2 Sol to add bot protection to 5 apps. This board contains 15 runs.

- **The leading choice (73%).** Cloudflare Turnstile was chosen in 11 runs. Vercel BotID followed with 1 run.
- **The project affects the choice (5 apps).** We tested 5 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Cloudflare Turnstile (cloudflare.com) | 11 | 73% |
| 2 | Vercel BotID (vercel.com) | 1 | 7% |

## Cloud

<a id="cloud"></a>AWS leads the product choices. 18 runs on 6 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/cloud

We asked GPT-6.2 Sol to pick a cloud for 6 apps. This board contains 18 runs.

- **The leading choice (67%).** AWS was chosen in 12 runs. Google Cloud followed with 3 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | AWS (aws.amazon.com) | 12 | 67% |
| 2 | Google Cloud (cloud.google.com) | 3 | 17% |
| 3 | Cloudflare (cloudflare.com) | 1 | 6% |
| 4 | Upstash (upstash.com) | 1 | 6% |
| 5 | Vercel (vercel.com) | 1 | 6% |

## Code review

<a id="code-review"></a>CodeRabbit leads the product choices. 30 runs on 10 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/code-review

We asked GPT-6.2 Sol to add automated code review to 10 apps. This board contains 30 runs.

- **The leading choice (40%).** CodeRabbit was chosen in 12 runs. Qodo Merge followed with 9 runs.
- **The project affects the choice (10 apps).** We tested 10 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | CodeRabbit (coderabbit.ai) | 12 | 40% |
| 2 | Qodo Merge (qodo.ai) | 9 | 30% |
| 3 | GitHub Copilot code review (github.com) | 2 | 7% |
| 4 | Codex code review (openai.com) | 2 | 7% |
| 5 | Sourcery (sourcery.ai) | 1 | 3% |
| 6 | cubic (cubic.dev) | 1 | 3% |
| 7 | reviewdog (github.com) | 1 | 3% |
| 8 | Semgrep (semgrep.dev) | 1 | 3% |
| 9 | Built in-house (outcome) | 1 | 3% |

## Databases

<a id="databases"></a>Neon leads the product choices. 18 runs on 6 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/databases

We asked GPT-6.2 Sol to pick a database for 6 apps. This board contains 18 runs.

- **The leading choice (44%).** Neon was chosen in 8 runs. Amazon RDS for PostgreSQL followed with 5 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Neon (neon.tech) | 8 | 44% |
| 2 | Amazon RDS for PostgreSQL (aws.amazon.com) | 5 | 28% |
| 3 | Azure Database for PostgreSQL Flexible Server (azure.microsoft.com) | 2 | 11% |
| 4 | Supabase (supabase.com) | 1 | 6% |
| 5 | DigitalOcean Managed PostgreSQL (digitalocean.com) | 1 | 6% |
| 6 | Azure SQL Database (azure.microsoft.com) | 1 | 6% |

## Deploy

<a id="deploy"></a>Render leads the product choices. 30 runs on 6 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/deploy

We asked GPT-6.2 Sol to deploy 6 apps. This board contains 30 runs.

- **The leading choice (43%).** Render was chosen in 13 runs. Vercel followed with 9 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Render (render.com) | 13 | 43% |
| 2 | Vercel (vercel.com) | 9 | 30% |
| 3 | Cloudflare (cloudflare.com) | 4 | 13% |
| 4 | Google Cloud (cloud.google.com) | 2 | 7% |
| 5 | GitHub Pages (pages.github.com) | 1 | 3% |
| 6 | Netlify (netlify.com) | 1 | 3% |

## Document processing & OCR

<a id="document-processing"></a>Azure AI Document Intelligence leads the product choices. 60 runs on 12 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/document-processing

We asked GPT-6.2 Sol to read the documents that arrive at 12 apps. This board contains 60 runs.

- **The leading choice (33%).** Azure AI Document Intelligence was chosen in 20 runs. Google Cloud Document AI followed with 16 runs.
- **The project affects the choice (12 apps).** We tested 12 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Azure AI Document Intelligence (azure.microsoft.com) | 20 | 33% |
| 2 | Google Cloud Document AI (cloud.google.com) | 16 | 27% |
| 3 | Docling (docling-project.github.io) | 7 | 12% |
| 4 | Amazon Textract (aws.amazon.com) | 6 | 10% |
| 5 | Apache PDFBox + Tesseract OCR | 3 | 5% |
| 6 | Google Cloud Document AI + Google Gemini | 2 | 3% |
| 7 | LlamaParse (llamaindex.ai) | 2 | 3% |
| 8 | Google Cloud Vision (cloud.google.com) | 1 | 2% |
| 9 | OpenAI models (openai.com) | 1 | 2% |
| 10 | Azure AI Content Understanding (azure.microsoft.com) | 1 | 2% |
| | 1 more on the board | | |

## E-signature

<a id="e-signature"></a>DocuSign leads the product choices. 18 runs on 9 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/e-signature

We asked GPT-6.2 Sol to add e-signature to 9 apps. This board contains 18 runs.

- **The leading choice (72%).** DocuSign was chosen in 13 runs. Documenso followed with 2 runs.
- **The project affects the choice (9 apps).** We tested 9 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | DocuSign (docusign.com) | 13 | 72% |
| 2 | Documenso (documenso.com) | 2 | 11% |
| 3 | SignWell (signwell.com) | 1 | 6% |
| 4 | Scrive (scrive.com) | 1 | 6% |
| 5 | YouTrust (youtrust.com) | 1 | 6% |

## LLM evals & observability

<a id="evals"></a>Langfuse leads the product choices. 24 runs on 2 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/evals

We asked GPT-6.2 Sol to add LLM evals to 2 apps. This board contains 24 runs.

- **The leading choice (33%).** Langfuse was chosen in 8 runs. Promptfoo followed with 5 runs.
- **The project affects the choice (2 apps).** We tested 2 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Langfuse (langfuse.com) | 8 | 33% |
| 2 | Promptfoo (promptfoo.dev) | 5 | 21% |
| 3 | Built in-house (outcome) | 5 | 21% |
| 4 | pydantic-evals (pydantic.dev) | 2 | 8% |
| 5 | Arize Phoenix (arize.com) | 2 | 8% |
| 6 | Comet Opik | 1 | 4% |
| 7 | Braintrust (braintrust.dev) | 1 | 4% |

## In-app chat & calls

<a id="in-app-communication"></a>Daily shares the lead. 18 runs on 6 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/in-app-communication

We asked GPT-6.2 Sol to add in-app chat and calls to 6 apps. This board contains 18 runs.

- **The leading products are tied (22%).** Daily was chosen in 4 runs. Stream Chat and Video also won 4 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Daily (daily.co) | 4 | 22% |
| 2 | Stream Chat and Video (getstream.io) | 4 | 22% |
| 3 | Supabase (supabase.com) | 3 | 17% |
| 4 | Ably + LiveKit | 2 | 11% |
| 5 | LiveKit + Twilio | 1 | 6% |
| 6 | TalkJS (talkjs.com) | 1 | 6% |
| 7 | Daily + Twilio | 1 | 6% |
| 8 | LiveKit (livekit.io) | 1 | 6% |
| 9 | Twilio (twilio.com) | 1 | 6% |

## Internationalization

<a id="internationalization"></a>next-intl shares the lead. 36 runs on 12 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/internationalization

We asked GPT-6.2 Sol to add internationalization to 12 apps. This board contains 36 runs.

- **The leading products are tied (17%).** next-intl was chosen in 6 runs. Django translation framework also won 6 runs.
- **The project affects the choice (12 apps).** We tested 12 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | next-intl (next-intl.dev) | 6 | 17% |
| 2 | Django translation framework (docs.djangoproject.com) | 6 | 17% |
| 3 | Rails I18n (guides.rubyonrails.org) | 3 | 8% |
| 4 | Paraglide JS (inlang.com) | 3 | 8% |
| 5 | Symfony Translation (symfony.com) | 3 | 8% |
| 6 | i18next (i18next.com) | 2 | 6% |
| 7 | Vue I18n (vue-i18n.intlify.dev) | 2 | 6% |
| 8 | Intl API (developer.mozilla.org) | 2 | 6% |
| 9 | Crowdin + next-intl | 2 | 6% |
| 10 | FormatJS (formatjs.github.io) | 2 | 6% |
| | 4 more on the board | | |

## Email providers

<a id="mail"></a>Resend leads the product choices. 18 runs on 9 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/mail

We asked GPT-6.2 Sol to add email sending to 9 apps. This board contains 18 runs.

- **The leading choice (56%).** Resend was chosen in 10 runs. SendGrid followed with 2 runs.
- **The project affects the choice (9 apps).** We tested 9 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Resend (resend.com) | 10 | 56% |
| 2 | SendGrid (sendgrid.com) | 2 | 11% |
| 3 | Postmark (postmarkapp.com) | 2 | 11% |
| 4 | Amazon SES (aws.amazon.com) | 2 | 11% |
| 5 | Azure Communication Services (azure.microsoft.com) | 1 | 6% |
| 6 | Generic SMTP (nodemailer.com) | 1 | 6% |

## Maps

<a id="maps"></a>Google Maps leads the product choices. 57 runs on 14 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/maps

We asked GPT-6.2 Sol to add a map to 14 apps. This board contains 57 runs.

- **The leading choice (12%).** Google Maps was chosen in 7 runs. Apple Maps (MapKit) followed with 6 runs.
- **The project affects the choice (14 apps).** We tested 14 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Google Maps (mapsplatform.google.com) | 7 | 12% |
| 2 | Apple Maps (MapKit) (developer.apple.com) | 6 | 11% |
| 3 | Leaflet (leafletjs.com) | 5 | 9% |
| 4 | Mapbox (mapbox.com) | 4 | 7% |
| 5 | react-native-maps (github.com) | 3 | 5% |
| 6 | OpenStreetMap (openstreetmap.org) | 3 | 5% |
| 7 | Geoapify (geoapify.com) | 3 | 5% |
| 8 | Leaflet + MapTiler | 3 | 5% |
| 9 | Azure Maps (azure.microsoft.com) | 3 | 5% |
| 10 | MapLibre (maplibre.org) | 2 | 4% |
| | 16 more on the board | | |

## MCP gateway

<a id="mcp-gateway"></a>agentgateway leads the product choices. 15 runs on 5 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/mcp-gateway

We asked GPT-6.2 Sol to add mcp gateway to 5 apps. This board contains 15 runs.

- **The leading choice (40%).** agentgateway was chosen in 6 runs. Amazon Bedrock AgentCore Gateway followed with 3 runs.
- **The project affects the choice (5 apps).** We tested 5 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | agentgateway (agentgateway.dev) | 6 | 40% |
| 2 | Amazon Bedrock AgentCore Gateway (aws.amazon.com) | 3 | 20% |
| 3 | Open Mcp Portal | 1 | 7% |
| 4 | R0wi Mcp Gateway | 1 | 7% |
| 5 | Cloudflare One Mcp Server Portals | 1 | 7% |
| 6 | Bifrost | 1 | 7% |
| 7 | Cloudflare MCP Server Portals (cloudflare.com) | 1 | 7% |
| 8 | IBM ContextForge (ibm.com) | 1 | 7% |

## Message queues

<a id="message-queues"></a>GPT-6.2 Sol most often built it in the app. 48 runs on 12 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/message-queues

We asked GPT-6.2 Sol to add a message queue to 12 apps. This board contains 48 runs.

- **Building in the app comes first (21%).** The agents built the capability in the app in 10 runs. This was more common than any single product choice.
- **The project affects the choice (12 apps).** We tested 12 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 10 | 21% |
| 2 | Apache Kafka (kafka.apache.org) | 8 | 17% |
| 3 | Amazon SQS (aws.amazon.com) | 6 | 13% |
| 4 | Laravel Queues (laravel.com) | 4 | 8% |
| 5 | Service Bus (azure.microsoft.com) | 4 | 8% |
| 6 | Amazon Kinesis (aws.amazon.com) | 4 | 8% |
| 7 | Cloudflare Queues (cloudflare.com) | 3 | 6% |
| 8 | Inngest (inngest.com) | 3 | 6% |
| 9 | Cloud Tasks (cloud.google.com) | 3 | 6% |
| 10 | Vercel Workflow (vercel.com) | 1 | 2% |
| | 2 more on the board | | |

## Model choice

<a id="model-choice"></a>OpenAI (audio) leads the product choices. 9 runs on 3 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/model-choice

We asked GPT-6.2 Sol to add model choice to 3 apps. This board contains 9 runs.

- **The leading choice (100%).** OpenAI (audio) was chosen in 9 runs. No other product won a run.
- **The project affects the choice (3 apps).** We tested 3 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | OpenAI (audio) (openai.com) | 9 | 100% |

## Observability

<a id="observability"></a>Sentry leads the product choices. 30 runs on 11 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/observability

We asked GPT-6.2 Sol to add observability to 11 apps. This board contains 30 runs.

- **The leading choice (30%).** Sentry was chosen in 9 runs. Grafana followed with 5 runs.
- **The project affects the choice (11 apps).** We tested 11 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Sentry (sentry.io) | 9 | 30% |
| 2 | Grafana (grafana.com) | 5 | 17% |
| 3 | Amazon CloudWatch (aws.amazon.com) | 4 | 13% |
| 4 | Checkly (checklyhq.com) | 4 | 13% |
| 5 | New Relic (newrelic.com) | 3 | 10% |
| 6 | Honeycomb (honeycomb.io) | 1 | 3% |
| 7 | GlitchTip (glitchtip.com) | 1 | 3% |
| 8 | Datadog (datadoghq.com) | 1 | 3% |
| 9 | AWS X-Ray (aws.amazon.com) | 1 | 3% |
| 10 | Azure Monitor (azure.microsoft.com) | 1 | 3% |

## OCR

<a id="ocr"></a>Azure AI Document Intelligence leads the product choices. 15 runs on 5 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/ocr

We asked GPT-6.2 Sol to add ocr to 5 apps. This board contains 15 runs.

- **The leading choice (33%).** Azure AI Document Intelligence was chosen in 5 runs. Tesseract OCR followed with 3 runs.
- **The project affects the choice (5 apps).** We tested 5 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Azure AI Document Intelligence (azure.microsoft.com) | 5 | 33% |
| 2 | Tesseract OCR (github.com) | 3 | 20% |
| 3 | Google Cloud Document AI (cloud.google.com) | 3 | 20% |
| 4 | Amazon Textract (aws.amazon.com) | 2 | 13% |
| 5 | OpenAI multimodal models (openai.com) | 1 | 7% |
| 6 | Qwen Qwen3 Vl 32b Instruct | 1 | 7% |

## Payments

<a id="payments"></a>Stripe leads the product choices. 33 runs on 11 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/payments

We asked GPT-6.2 Sol to add payments to 11 apps. This board contains 33 runs.

- **The leading choice (91%).** Stripe was chosen in 30 runs. Paddle followed with 2 runs.
- **The project affects the choice (11 apps).** We tested 11 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Stripe (stripe.com) | 30 | 91% |
| 2 | Paddle (paddle.com) | 2 | 6% |
| 3 | Mollie (mollie.com) | 1 | 3% |

## Performance in CI

<a id="perf-ci"></a>GPT-6.2 Sol most often built it in the app. 18 runs on 8 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/perf-ci

We asked GPT-6.2 Sol to add a performance gate to 8 apps. This board contains 18 runs.

- **Building in the app comes first (56%).** The agents built the capability in the app in 10 runs. This was more common than any single product choice.
- **The project affects the choice (8 apps).** We tested 8 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 10 | 56% |
| 2 | Grafana k6 (k6.io) | 5 | 28% |
| 3 | Autocannon (github.com) | 1 | 6% |
| 4 | PHPBench (phpbench.readthedocs.io) | 1 | 6% |
| 5 | hyperfine (github.com) | 1 | 6% |

## Product analytics

<a id="product-analytics"></a>PostHog leads the product choices. 29 runs on 10 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/product-analytics

We asked GPT-6.2 Sol to add product analytics to 10 apps. This board contains 29 runs.

- **The leading choice (55%).** PostHog was chosen in 16 runs. Metabase followed with 4 runs.
- **The project affects the choice (10 apps).** We tested 10 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | PostHog (posthog.com) | 16 | 55% |
| 2 | Metabase (metabase.com) | 4 | 14% |
| 3 | Mixpanel (mixpanel.com) | 2 | 7% |
| 4 | Mitzu (mitzu.io) | 1 | 3% |
| 5 | Umami (umami.is) | 1 | 3% |
| 6 | Datadog Product Analytics (datadoghq.com) | 1 | 3% |
| 7 | Amazon Quick Sight | 1 | 3% |
| 8 | Looker (cloud.google.com) | 1 | 3% |

## Agent sandboxes

<a id="sandboxes"></a>Vercel Sandbox leads the product choices. 17 runs on 5 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/sandboxes

We asked GPT-6.2 Sol to add a code sandbox to 5 apps. This board contains 17 runs.

- **The leading choice (59%).** Vercel Sandbox was chosen in 10 runs. Modal followed with 5 runs.
- **The project affects the choice (5 apps).** We tested 5 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Vercel Sandbox (vercel.com) | 10 | 59% |
| 2 | Modal (modal.com) | 5 | 29% |
| 3 | Daytona (daytona.io) | 1 | 6% |
| 4 | E2B (e2b.dev) | 1 | 6% |

## Search

<a id="search"></a>GPT-6.2 Sol most often built it in the app. 53 runs on 11 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/search

We asked GPT-6.2 Sol to add search to 11 apps. This board contains 53 runs.

- **Building in the app comes first (32%).** The agents built the capability in the app in 17 runs. This was more common than any single product choice.
- **The project affects the choice (11 apps).** We tested 11 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 17 | 32% |
| 2 | Typesense (typesense.org) | 8 | 15% |
| 3 | pg_trgm (postgresql.org) | 7 | 13% |
| 4 | OpenSearch (opensearch.org) | 6 | 11% |
| 5 | Elasticsearch (elastic.co) | 5 | 9% |
| 6 | Oracle Indexed Search (oracle.com) | 2 | 4% |
| 7 | Atlas Search (mongodb.com) | 2 | 4% |
| 8 | Algolia (algolia.com) | 2 | 4% |
| 9 | MongoDB Text Search (mongodb.com) | 1 | 2% |
| 10 | Fuse.js (fusejs.io) | 1 | 2% |
| | 2 more on the board | | |

## Serverless functions

<a id="serverless"></a>AWS Lambda leads the product choices. 24 runs on 8 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/serverless

We asked GPT-6.2 Sol to add serverless functions to 8 apps. This board contains 24 runs.

- **The leading choice (50%).** AWS Lambda was chosen in 12 runs. Cloudflare Workers followed with 3 runs.
- **The project affects the choice (8 apps).** We tested 8 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | AWS Lambda (aws.amazon.com) | 12 | 50% |
| 2 | Cloudflare Workers (workers.cloudflare.com) | 3 | 13% |
| 3 | Azure Functions (azure.microsoft.com) | 3 | 13% |
| 4 | Vercel Functions (vercel.com) | 3 | 13% |
| 5 | Google Cloud Run (cloud.google.com) | 2 | 8% |
| 6 | Netlify Functions (netlify.com) | 1 | 4% |

## File storage

<a id="storage"></a>Amazon S3 leads the product choices. 24 runs on 8 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/storage

We asked GPT-6.2 Sol to add file storage to 8 apps. This board contains 24 runs.

- **The leading choice (67%).** Amazon S3 was chosen in 16 runs. Google Cloud Storage followed with 3 runs.
- **The project affects the choice (8 apps).** We tested 8 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Amazon S3 (aws.amazon.com) | 16 | 67% |
| 2 | Google Cloud Storage (cloud.google.com) | 3 | 13% |
| 3 | Azure Blob Storage (azure.microsoft.com) | 3 | 13% |
| 4 | Cloudflare R2 (cloudflare.com) | 2 | 8% |

## Usage-based billing

<a id="usage-based-billing"></a>Metronome leads the product choices. 33 runs on 11 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/usage-based-billing

We asked GPT-6.2 Sol to bill customers for what they use in 11 apps. This board contains 33 runs.

- **The leading choice (48%).** Metronome was chosen in 16 runs. Lago followed with 7 runs.
- **The project affects the choice (11 apps).** We tested 11 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Metronome (metronome.com) | 16 | 48% |
| 2 | Lago (getlago.com) | 7 | 21% |
| 3 | Built in-house (outcome) | 5 | 15% |
| 4 | Orb (withorb.com) | 2 | 6% |
| 5 | Oracle BRM (oracle.com) | 2 | 6% |
| 6 | PortaBilling | 1 | 3% |

## Vector search

<a id="vector-search"></a>Pinecone leads the product choices. 17 runs on 6 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/vector-search

We asked GPT-6.2 Sol to add vector search to 6 apps. This board contains 17 runs.

- **The leading choice (24%).** Pinecone was chosen in 4 runs. Typesense followed with 3 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Pinecone (pinecone.io) | 4 | 24% |
| 2 | Typesense (typesense.org) | 3 | 18% |
| 3 | Neon (neon.tech) | 3 | 18% |
| 4 | OpenAI Vector Stores (openai.com) | 2 | 12% |
| 5 | Qdrant (qdrant.tech) | 2 | 12% |
| 6 | Supabase Vector (supabase.com) | 1 | 6% |
| 7 | Meilisearch (meilisearch.com) | 1 | 6% |
| 8 | Azure AI Search (azure.microsoft.com) | 1 | 6% |

## Voice

<a id="voice"></a>Deepgram leads the product choices. 9 runs on 6 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/voice

We asked GPT-6.2 Sol to add voice to 6 apps. This board contains 9 runs.

- **The leading choice (33%).** Deepgram was chosen in 3 runs. Microsoft Azure followed with 2 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Deepgram (deepgram.com) | 3 | 33% |
| 2 | Microsoft Azure (azure.microsoft.com) | 2 | 22% |
| 3 | OpenAI (audio) (openai.com) | 1 | 11% |
| 4 | ElevenLabs (elevenlabs.io) | 1 | 11% |
| 5 | Web Speech API (developer.mozilla.org) | 1 | 11% |
| 6 | Piper (github.com) | 1 | 11% |

## Voice Agents

<a id="voice-agents"></a>Retell AI leads the product choices. 45 runs on 10 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/sol-6-2#app/voice-agents

We asked GPT-6.2 Sol to add a voice agent to 10 apps. This board contains 45 runs.

- **The leading choice (60%).** Retell AI was chosen in 27 runs. OpenAI Realtime API followed with 7 runs.
- **The project affects the choice (10 apps).** We tested 10 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Retell AI (retellai.com) | 27 | 60% |
| 2 | OpenAI Realtime API (openai.com) | 7 | 16% |
| 3 | LiveKit Agents (livekit.io) | 5 | 11% |
| 4 | ElevenLabs Agents (elevenlabs.io) | 2 | 4% |
| 5 | Amazon Connect (aws.amazon.com) | 1 | 2% |
| 6 | OpenAI GPT-Live (openai.com) | 1 | 2% |
| 7 | sherpa-onnx (github.com) | 1 | 2% |
| 8 | Reception.ai (elevenlabs.io) | 1 | 2% |

## How this was measured

Every number comes from a controlled experiment. GPT-6.2 Sol ran in Codex on 97 existing codebases, asked in several wordings and as different kinds of user to add one capability (the sector) and to choose the product itself. Each run happened in a sandbox at a pinned agent version, and a judge read the session to record the choice. A choice counts when the session captured it and the judge confirmed it, even when the implementation was incomplete; runs without captured evidence are left out. Shares are wins over all runs in the sector. Several tools adopted together count as one combination. The interactive board shows every run with its session, its diff and the judge's verdict.

Methodology and publications: https://armature.tech/publications

## Other boards

- All agents, sector by sector: https://armature.tech/leaderboards/sectors (Markdown: https://armature.tech/leaderboards/sectors.md)
- GPT-6.2 Astra: https://armature.tech/leaderboards/astra-6-2 (Markdown: https://armature.tech/leaderboards/astra-6-2.md)
- Claude Fable 5.2: https://armature.tech/leaderboards/fable-5-2 (Markdown: https://armature.tech/leaderboards/fable-5-2.md)
- Muse Ember: https://armature.tech/leaderboards/muse-ember (Markdown: https://armature.tech/leaderboards/muse-ember.md)
- Muse Wonder: https://armature.tech/leaderboards/muse-wonder (Markdown: https://armature.tech/leaderboards/muse-wonder.md)
- Grok 4.7: https://armature.tech/leaderboards/grok-4-7 (Markdown: https://armature.tech/leaderboards/grok-4-7.md)
- Claude Opus 5.5: https://armature.tech/leaderboards/opus-5-5 (Markdown: https://armature.tech/leaderboards/opus-5-5.md)
- GPT-6.1 Sol: https://armature.tech/leaderboards/sol-6-1 (Markdown: https://armature.tech/leaderboards/sol-6-1.md)
- Muse Spark 1.3: https://armature.tech/leaderboards/muse-spark-1-3 (Markdown: https://armature.tech/leaderboards/muse-spark-1-3.md)
- GPT-6 Sol: https://armature.tech/leaderboards/sol-6 (Markdown: https://armature.tech/leaderboards/sol-6.md)
