# GPT-6.2 Astra leaderboards: which developer tools GPT-6.2 Astra chooses in Codex

Source: https://armature.tech/leaderboards/astra-6-2 (also https://leaderboards.armature.tech/astra-6-2), Armature agent leaderboards. 989 runs, 34 sectors, 97 apps, one model (GPT-6.2 Astra) in one agent (Codex), updated 2026-11-17. Interactive board with every run: https://armature.tech/leaderboards/astra-6-2#app

We tested GPT-6.2 Astra in Codex across 34 sectors. These boards contain 989 runs.

- Open a sector to see its product rankings, key learnings, and evidence. Open a product to see where it wins and loses.
- The test set is the GPT-6.2 Astra campaign manifest, with one run per prompt input. The agent frameworks sector is left out.
- Product choices count even when the implementation was incomplete. Runs without captured evidence are not counted.

## Sectors at a glance

| Sector | Runs | First choice | Share |
|---|---:|---|---:|
| [Agent runtime](#agent-runtime) | 60 | eve | 15% |
| [AI gateway](#ai-gateway) | 24 | Cloudflare AI Gateway | 42% |
| [AI search](#ai-search) | 48 | Brave Search API | 25% |
| [AI SRE](#ai-sre) | 17 | Sentry Seer | 18% |
| [Auth](#auth) | 68 | WorkOS AuthKit | 32% |
| [Background jobs](#background-jobs) | 16 | Celery | 13% |
| [Bot protection](#bot-protection) | 15 | Cloudflare Turnstile | 53% |
| [Cloud](#cloud) | 18 | AWS | 67% |
| [Code review](#code-review) | 30 | CodeRabbit | 47% |
| [Databases](#databases) | 18 | Neon | 44% |
| [Deploy](#deploy) | 30 | Render | 40% |
| [Document processing & OCR](#document-processing) | 59 | Azure AI Document Intelligence | 36% |
| [E-signature](#e-signature) | 18 | DocuSign | 56% |
| [LLM evals & observability](#evals) | 24 | Cloudflare AI Gateway | 17% |
| [In-app chat & calls](#in-app-communication) | 18 | Stream Chat and Video | 33% |
| [Internationalization](#internationalization) | 36 | next-intl | 11% |
| [Email providers](#mail) | 17 | Resend | 41% |
| [Maps](#maps) | 59 | Google Maps | 14% |
| [MCP gateway](#mcp-gateway) | 15 | agentgateway | 27% |
| [Message queues](#message-queues) | 45 | Apache Kafka | 16% |
| [Model choice](#model-choice) | 9 | OpenAI (audio) | 100% |
| [Observability](#observability) | 30 | Sentry | 27% |
| [OCR](#ocr) | 15 | Azure AI Document Intelligence | 40% |
| [Payments](#payments) | 32 | Stripe | 81% |
| [Performance in CI](#perf-ci) | 18 | Grafana k6 | 17% |
| [Product analytics](#product-analytics) | 30 | PostHog | 57% |
| [Agent sandboxes](#sandboxes) | 17 | Vercel Sandbox | 59% |
| [Search](#search) | 53 | pg_trgm | 15% |
| [Serverless functions](#serverless) | 24 | AWS Lambda | 38% |
| [File storage](#storage) | 24 | Amazon S3 | 58% |
| [Usage-based billing](#usage-based-billing) | 33 | Metronome | 58% |
| [Vector search](#vector-search) | 16 | Neon | 31% |
| [Voice](#voice) | 9 | Deepgram | 33% |
| [Voice Agents](#voice-agents) | 44 | Retell AI | 50% |

## Agent runtime

<a id="agent-runtime"></a>eve leads the product choices. 60 runs on 12 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/agent-runtime

We asked GPT-6.2 Astra to add agent runtime to 12 apps. This board contains 60 runs.

- **The leading choice (15%).** eve was chosen in 9 runs. E2B + Temporal + Vercel AI SDK followed with 3 runs.
- **The project affects the choice (12 apps).** We tested 12 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | eve (vercel.com) | 9 | 15% |
| 2 | E2B + Temporal + Vercel AI SDK | 3 | 5% |
| 3 | E2B + LangChain4j + Temporal | 3 | 5% |
| 4 | eve + Vercel Sandbox + Vercel Workflow | 3 | 5% |
| 5 | E2B + Pydantic AI + Temporal | 2 | 3% |
| 6 | RubyLLM + Temporal | 2 | 3% |
| 7 | Temporal + Vercel AI SDK | 2 | 3% |
| 8 | Vercel AI SDK + Vercel Sandbox + Vercel Workflow | 2 | 3% |
| 9 | AWS Bedrock AgentCore + Pydantic AI + Temporal | 2 | 3% |
| 10 | LangChain + LangGraph + Modal | 2 | 3% |
| | 25 more on the board | | |

## AI gateway

<a id="ai-gateway"></a>Cloudflare AI Gateway leads the product choices. 24 runs on 8 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/ai-gateway

We asked GPT-6.2 Astra to add an AI gateway to 8 apps. This board contains 24 runs.

- **The leading choice (42%).** Cloudflare AI Gateway was chosen in 10 runs. Vercel AI Gateway followed with 6 runs.
- **The project affects the choice (8 apps).** We tested 8 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Cloudflare AI Gateway (cloudflare.com) | 10 | 42% |
| 2 | Vercel AI Gateway (vercel.com) | 6 | 25% |
| 3 | OpenRouter (openrouter.ai) | 6 | 25% |
| 4 | Amazon Bedrock (aws.amazon.com) | 1 | 4% |

## AI search

<a id="ai-search"></a>Brave Search API leads the product choices. 48 runs on 12 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/ai-search

We asked GPT-6.2 Astra to get live web information into 12 apps. This board contains 48 runs.

- **The leading choice (25%).** Brave Search API was chosen in 12 runs. Exa followed with 9 runs.
- **The project affects the choice (12 apps).** We tested 12 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Brave Search API (brave.com) | 12 | 25% |
| 2 | Exa (exa.ai) | 9 | 19% |
| 3 | Parallel (parallel.ai) | 8 | 17% |
| 4 | Linkup (linkup.so) | 3 | 6% |
| 5 | Firecrawl (firecrawl.dev) | 3 | 6% |
| 6 | Tavily (tavily.com) | 3 | 6% |
| 7 | Mistral web search (mistral.ai) | 2 | 4% |
| 8 | NewsAPI.ai (newsapi.ai) | 1 | 2% |
| 9 | Mojeek (mojeek.com) | 1 | 2% |
| 10 | Feedly | 1 | 2% |
| | 5 more on the board | | |

## AI SRE

<a id="ai-sre"></a>Sentry Seer shares the lead. 17 runs on 6 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/ai-sre

We asked GPT-6.2 Astra to add automatic incident investigation and fixes to 6 apps. This board contains 17 runs.

- **The leading products are tied (18%).** Sentry Seer was chosen in 3 runs. Datadog Bits AI Dev Agent + Datadog Bits Investigation also won 3 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Sentry Seer (sentry.io) | 3 | 18% |
| 2 | Datadog Bits AI Dev Agent + Datadog Bits Investigation | 3 | 18% |
| 3 | Corvo | 2 | 12% |
| 4 | Rush | 2 | 12% |
| 5 | AWS DevOps Agent (aws.amazon.com) | 1 | 6% |
| 6 | StackPilot (stack-pilot.com) | 1 | 6% |
| 7 | Better Stack AI SRE (betterstack.com) | 1 | 6% |
| 8 | AWS DevOps Agent + Github Copilot Cloud Agent | 1 | 6% |
| 9 | Resolve AI (resolve.ai) | 1 | 6% |
| 10 | Cleric (cleric.ai) | 1 | 6% |

## Auth

<a id="auth"></a>WorkOS AuthKit leads the product choices. 68 runs on 17 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/auth

We asked GPT-6.2 Astra to add auth to 17 apps. This board contains 68 runs.

- **The leading choice (32%).** WorkOS AuthKit was chosen in 22 runs. Auth0 followed with 12 runs.
- **The project affects the choice (17 apps).** We tested 17 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | WorkOS AuthKit (workos.com) | 22 | 32% |
| 2 | Auth0 (auth0.com) | 12 | 18% |
| 3 | Microsoft Entra ID (entra.microsoft.com) | 7 | 10% |
| 4 | Google Identity (developers.google.com) | 6 | 9% |
| 5 | Clerk (clerk.com) | 5 | 7% |
| 6 | Google Cloud Identity Platform (cloud.google.com) | 4 | 6% |
| 7 | Keycloak (keycloak.org) | 4 | 6% |
| 8 | Google Sign-In (developers.google.com) | 2 | 3% |
| 9 | Amazon Cognito (aws.amazon.com) | 2 | 3% |
| 10 | Better Auth (better-auth.com) | 1 | 1% |
| | 3 more on the board | | |

## Background jobs

<a id="background-jobs"></a>GPT-6.2 Astra most often built it in the app. 16 runs on 11 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/background-jobs

We asked GPT-6.2 Astra to add background jobs to 11 apps. This board contains 16 runs.

- **Building in the app comes first (63%).** The agents built the capability in the app in 10 runs. This was more common than any single product choice.
- **The project affects the choice (11 apps).** We tested 11 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 10 | 63% |
| 2 | Celery (celeryq.dev) | 2 | 13% |
| 3 | pg-boss (github.com) | 1 | 6% |
| 4 | GoodJob (github.com) | 1 | 6% |
| 5 | Trigger.dev (trigger.dev) | 1 | 6% |
| 6 | Supabase Queues (supabase.com) | 1 | 6% |

## Bot protection

<a id="bot-protection"></a>Cloudflare Turnstile leads the product choices. 15 runs on 5 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/bot-protection

We asked GPT-6.2 Astra to add bot protection to 5 apps. This board contains 15 runs.

- **The leading choice (53%).** Cloudflare Turnstile was chosen in 8 runs. Vercel BotID followed with 1 run.
- **The project affects the choice (5 apps).** We tested 5 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Cloudflare Turnstile (cloudflare.com) | 8 | 53% |
| 2 | Built in-house (outcome) | 2 | 13% |
| 3 | Vercel BotID (vercel.com) | 1 | 7% |
| 4 | Google reCAPTCHA (google.com) | 1 | 7% |

## Cloud

<a id="cloud"></a>AWS leads the product choices. 18 runs on 6 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/cloud

We asked GPT-6.2 Astra to pick a cloud for 6 apps. This board contains 18 runs.

- **The leading choice (67%).** AWS was chosen in 12 runs. Google Cloud followed with 2 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | AWS (aws.amazon.com) | 12 | 67% |
| 2 | Google Cloud (cloud.google.com) | 2 | 11% |
| 3 | Vercel Queues (vercel.com) | 2 | 11% |
| 4 | Cloudflare (cloudflare.com) | 1 | 6% |
| 5 | Google Cloud + Redis | 1 | 6% |

## Code review

<a id="code-review"></a>CodeRabbit leads the product choices. 30 runs on 10 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/code-review

We asked GPT-6.2 Astra to add automated code review to 10 apps. This board contains 30 runs.

- **The leading choice (47%).** CodeRabbit was chosen in 14 runs. Qodo Merge followed with 8 runs.
- **The project affects the choice (10 apps).** We tested 10 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | CodeRabbit (coderabbit.ai) | 14 | 47% |
| 2 | Qodo Merge (qodo.ai) | 8 | 27% |
| 3 | GitHub Copilot code review (github.com) | 3 | 10% |
| 4 | Sourcery (sourcery.ai) | 2 | 7% |
| 5 | cubic (cubic.dev) | 1 | 3% |
| 6 | SonarQube (sonarsource.com) | 1 | 3% |
| 7 | Built in-house (outcome) | 1 | 3% |

## Databases

<a id="databases"></a>Neon leads the product choices. 18 runs on 6 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/databases

We asked GPT-6.2 Astra to pick a database for 6 apps. This board contains 18 runs.

- **The leading choice (44%).** Neon was chosen in 8 runs. Amazon RDS for PostgreSQL followed with 5 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Neon (neon.tech) | 8 | 44% |
| 2 | Amazon RDS for PostgreSQL (aws.amazon.com) | 5 | 28% |
| 3 | Azure Database for PostgreSQL Flexible Server (azure.microsoft.com) | 2 | 11% |
| 4 | Supabase (supabase.com) | 1 | 6% |
| 5 | Azure SQL Database (azure.microsoft.com) | 1 | 6% |
| 6 | Turso (turso.tech) | 1 | 6% |

## Deploy

<a id="deploy"></a>Render leads the product choices. 30 runs on 6 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/deploy

We asked GPT-6.2 Astra to deploy 6 apps. This board contains 30 runs.

- **The leading choice (40%).** Render was chosen in 12 runs. Vercel followed with 9 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Render (render.com) | 12 | 40% |
| 2 | Vercel (vercel.com) | 9 | 30% |
| 3 | Cloudflare (cloudflare.com) | 4 | 13% |
| 4 | Netlify (netlify.com) | 2 | 7% |
| 5 | Google Cloud (cloud.google.com) | 2 | 7% |
| 6 | GitHub Pages (pages.github.com) | 1 | 3% |

## Document processing & OCR

<a id="document-processing"></a>Azure AI Document Intelligence leads the product choices. 59 runs on 12 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/document-processing

We asked GPT-6.2 Astra to read the documents that arrive at 12 apps. This board contains 59 runs.

- **The leading choice (36%).** Azure AI Document Intelligence was chosen in 21 runs. Docling followed with 7 runs.
- **The project affects the choice (12 apps).** We tested 12 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Azure AI Document Intelligence (azure.microsoft.com) | 21 | 36% |
| 2 | Docling (docling-project.github.io) | 7 | 12% |
| 3 | Amazon Textract (aws.amazon.com) | 5 | 8% |
| 4 | Ocrolus (ocrolus.com) | 5 | 8% |
| 5 | Extend (extend.ai) | 4 | 7% |
| 6 | Reducto (reducto.ai) | 4 | 7% |
| 7 | Google Cloud Document AI (cloud.google.com) | 3 | 5% |
| 8 | LlamaParse (llamaindex.ai) | 3 | 5% |
| 9 | Veryfi (veryfi.com) | 3 | 5% |
| 10 | Indico Data (indicodata.ai) | 1 | 2% |
| | 3 more on the board | | |

## E-signature

<a id="e-signature"></a>DocuSign leads the product choices. 18 runs on 9 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/e-signature

We asked GPT-6.2 Astra to add e-signature to 9 apps. This board contains 18 runs.

- **The leading choice (56%).** DocuSign was chosen in 10 runs. Dropbox Sign followed with 3 runs.
- **The project affects the choice (9 apps).** We tested 9 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | DocuSign (docusign.com) | 10 | 56% |
| 2 | Dropbox Sign (sign.dropbox.com) | 3 | 17% |
| 3 | DocuSeal (docuseal.com) | 2 | 11% |
| 4 | YouTrust (youtrust.com) | 1 | 6% |
| 5 | Docuseal Pro On Premises | 1 | 6% |
| 6 | DocuSeal (self-hosted) (github.com) | 1 | 6% |

## LLM evals & observability

<a id="evals"></a>GPT-6.2 Astra most often built it in the app. 24 runs on 2 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/evals

We asked GPT-6.2 Astra to add LLM evals to 2 apps. This board contains 24 runs.

- **Building in the app comes first (25%).** The agents built the capability in the app in 6 runs. This was more common than any single product choice.
- **The project affects the choice (2 apps).** We tested 2 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 6 | 25% |
| 2 | Cloudflare AI Gateway (cloudflare.com) | 4 | 17% |
| 3 | Braintrust (braintrust.dev) | 3 | 13% |
| 4 | Arize Phoenix (arize.com) | 3 | 13% |
| 5 | LangSmith (smith.langchain.com) | 3 | 13% |
| 6 | Promptfoo (promptfoo.dev) | 3 | 13% |
| 7 | Langfuse (langfuse.com) | 1 | 4% |
| 8 | Portkey (portkey.ai) | 1 | 4% |

## In-app chat & calls

<a id="in-app-communication"></a>Stream Chat and Video leads the product choices. 18 runs on 6 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/in-app-communication

We asked GPT-6.2 Astra to add in-app chat and calls to 6 apps. This board contains 18 runs.

- **The leading choice (33%).** Stream Chat and Video was chosen in 6 runs. Daily followed with 3 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Stream Chat and Video (getstream.io) | 6 | 33% |
| 2 | Daily (daily.co) | 3 | 17% |
| 3 | Supabase (supabase.com) | 3 | 17% |
| 4 | Twilio (twilio.com) | 2 | 11% |
| 5 | LiveKit + Stream Chat and Video | 1 | 6% |
| 6 | LiveKit (livekit.io) | 1 | 6% |
| 7 | Ably + Stream Video Audio | 1 | 6% |
| 8 | TalkJS (talkjs.com) | 1 | 6% |

## Internationalization

<a id="internationalization"></a>next-intl leads the product choices. 36 runs on 12 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/internationalization

We asked GPT-6.2 Astra to add internationalization to 12 apps. This board contains 36 runs.

- **The leading choice (11%).** next-intl was chosen in 4 runs. Rails I18n followed with 3 runs.
- **The project affects the choice (12 apps).** We tested 12 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | next-intl (next-intl.dev) | 4 | 11% |
| 2 | Rails I18n (guides.rubyonrails.org) | 3 | 8% |
| 3 | Symfony Translation (symfony.com) | 3 | 8% |
| 4 | Django translation framework (docs.djangoproject.com) | 3 | 8% |
| 5 | i18next (i18next.com) | 3 | 8% |
| 6 | Paraglide JS (inlang.com) | 3 | 8% |
| 7 | Django translation framework + Weblate | 3 | 8% |
| 8 | Weblate (weblate.org) | 2 | 6% |
| 9 | Crowdin + next-intl | 2 | 6% |
| 10 | FormatJS (formatjs.github.io) | 2 | 6% |
| | 7 more on the board | | |

## Email providers

<a id="mail"></a>Resend leads the product choices. 17 runs on 9 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/mail

We asked GPT-6.2 Astra to add email sending to 9 apps. This board contains 17 runs.

- **The leading choice (41%).** Resend was chosen in 7 runs. Postmark followed with 3 runs.
- **The project affects the choice (9 apps).** We tested 9 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Resend (resend.com) | 7 | 41% |
| 2 | Postmark (postmarkapp.com) | 3 | 18% |
| 3 | Amazon SES (aws.amazon.com) | 2 | 12% |
| 4 | SendGrid (sendgrid.com) | 2 | 12% |
| 5 | Generic SMTP (nodemailer.com) | 2 | 12% |
| 6 | Azure Communication Services (azure.microsoft.com) | 1 | 6% |

## Maps

<a id="maps"></a>Google Maps shares the lead. 59 runs on 14 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/maps

We asked GPT-6.2 Astra to add a map to 14 apps. This board contains 59 runs.

- **The leading products are tied (14%).** Google Maps was chosen in 8 runs. Apple Maps (MapKit) also won 8 runs.
- **The project affects the choice (14 apps).** We tested 14 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Google Maps (mapsplatform.google.com) | 8 | 14% |
| 2 | Apple Maps (MapKit) (developer.apple.com) | 8 | 14% |
| 3 | Leaflet (leafletjs.com) | 4 | 7% |
| 4 | Base Adresse Nationale (adresse.data.gouv.fr) | 4 | 7% |
| 5 | Leaflet + Maa-amet (Estonian Land Board) | 3 | 5% |
| 6 | Azure Maps (azure.microsoft.com) | 3 | 5% |
| 7 | react-native-maps (github.com) | 3 | 5% |
| 8 | OpenStreetMap (openstreetmap.org) | 3 | 5% |
| 9 | Mapbox (mapbox.com) | 3 | 5% |
| 10 | MapTiler (maptiler.com) | 2 | 3% |
| | 16 more on the board | | |

## MCP gateway

<a id="mcp-gateway"></a>agentgateway shares the lead. 15 runs on 5 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/mcp-gateway

We asked GPT-6.2 Astra to add mcp gateway to 5 apps. This board contains 15 runs.

- **The leading products are tied (27%).** agentgateway was chosen in 4 runs. Amazon Bedrock AgentCore Gateway also won 4 runs.
- **The project affects the choice (5 apps).** We tested 5 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | agentgateway (agentgateway.dev) | 4 | 27% |
| 2 | Amazon Bedrock AgentCore Gateway (aws.amazon.com) | 4 | 27% |
| 3 | MintMCP Gateway (mintmcp.com) | 2 | 13% |
| 4 | IBM ContextForge (ibm.com) | 1 | 7% |
| 5 | Kong MCP Gateway (konghq.com) | 1 | 7% |
| 6 | Withhuman Mcp Gateway | 1 | 7% |
| 7 | Mintive | 1 | 7% |
| 8 | Built in-house (outcome) | 1 | 7% |

## Message queues

<a id="message-queues"></a>GPT-6.2 Astra most often built it in the app. 45 runs on 12 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/message-queues

We asked GPT-6.2 Astra to add a message queue to 12 apps. This board contains 45 runs.

- **Building in the app comes first (20%).** The agents built the capability in the app in 9 runs. This was more common than any single product choice.
- **The project affects the choice (12 apps).** We tested 12 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 9 | 20% |
| 2 | Apache Kafka (kafka.apache.org) | 7 | 16% |
| 3 | Amazon SQS (aws.amazon.com) | 5 | 11% |
| 4 | Laravel Queues (laravel.com) | 4 | 9% |
| 5 | Service Bus (azure.microsoft.com) | 4 | 9% |
| 6 | Amazon Kinesis (aws.amazon.com) | 4 | 9% |
| 7 | Cloud Tasks (cloud.google.com) | 3 | 7% |
| 8 | Cloudflare Queues (cloudflare.com) | 3 | 7% |
| 9 | Graphile Worker (worker.graphile.org) | 2 | 4% |
| 10 | Inngest (inngest.com) | 2 | 4% |
| | 2 more on the board | | |

## Model choice

<a id="model-choice"></a>OpenAI (audio) leads the product choices. 9 runs on 3 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/model-choice

We asked GPT-6.2 Astra to add model choice to 3 apps. This board contains 9 runs.

- **The leading choice (100%).** OpenAI (audio) was chosen in 9 runs. No other product won a run.
- **The project affects the choice (3 apps).** We tested 3 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | OpenAI (audio) (openai.com) | 9 | 100% |

## Observability

<a id="observability"></a>Sentry leads the product choices. 30 runs on 11 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/observability

We asked GPT-6.2 Astra to add observability to 11 apps. This board contains 30 runs.

- **The leading choice (27%).** Sentry was chosen in 8 runs. New Relic followed with 5 runs.
- **The project affects the choice (11 apps).** We tested 11 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Sentry (sentry.io) | 8 | 27% |
| 2 | New Relic (newrelic.com) | 5 | 17% |
| 3 | Amazon CloudWatch (aws.amazon.com) | 3 | 10% |
| 4 | Checkly (checklyhq.com) | 3 | 10% |
| 5 | Grafana (grafana.com) | 3 | 10% |
| 6 | Honeycomb (honeycomb.io) | 3 | 10% |
| 7 | Amazon CloudWatch + AWS X-Ray | 1 | 3% |
| 8 | Bugsink (bugsink.com) | 1 | 3% |
| 9 | Azure Application Insights (azure.microsoft.com) | 1 | 3% |
| 10 | Vector | 1 | 3% |

## OCR

<a id="ocr"></a>Azure AI Document Intelligence leads the product choices. 15 runs on 5 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/ocr

We asked GPT-6.2 Astra to add ocr to 5 apps. This board contains 15 runs.

- **The leading choice (40%).** Azure AI Document Intelligence was chosen in 6 runs. Amazon Textract followed with 5 runs.
- **The project affects the choice (5 apps).** We tested 5 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Azure AI Document Intelligence (azure.microsoft.com) | 6 | 40% |
| 2 | Amazon Textract (aws.amazon.com) | 5 | 33% |
| 3 | Tesseract OCR (github.com) | 3 | 20% |
| 4 | PaddleOCR (paddleocr.ai) | 1 | 7% |

## Payments

<a id="payments"></a>Stripe leads the product choices. 32 runs on 11 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/payments

We asked GPT-6.2 Astra to add payments to 11 apps. This board contains 32 runs.

- **The leading choice (81%).** Stripe was chosen in 26 runs. Paddle followed with 1 run.
- **The project affects the choice (11 apps).** We tested 11 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Stripe (stripe.com) | 26 | 81% |
| 2 | Paddle (paddle.com) | 1 | 3% |
| 3 | Adyen (adyen.com) | 1 | 3% |
| 4 | Mollie (mollie.com) | 1 | 3% |
| 5 | GoCardless (gocardless.com) | 1 | 3% |
| 6 | Zelle | 1 | 3% |
| 7 | Built in-house (outcome) | 1 | 3% |

## Performance in CI

<a id="perf-ci"></a>GPT-6.2 Astra most often built it in the app. 18 runs on 8 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/perf-ci

We asked GPT-6.2 Astra to add a performance gate to 8 apps. This board contains 18 runs.

- **Building in the app comes first (67%).** The agents built the capability in the app in 12 runs. This was more common than any single product choice.
- **The project affects the choice (8 apps).** We tested 8 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 12 | 67% |
| 2 | Grafana k6 (k6.io) | 3 | 17% |
| 3 | JMH (openjdk.org) | 2 | 11% |
| 4 | hyperfine (github.com) | 1 | 6% |

## Product analytics

<a id="product-analytics"></a>PostHog leads the product choices. 30 runs on 10 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/product-analytics

We asked GPT-6.2 Astra to add product analytics to 10 apps. This board contains 30 runs.

- **The leading choice (57%).** PostHog was chosen in 17 runs. Metabase followed with 7 runs.
- **The project affects the choice (10 apps).** We tested 10 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | PostHog (posthog.com) | 17 | 57% |
| 2 | Metabase (metabase.com) | 7 | 23% |
| 3 | Mixpanel (mixpanel.com) | 2 | 7% |
| 4 | Umami (umami.is) | 1 | 3% |

## Agent sandboxes

<a id="sandboxes"></a>Vercel Sandbox leads the product choices. 17 runs on 5 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/sandboxes

We asked GPT-6.2 Astra to add a code sandbox to 5 apps. This board contains 17 runs.

- **The leading choice (59%).** Vercel Sandbox was chosen in 10 runs. Modal followed with 7 runs.
- **The project affects the choice (5 apps).** We tested 5 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Vercel Sandbox (vercel.com) | 10 | 59% |
| 2 | Modal (modal.com) | 7 | 41% |

## Search

<a id="search"></a>GPT-6.2 Astra most often built it in the app. 53 runs on 11 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/search

We asked GPT-6.2 Astra to add search to 11 apps. This board contains 53 runs.

- **Building in the app comes first (28%).** The agents built the capability in the app in 15 runs. This was more common than any single product choice.
- **The project affects the choice (11 apps).** We tested 11 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 15 | 28% |
| 2 | pg_trgm (postgresql.org) | 8 | 15% |
| 3 | OpenSearch (opensearch.org) | 8 | 15% |
| 4 | Typesense (typesense.org) | 5 | 9% |
| 5 | Elasticsearch (elastic.co) | 4 | 8% |
| 6 | Atlas Search (mongodb.com) | 3 | 6% |
| 7 | Fuse.js (fusejs.io) | 2 | 4% |
| 8 | MongoDB Text Search (mongodb.com) | 2 | 4% |
| 9 | Meilisearch (meilisearch.com) | 2 | 4% |
| 10 | Algolia (algolia.com) | 1 | 2% |
| | 3 more on the board | | |

## Serverless functions

<a id="serverless"></a>AWS Lambda leads the product choices. 24 runs on 8 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/serverless

We asked GPT-6.2 Astra to add serverless functions to 8 apps. This board contains 24 runs.

- **The leading choice (38%).** AWS Lambda was chosen in 9 runs. Cloudflare Workers followed with 4 runs.
- **The project affects the choice (8 apps).** We tested 8 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | AWS Lambda (aws.amazon.com) | 9 | 38% |
| 2 | Cloudflare Workers (workers.cloudflare.com) | 4 | 17% |
| 3 | Netlify Functions (netlify.com) | 3 | 13% |
| 4 | Azure Functions (azure.microsoft.com) | 3 | 13% |
| 5 | Vercel Functions (vercel.com) | 2 | 8% |
| 6 | Trigger.dev (trigger.dev) | 1 | 4% |
| 7 | Firebase Cloud Functions | 1 | 4% |
| 8 | Upstash QStash (upstash.com) | 1 | 4% |

## File storage

<a id="storage"></a>Amazon S3 leads the product choices. 24 runs on 8 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/storage

We asked GPT-6.2 Astra to add file storage to 8 apps. This board contains 24 runs.

- **The leading choice (58%).** Amazon S3 was chosen in 14 runs. Supabase Storage followed with 3 runs.
- **The project affects the choice (8 apps).** We tested 8 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Amazon S3 (aws.amazon.com) | 14 | 58% |
| 2 | Supabase Storage (supabase.com) | 3 | 13% |
| 3 | Azure Blob Storage (azure.microsoft.com) | 3 | 13% |
| 4 | Google Cloud Storage (cloud.google.com) | 3 | 13% |
| 5 | Cloudflare R2 (cloudflare.com) | 1 | 4% |

## Usage-based billing

<a id="usage-based-billing"></a>Metronome leads the product choices. 33 runs on 11 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/usage-based-billing

We asked GPT-6.2 Astra to bill customers for what they use in 11 apps. This board contains 33 runs.

- **The leading choice (58%).** Metronome was chosen in 19 runs. Oracle BRM followed with 4 runs.
- **The project affects the choice (11 apps).** We tested 11 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Metronome (metronome.com) | 19 | 58% |
| 2 | Oracle BRM (oracle.com) | 4 | 12% |
| 3 | Lago (getlago.com) | 4 | 12% |
| 4 | Orb (withorb.com) | 4 | 12% |
| 5 | CGRateS (github.com) | 1 | 3% |
| 6 | PortaBilling | 1 | 3% |

## Vector search

<a id="vector-search"></a>Neon leads the product choices. 16 runs on 6 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/vector-search

We asked GPT-6.2 Astra to add vector search to 6 apps. This board contains 16 runs.

- **The leading choice (31%).** Neon was chosen in 5 runs. Qdrant followed with 3 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Neon (neon.tech) | 5 | 31% |
| 2 | Qdrant (qdrant.tech) | 3 | 19% |
| 3 | OpenAI Vector Stores (openai.com) | 3 | 19% |
| 4 | Typesense (typesense.org) | 3 | 19% |
| 5 | Azure AI Search (azure.microsoft.com) | 1 | 6% |
| 6 | Pinecone (pinecone.io) | 1 | 6% |

## Voice

<a id="voice"></a>Deepgram leads the product choices. 9 runs on 6 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/voice

We asked GPT-6.2 Astra to add voice to 6 apps. This board contains 9 runs.

- **The leading choice (33%).** Deepgram was chosen in 3 runs. OpenAI (audio) followed with 1 run.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Deepgram (deepgram.com) | 3 | 33% |
| 2 | OpenAI (audio) (openai.com) | 1 | 11% |
| 3 | Gemini 3 8 Live | 1 | 11% |
| 4 | Web Speech API (developer.mozilla.org) | 1 | 11% |
| 5 | Google Cloud Speech (cloud.google.com) | 1 | 11% |
| 6 | Piper (github.com) | 1 | 11% |
| 7 | ElevenLabs (elevenlabs.io) | 1 | 11% |

## Voice Agents

<a id="voice-agents"></a>Retell AI leads the product choices. 44 runs on 10 apps, updated 2026-11-17. Board: https://armature.tech/leaderboards/astra-6-2#app/voice-agents

We asked GPT-6.2 Astra to add a voice agent to 10 apps. This board contains 44 runs.

- **The leading choice (50%).** Retell AI was chosen in 22 runs. LiveKit Agents followed with 7 runs.
- **The project affects the choice (10 apps).** We tested 10 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Retell AI (retellai.com) | 22 | 50% |
| 2 | LiveKit Agents (livekit.io) | 7 | 16% |
| 3 | ElevenLabs Agents (elevenlabs.io) | 6 | 14% |
| 4 | Vapi (vapi.ai) | 3 | 7% |
| 5 | OpenAI Realtime API (openai.com) | 2 | 5% |
| 6 | Twilio ConversationRelay (twilio.com) | 1 | 2% |
| 7 | OpenAI GPT-Live (openai.com) | 1 | 2% |
| 8 | Cognigy (cognigy.com) | 1 | 2% |
| 9 | Built in-house (outcome) | 1 | 2% |

## How this was measured

Every number comes from a controlled experiment. GPT-6.2 Astra ran in Codex on 97 existing codebases, asked in several wordings and as different kinds of user to add one capability (the sector) and to choose the product itself. Each run happened in a sandbox at a pinned agent version, and a judge read the session to record the choice. A choice counts when the session captured it and the judge confirmed it, even when the implementation was incomplete; runs without captured evidence are left out. Shares are wins over all runs in the sector. Several tools adopted together count as one combination. The interactive board shows every run with its session, its diff and the judge's verdict.

Methodology and publications: https://armature.tech/publications

## Other boards

- All agents, sector by sector: https://armature.tech/leaderboards/sectors (Markdown: https://armature.tech/leaderboards/sectors.md)
- GPT-6.2 Sol: https://armature.tech/leaderboards/sol-6-2 (Markdown: https://armature.tech/leaderboards/sol-6-2.md)
- Claude Fable 5.2: https://armature.tech/leaderboards/fable-5-2 (Markdown: https://armature.tech/leaderboards/fable-5-2.md)
- Muse Ember: https://armature.tech/leaderboards/muse-ember (Markdown: https://armature.tech/leaderboards/muse-ember.md)
- Muse Wonder: https://armature.tech/leaderboards/muse-wonder (Markdown: https://armature.tech/leaderboards/muse-wonder.md)
- Grok 4.7: https://armature.tech/leaderboards/grok-4-7 (Markdown: https://armature.tech/leaderboards/grok-4-7.md)
- Claude Opus 5.5: https://armature.tech/leaderboards/opus-5-5 (Markdown: https://armature.tech/leaderboards/opus-5-5.md)
- GPT-6.1 Sol: https://armature.tech/leaderboards/sol-6-1 (Markdown: https://armature.tech/leaderboards/sol-6-1.md)
- Muse Spark 1.3: https://armature.tech/leaderboards/muse-spark-1-3 (Markdown: https://armature.tech/leaderboards/muse-spark-1-3.md)
- GPT-6 Sol: https://armature.tech/leaderboards/sol-6 (Markdown: https://armature.tech/leaderboards/sol-6.md)
