# Muse Ember leaderboards: which developer tools Muse Ember chooses in Muse Code

Source: https://armature.tech/leaderboards/muse-ember (also https://leaderboards.armature.tech/muse-ember), Armature agent leaderboards. 923 runs, 33 sectors, 94 apps, one model (Muse Ember) in one agent (Muse Code), updated 2026-11-09. Interactive board with every run: https://armature.tech/leaderboards/muse-ember#app

We tested Muse Ember in Muse Code across 33 sectors. These boards contain 923 runs.

- Open a sector to see its product rankings, key learnings, and evidence. Open a product to see where it wins and loses.
- The test set is the Muse Ember campaign manifest, with one run per prompt input. The agent frameworks and agent runtime sectors are left out.
- Product choices count even when the implementation was incomplete. Runs without captured evidence are not counted.

## Sectors at a glance

| Sector | Runs | First choice | Share |
|---|---:|---|---:|
| [AI gateway](#ai-gateway) | 24 | OpenRouter | 58% |
| [AI search](#ai-search) | 47 | Tavily | 32% |
| [AI SRE](#ai-sre) | 18 | Sentry Seer | 22% |
| [Auth](#auth) | 67 | Clerk | 22% |
| [Background jobs](#background-jobs) | 15 | Celery | 13% |
| [Bot protection](#bot-protection) | 15 | Cloudflare Turnstile | 27% |
| [Cloud](#cloud) | 18 | AWS | 67% |
| [Code review](#code-review) | 30 | CodeRabbit | 40% |
| [Databases](#databases) | 18 | Neon | 72% |
| [Deploy](#deploy) | 30 | Render | 33% |
| [Document processing & OCR](#document-processing) | 60 | Azure AI Document Intelligence | 33% |
| [E-signature](#e-signature) | 17 | DocuSign | 35% |
| [LLM evals & observability](#evals) | 24 | Langfuse | 21% |
| [In-app chat & calls](#in-app-communication) | 18 | Daily | 22% |
| [Internationalization](#internationalization) | 36 | next-intl | 22% |
| [Email providers](#mail) | 18 | Resend | 39% |
| [Maps](#maps) | 57 | Leaflet | 19% |
| [MCP gateway](#mcp-gateway) | 15 | agentgateway | 7% |
| [Message queues](#message-queues) | 48 | Apache Kafka | 13% |
| [Model choice](#model-choice) | 9 | OpenAI (audio) | 78% |
| [Observability](#observability) | 30 | Grafana | 33% |
| [OCR](#ocr) | 12 | Amazon Textract | 42% |
| [Payments](#payments) | 33 | Stripe | 91% |
| [Performance in CI](#perf-ci) | 18 | JMH | 11% |
| [Product analytics](#product-analytics) | 30 | PostHog | 60% |
| [Agent sandboxes](#sandboxes) | 17 | E2B | 100% |
| [Search](#search) | 51 | Postgres Full-Text Search | 22% |
| [Serverless functions](#serverless) | 22 | Vercel Functions | 18% |
| [File storage](#storage) | 23 | Amazon S3 | 74% |
| [Usage-based billing](#usage-based-billing) | 33 | Lago | 12% |
| [Vector search](#vector-search) | 16 | Neon | 38% |
| [Voice](#voice) | 9 | Microsoft Azure | 22% |
| [Voice Agents](#voice-agents) | 45 | Retell AI | 47% |

## AI gateway

<a id="ai-gateway"></a>OpenRouter leads the product choices. 24 runs on 8 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/ai-gateway

We asked Muse Ember to add an AI gateway to 8 apps. This board contains 24 runs.

- **The leading choice (58%).** OpenRouter was chosen in 14 runs. Vercel AI Gateway followed with 3 runs.
- **The project affects the choice (8 apps).** We tested 8 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | OpenRouter (openrouter.ai) | 14 | 58% |
| 2 | Vercel AI Gateway (vercel.com) | 3 | 13% |
| 3 | Portkey (portkey.ai) | 3 | 13% |
| 4 | Helicone (helicone.ai) | 1 | 4% |
| 5 | LiteLLM (litellm.ai) | 1 | 4% |
| 6 | Built in-house (outcome) | 1 | 4% |

## AI search

<a id="ai-search"></a>Tavily leads the product choices. 47 runs on 12 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/ai-search

We asked Muse Ember to get live web information into 12 apps. This board contains 47 runs.

- **The leading choice (32%).** Tavily was chosen in 15 runs. Perplexity Sonar followed with 4 runs.
- **The project affects the choice (12 apps).** We tested 12 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Tavily (tavily.com) | 15 | 32% |
| 2 | Built in-house (outcome) | 6 | 13% |
| 3 | Perplexity Sonar (perplexity.ai) | 4 | 9% |
| 4 | Exa (exa.ai) | 4 | 9% |
| 5 | Brave Search API (brave.com) | 4 | 9% |
| 6 | Apify (apify.com) | 2 | 4% |
| 7 | Complyadvantage | 2 | 4% |
| 8 | Bing grounding (azure.microsoft.com) | 1 | 2% |
| 9 | Bing Search API (microsoft.com) | 1 | 2% |
| 10 | Serper (serper.dev) | 1 | 2% |
| | 7 more on the board | | |

## AI SRE

<a id="ai-sre"></a>Sentry Seer leads the product choices. 18 runs on 6 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/ai-sre

We asked Muse Ember to add automatic incident investigation and fixes to 6 apps. This board contains 18 runs.

- **The leading choice (22%).** Sentry Seer was chosen in 4 runs. Datadog Bits AI Dev Agent + Datadog Bits Investigation followed with 3 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Sentry Seer (sentry.io) | 4 | 22% |
| 2 | Datadog Bits AI Dev Agent + Datadog Bits Investigation | 3 | 17% |
| 3 | Resolve AI (resolve.ai) | 3 | 17% |
| 4 | AWS DevOps Agent (aws.amazon.com) | 2 | 11% |
| 5 | HolmesGPT (holmesgpt.dev) | 1 | 6% |
| 6 | Grafana Assistant Investigations (grafana.com) | 1 | 6% |
| 7 | Amazon Q Developer Agent | 1 | 6% |

## Auth

<a id="auth"></a>Clerk leads the product choices. 67 runs on 17 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/auth

We asked Muse Ember to add auth to 17 apps. This board contains 67 runs.

- **The leading choice (22%).** Clerk was chosen in 15 runs. Auth0 followed with 9 runs.
- **The project affects the choice (17 apps).** We tested 17 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Clerk (clerk.com) | 15 | 22% |
| 2 | Auth0 (auth0.com) | 9 | 13% |
| 3 | WorkOS AuthKit (workos.com) | 8 | 12% |
| 4 | Microsoft Entra ID (entra.microsoft.com) | 7 | 10% |
| 5 | Keycloak (keycloak.org) | 5 | 7% |
| 6 | Built in-house (outcome) | 5 | 7% |
| 7 | Amazon Cognito (aws.amazon.com) | 4 | 6% |
| 8 | Better Auth (better-auth.com) | 2 | 3% |
| 9 | Zitadel (zitadel.com) | 1 | 1% |
| 10 | Google Sign-In (developers.google.com) | 1 | 1% |
| | 6 more on the board | | |

## Background jobs

<a id="background-jobs"></a>Muse Ember most often built it in the app. 15 runs on 10 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/background-jobs

We asked Muse Ember to add background jobs to 10 apps. This board contains 15 runs.

- **Building in the app comes first (60%).** The agents built the capability in the app in 9 runs. This was more common than any single product choice.
- **The project affects the choice (10 apps).** We tested 10 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 9 | 60% |
| 2 | Celery (celeryq.dev) | 2 | 13% |
| 3 | Inngest (inngest.com) | 2 | 13% |
| 4 | Solid Queue (github.com) | 1 | 7% |
| 5 | Amazon SQS (aws.amazon.com) | 1 | 7% |

## Bot protection

<a id="bot-protection"></a>Muse Ember most often built it in the app. 15 runs on 5 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/bot-protection

We asked Muse Ember to add bot protection to 5 apps. This board contains 15 runs.

- **Building in the app comes first (53%).** The agents built the capability in the app in 8 runs. This was more common than any single product choice.
- **The project affects the choice (5 apps).** We tested 5 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 8 | 53% |
| 2 | Cloudflare Turnstile (cloudflare.com) | 4 | 27% |
| 3 | django-simple-captcha (github.com) | 2 | 13% |
| 4 | django-axes (github.com) | 1 | 7% |

## Cloud

<a id="cloud"></a>AWS leads the product choices. 18 runs on 6 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/cloud

We asked Muse Ember to pick a cloud for 6 apps. This board contains 18 runs.

- **The leading choice (67%).** AWS was chosen in 12 runs. Google Cloud followed with 3 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | AWS (aws.amazon.com) | 12 | 67% |
| 2 | Google Cloud (cloud.google.com) | 3 | 17% |
| 3 | Upstash (upstash.com) | 2 | 11% |
| 4 | Cloudflare (cloudflare.com) | 1 | 6% |

## Code review

<a id="code-review"></a>CodeRabbit leads the product choices. 30 runs on 10 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/code-review

We asked Muse Ember to add automated code review to 10 apps. This board contains 30 runs.

- **The leading choice (40%).** CodeRabbit was chosen in 12 runs. Qodo Merge followed with 4 runs.
- **The project affects the choice (10 apps).** We tested 10 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | CodeRabbit (coderabbit.ai) | 12 | 40% |
| 2 | Qodo Merge (qodo.ai) | 4 | 13% |
| 3 | Built in-house (outcome) | 3 | 10% |
| 4 | Semgrep (semgrep.dev) | 2 | 7% |
| 5 | SonarQube (sonarsource.com) | 2 | 7% |
| 6 | GitHub Copilot code review (github.com) | 2 | 7% |
| 7 | reviewdog + Semgrep | 1 | 3% |
| 8 | Prxref | 1 | 3% |
| 9 | reviewdog (github.com) | 1 | 3% |
| 10 | Muse Code (dev.meta.ai) | 1 | 3% |

## Databases

<a id="databases"></a>Neon leads the product choices. 18 runs on 6 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/databases

We asked Muse Ember to pick a database for 6 apps. This board contains 18 runs.

- **The leading choice (72%).** Neon was chosen in 13 runs. Azure Database for PostgreSQL Flexible Server followed with 2 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Neon (neon.tech) | 13 | 72% |
| 2 | Azure Database for PostgreSQL Flexible Server (azure.microsoft.com) | 2 | 11% |
| 3 | MotherDuck (motherduck.com) | 1 | 6% |
| 4 | Scaleway Managed Database for PostgreSQL (scaleway.com) | 1 | 6% |
| 5 | Azure SQL Database (azure.microsoft.com) | 1 | 6% |

## Deploy

<a id="deploy"></a>Render shares the lead. 30 runs on 6 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/deploy

We asked Muse Ember to deploy 6 apps. This board contains 30 runs.

- **The leading products are tied (33%).** Render was chosen in 10 runs. Vercel also won 10 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Render (render.com) | 10 | 33% |
| 2 | Vercel (vercel.com) | 10 | 33% |
| 3 | Cloudflare (cloudflare.com) | 3 | 10% |
| 4 | Fly.io (fly.io) | 3 | 10% |
| 5 | Netlify (netlify.com) | 2 | 7% |
| 6 | Railway (railway.app) | 1 | 3% |
| 7 | GitHub Pages (pages.github.com) | 1 | 3% |

## Document processing & OCR

<a id="document-processing"></a>Azure AI Document Intelligence leads the product choices. 60 runs on 12 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/document-processing

We asked Muse Ember to read the documents that arrive at 12 apps. This board contains 60 runs.

- **The leading choice (33%).** Azure AI Document Intelligence was chosen in 20 runs. Anthropic Claude followed with 13 runs.
- **The project affects the choice (12 apps).** We tested 12 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Azure AI Document Intelligence (azure.microsoft.com) | 20 | 33% |
| 2 | Anthropic Claude (anthropic.com) | 13 | 22% |
| 3 | OpenAI models (openai.com) | 7 | 12% |
| 4 | Amazon Textract (aws.amazon.com) | 4 | 7% |
| 5 | Apache PDFBox (pdfbox.apache.org) | 4 | 7% |
| 6 | Google Gemini (ai.google.dev) | 2 | 3% |
| 7 | Azure AI Document Intelligence + OpenAI models | 2 | 3% |
| 8 | Docling (docling-project.github.io) | 1 | 2% |
| 9 | Mindee (mindee.com) | 1 | 2% |
| 10 | Google Cloud Vision Api | 1 | 2% |
| | 5 more on the board | | |

## E-signature

<a id="e-signature"></a>DocuSign leads the product choices. 17 runs on 9 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/e-signature

We asked Muse Ember to add e-signature to 9 apps. This board contains 17 runs.

- **The leading choice (35%).** DocuSign was chosen in 6 runs. Dropbox Sign followed with 3 runs.
- **The project affects the choice (9 apps).** We tested 9 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | DocuSign (docusign.com) | 6 | 35% |
| 2 | Built in-house (outcome) | 4 | 24% |
| 3 | Dropbox Sign (sign.dropbox.com) | 3 | 18% |
| 4 | Documenso (documenso.com) | 2 | 12% |

## LLM evals & observability

<a id="evals"></a>Muse Ember most often built it in the app. 24 runs on 2 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/evals

We asked Muse Ember to add LLM evals to 2 apps. This board contains 24 runs.

- **Building in the app comes first (58%).** The agents built the capability in the app in 14 runs. This was more common than any single product choice.
- **The project affects the choice (2 apps).** We tested 2 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 14 | 58% |
| 2 | Langfuse (langfuse.com) | 5 | 21% |
| 3 | Promptfoo (promptfoo.dev) | 3 | 13% |
| 4 | Arize Phoenix (arize.com) | 1 | 4% |
| 5 | DeepEval (confident-ai.com) | 1 | 4% |

## In-app chat & calls

<a id="in-app-communication"></a>Daily leads the product choices. 18 runs on 6 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/in-app-communication

We asked Muse Ember to add in-app chat and calls to 6 apps. This board contains 18 runs.

- **The leading choice (22%).** Daily was chosen in 4 runs. Stream Chat and Video followed with 3 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Daily (daily.co) | 4 | 22% |
| 2 | Stream Chat and Video (getstream.io) | 3 | 17% |
| 3 | Supabase (supabase.com) | 3 | 17% |
| 4 | LiveKit (livekit.io) | 2 | 11% |
| 5 | Ably (ably.com) | 1 | 6% |
| 6 | Twilio (twilio.com) | 1 | 6% |
| 7 | Jitsi Meet + Sendbird | 1 | 6% |
| 8 | Daily + Stream Chat and Video | 1 | 6% |
| 9 | Ably + LiveKit | 1 | 6% |
| 10 | Daily + Sendbird | 1 | 6% |

## Internationalization

<a id="internationalization"></a>next-intl leads the product choices. 36 runs on 12 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/internationalization

We asked Muse Ember to add internationalization to 12 apps. This board contains 36 runs.

- **The leading choice (22%).** next-intl was chosen in 8 runs. Paraglide JS followed with 3 runs.
- **The project affects the choice (12 apps).** We tested 12 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | next-intl (next-intl.dev) | 8 | 22% |
| 2 | Paraglide JS (inlang.com) | 3 | 8% |
| 3 | Rails I18n (guides.rubyonrails.org) | 3 | 8% |
| 4 | Symfony Translation (symfony.com) | 3 | 8% |
| 5 | Django translation framework (docs.djangoproject.com) | 3 | 8% |
| 6 | Vue I18n (vue-i18n.intlify.dev) | 3 | 8% |
| 7 | Built in-house (outcome) | 3 | 8% |
| 8 | Tolgee (tolgee.io) | 2 | 6% |
| 9 | Crowdin + Django translation framework | 2 | 6% |
| 10 | next-intl + Tolgee | 1 | 3% |
| | 5 more on the board | | |

## Email providers

<a id="mail"></a>Resend leads the product choices. 18 runs on 9 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/mail

We asked Muse Ember to add email sending to 9 apps. This board contains 18 runs.

- **The leading choice (39%).** Resend was chosen in 7 runs. Postmark followed with 4 runs.
- **The project affects the choice (9 apps).** We tested 9 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Resend (resend.com) | 7 | 39% |
| 2 | Postmark (postmarkapp.com) | 4 | 22% |
| 3 | Azure Communication Services (azure.microsoft.com) | 3 | 17% |
| 4 | SendGrid (sendgrid.com) | 2 | 11% |
| 5 | Amazon SES (aws.amazon.com) | 2 | 11% |

## Maps

<a id="maps"></a>Leaflet leads the product choices. 57 runs on 14 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/maps

We asked Muse Ember to add a map to 14 apps. This board contains 57 runs.

- **The leading choice (19%).** Leaflet was chosen in 11 runs. Google Maps followed with 10 runs.
- **The project affects the choice (14 apps).** We tested 14 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Leaflet (leafletjs.com) | 11 | 19% |
| 2 | Google Maps (mapsplatform.google.com) | 10 | 18% |
| 3 | Apple Maps (MapKit) (developer.apple.com) | 7 | 12% |
| 4 | MapLibre (maplibre.org) | 4 | 7% |
| 5 | OpenStreetMap (openstreetmap.org) | 4 | 7% |
| 6 | Leaflet + Nominatim + OpenStreetMap | 2 | 4% |
| 7 | react-native-maps (github.com) | 2 | 4% |
| 8 | osmdroid (github.com) | 2 | 4% |
| 9 | flutter_map (docs.fleaflet.dev) | 2 | 4% |
| 10 | Built in-house (outcome) | 2 | 4% |
| | 11 more on the board | | |

## MCP gateway

<a id="mcp-gateway"></a>Muse Ember most often built it in the app. 15 runs on 5 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/mcp-gateway

We asked Muse Ember to add mcp gateway to 5 apps. This board contains 15 runs.

- **Building in the app comes first (20%).** The agents built the capability in the app in 3 runs. This was more common than any single product choice.
- **The project affects the choice (5 apps).** We tested 5 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 3 | 20% |
| 2 | agentgateway (agentgateway.dev) | 1 | 7% |
| 3 | IBM ContextForge (ibm.com) | 1 | 7% |
| 4 | Auth0 Mcp Gateway | 1 | 7% |
| 5 | Docker MCP Gateway (docker.com) | 1 | 7% |
| 6 | Envoy AI Gateway (envoyproxy.io) | 1 | 7% |
| 7 | Stacklok Toolhive Vmcp | 1 | 7% |
| 8 | Kong MCP Gateway (konghq.com) | 1 | 7% |
| 9 | Portkey Mcp Gateway | 1 | 7% |
| 10 | Smithery (smithery.ai) | 1 | 7% |
| | 3 more on the board | | |

## Message queues

<a id="message-queues"></a>Muse Ember most often built it in the app. 48 runs on 12 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/message-queues

We asked Muse Ember to add a message queue to 12 apps. This board contains 48 runs.

- **Building in the app comes first (21%).** The agents built the capability in the app in 10 runs. This was more common than any single product choice.
- **The project affects the choice (12 apps).** We tested 12 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 10 | 21% |
| 2 | Apache Kafka (kafka.apache.org) | 6 | 13% |
| 3 | Amazon SQS (aws.amazon.com) | 5 | 10% |
| 4 | Service Bus (azure.microsoft.com) | 4 | 8% |
| 5 | Amazon Kinesis (aws.amazon.com) | 4 | 8% |
| 6 | Laravel Queues (laravel.com) | 3 | 6% |
| 7 | Cloud Tasks (cloud.google.com) | 3 | 6% |
| 8 | Inngest (inngest.com) | 2 | 4% |
| 9 | Aiven (aiven.io) | 1 | 2% |
| 10 | pg-boss (github.com) | 1 | 2% |
| | 9 more on the board | | |

## Model choice

<a id="model-choice"></a>OpenAI (audio) leads the product choices. 9 runs on 3 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/model-choice

We asked Muse Ember to add model choice to 3 apps. This board contains 9 runs.

- **The leading choice (78%).** OpenAI (audio) was chosen in 7 runs. Anthropic followed with 2 runs.
- **The project affects the choice (3 apps).** We tested 3 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | OpenAI (audio) (openai.com) | 7 | 78% |
| 2 | Anthropic (anthropic.com) | 2 | 22% |

## Observability

<a id="observability"></a>Grafana leads the product choices. 30 runs on 11 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/observability

We asked Muse Ember to add observability to 11 apps. This board contains 30 runs.

- **The leading choice (33%).** Grafana was chosen in 10 runs. Sentry followed with 7 runs.
- **The project affects the choice (11 apps).** We tested 11 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Grafana (grafana.com) | 10 | 33% |
| 2 | Sentry (sentry.io) | 7 | 23% |
| 3 | Amazon CloudWatch (aws.amazon.com) | 3 | 10% |
| 4 | Better Stack (betterstack.com) | 2 | 7% |
| 5 | UptimeRobot (uptimerobot.com) | 2 | 7% |
| 6 | Built in-house (outcome) | 2 | 7% |
| 7 | AWS X-Ray (aws.amazon.com) | 1 | 3% |
| 8 | GlitchTip (glitchtip.com) | 1 | 3% |
| 9 | Amazon CloudWatch + AWS X-Ray | 1 | 3% |
| 10 | Azure Monitor (azure.microsoft.com) | 1 | 3% |

## OCR

<a id="ocr"></a>Amazon Textract leads the product choices. 12 runs on 4 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/ocr

We asked Muse Ember to add ocr to 4 apps. This board contains 12 runs.

- **The leading choice (42%).** Amazon Textract was chosen in 5 runs. Azure AI Document Intelligence followed with 4 runs.
- **The project affects the choice (4 apps).** We tested 4 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Amazon Textract (aws.amazon.com) | 5 | 42% |
| 2 | Azure AI Document Intelligence (azure.microsoft.com) | 4 | 33% |
| 3 | OpenAI multimodal models (openai.com) | 1 | 8% |
| 4 | Anthropic Claude (anthropic.com) | 1 | 8% |
| 5 | Gemini | 1 | 8% |

## Payments

<a id="payments"></a>Stripe leads the product choices. 33 runs on 11 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/payments

We asked Muse Ember to add payments to 11 apps. This board contains 33 runs.

- **The leading choice (91%).** Stripe was chosen in 30 runs. Paddle followed with 1 run.
- **The project affects the choice (11 apps).** We tested 11 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Stripe (stripe.com) | 30 | 91% |
| 2 | Paddle (paddle.com) | 1 | 3% |
| 3 | GoCardless (gocardless.com) | 1 | 3% |
| 4 | Adyen (adyen.com) | 1 | 3% |

## Performance in CI

<a id="perf-ci"></a>Muse Ember most often built it in the app. 18 runs on 8 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/perf-ci

We asked Muse Ember to add a performance gate to 8 apps. This board contains 18 runs.

- **Building in the app comes first (72%).** The agents built the capability in the app in 13 runs. This was more common than any single product choice.
- **The project affects the choice (8 apps).** We tested 8 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 13 | 72% |
| 2 | JMH (openjdk.org) | 2 | 11% |
| 3 | Benchstat (github.com) | 1 | 6% |
| 4 | Autocannon (github.com) | 1 | 6% |
| 5 | BenchmarkDotNet (benchmarkdotnet.org) | 1 | 6% |

## Product analytics

<a id="product-analytics"></a>PostHog leads the product choices. 30 runs on 10 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/product-analytics

We asked Muse Ember to add product analytics to 10 apps. This board contains 30 runs.

- **The leading choice (60%).** PostHog was chosen in 18 runs. Metabase followed with 10 runs.
- **The project affects the choice (10 apps).** We tested 10 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | PostHog (posthog.com) | 18 | 60% |
| 2 | Metabase (metabase.com) | 10 | 33% |
| 3 | Looker Studio (lookerstudio.google.com) | 1 | 3% |
| 4 | Plausible (plausible.io) | 1 | 3% |

## Agent sandboxes

<a id="sandboxes"></a>E2B leads the product choices. 17 runs on 5 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/sandboxes

We asked Muse Ember to add a code sandbox to 5 apps. This board contains 17 runs.

- **The leading choice (100%).** E2B was chosen in 17 runs. No other product won a run.
- **The project affects the choice (5 apps).** We tested 5 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | E2B (e2b.dev) | 17 | 100% |

## Search

<a id="search"></a>Muse Ember most often built it in the app. 51 runs on 11 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/search

We asked Muse Ember to add search to 11 apps. This board contains 51 runs.

- **Building in the app comes first (24%).** The agents built the capability in the app in 12 runs. This was more common than any single product choice.
- **The project affects the choice (11 apps).** We tested 11 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 12 | 24% |
| 2 | Postgres Full-Text Search (postgresql.org) | 11 | 22% |
| 3 | OpenSearch (opensearch.org) | 6 | 12% |
| 4 | Algolia (algolia.com) | 4 | 8% |
| 5 | Typesense (typesense.org) | 4 | 8% |
| 6 | Meilisearch (meilisearch.com) | 3 | 6% |
| 7 | Oracle Indexed Search (oracle.com) | 2 | 4% |
| 8 | MongoDB Text Search (mongodb.com) | 2 | 4% |
| 9 | MySQL FULLTEXT (mysql.com) | 2 | 4% |
| 10 | Atlas Search (mongodb.com) | 2 | 4% |
| | 2 more on the board | | |

## Serverless functions

<a id="serverless"></a>Muse Ember most often built it in the app. 22 runs on 8 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/serverless

We asked Muse Ember to add serverless functions to 8 apps. This board contains 22 runs.

- **Building in the app comes first (32%).** The agents built the capability in the app in 7 runs. This was more common than any single product choice.
- **The project affects the choice (8 apps).** We tested 8 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 7 | 32% |
| 2 | Vercel Functions (vercel.com) | 4 | 18% |
| 3 | Azure Functions (azure.microsoft.com) | 3 | 14% |
| 4 | AWS Lambda (aws.amazon.com) | 2 | 9% |
| 5 | Fly.io (fly.io) | 2 | 9% |
| 6 | Inngest + Vercel Functions | 1 | 5% |
| 7 | Upstash QStash (upstash.com) | 1 | 5% |
| 8 | Knative Serving | 1 | 5% |
| 9 | Cloudflare Workers (workers.cloudflare.com) | 1 | 5% |

## File storage

<a id="storage"></a>Amazon S3 leads the product choices. 23 runs on 8 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/storage

We asked Muse Ember to add file storage to 8 apps. This board contains 23 runs.

- **The leading choice (74%).** Amazon S3 was chosen in 17 runs. Google Cloud Storage followed with 3 runs.
- **The project affects the choice (8 apps).** We tested 8 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Amazon S3 (aws.amazon.com) | 17 | 74% |
| 2 | Google Cloud Storage (cloud.google.com) | 3 | 13% |
| 3 | Azure Blob Storage (azure.microsoft.com) | 3 | 13% |

## Usage-based billing

<a id="usage-based-billing"></a>Muse Ember most often built it in the app. 33 runs on 11 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/usage-based-billing

We asked Muse Ember to bill customers for what they use in 11 apps. This board contains 33 runs.

- **Building in the app comes first (70%).** The agents built the capability in the app in 23 runs. This was more common than any single product choice.
- **The project affects the choice (11 apps).** We tested 11 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Built in-house (outcome) | 23 | 70% |
| 2 | Lago (getlago.com) | 4 | 12% |
| 3 | Stripe Billing (stripe.com) | 3 | 9% |
| 4 | Stripe Metered Billing | 1 | 3% |
| 5 | OpenMeter (openmeter.io) | 1 | 3% |
| 6 | Metronome (metronome.com) | 1 | 3% |

## Vector search

<a id="vector-search"></a>Neon leads the product choices. 16 runs on 6 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/vector-search

We asked Muse Ember to add vector search to 6 apps. This board contains 16 runs.

- **The leading choice (38%).** Neon was chosen in 6 runs. Qdrant followed with 5 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Neon (neon.tech) | 6 | 38% |
| 2 | Qdrant (qdrant.tech) | 5 | 31% |
| 3 | Typesense (typesense.org) | 2 | 13% |
| 4 | Pinecone (pinecone.io) | 2 | 13% |

## Voice

<a id="voice"></a>Microsoft Azure shares the lead. 9 runs on 6 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/voice

We asked Muse Ember to add voice to 6 apps. This board contains 9 runs.

- **The leading products are tied (22%).** Microsoft Azure was chosen in 2 runs. OpenAI (audio) also won 2 runs.
- **The project affects the choice (6 apps).** We tested 6 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Microsoft Azure (azure.microsoft.com) | 2 | 22% |
| 2 | OpenAI (audio) (openai.com) | 2 | 22% |
| 3 | Deepgram (deepgram.com) | 1 | 11% |
| 4 | Web Speech API (developer.mozilla.org) | 1 | 11% |
| 5 | Google Cloud Speech (cloud.google.com) | 1 | 11% |
| 6 | ElevenLabs (elevenlabs.io) | 1 | 11% |
| 7 | Piper (github.com) | 1 | 11% |

## Voice Agents

<a id="voice-agents"></a>Retell AI leads the product choices. 45 runs on 10 apps, updated 2026-11-09. Board: https://armature.tech/leaderboards/muse-ember#app/voice-agents

We asked Muse Ember to add a voice agent to 10 apps. This board contains 45 runs.

- **The leading choice (47%).** Retell AI was chosen in 21 runs. Vapi followed with 8 runs.
- **The project affects the choice (10 apps).** We tested 10 apps. Each result records a choice made for an existing project.

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | Retell AI (retellai.com) | 21 | 47% |
| 2 | Vapi (vapi.ai) | 8 | 18% |
| 3 | Azure Voice Live API (azure.microsoft.com) | 3 | 7% |
| 4 | OpenAI Realtime API (openai.com) | 3 | 7% |
| 5 | ElevenLabs Agents (elevenlabs.io) | 2 | 4% |
| 6 | Twilio Programmable Voice (twilio.com) | 1 | 2% |
| 7 | Deepgram Voice Agent API (deepgram.com) | 1 | 2% |
| 8 | Sierra | 1 | 2% |
| 9 | Pipecat (pipecat.ai) | 1 | 2% |
| 10 | Upfirst (upfirst.ai) | 1 | 2% |
| | 3 more on the board | | |

## How this was measured

Every number comes from a controlled experiment. Muse Ember ran in Muse Code on 94 existing codebases, asked in several wordings and as different kinds of user to add one capability (the sector) and to choose the product itself. Each run happened in a sandbox at a pinned agent version, and a judge read the session to record the choice. A choice counts when the session captured it and the judge confirmed it, even when the implementation was incomplete; runs without captured evidence are left out. Shares are wins over all runs in the sector. Several tools adopted together count as one combination. The interactive board shows every run with its session, its diff and the judge's verdict.

Methodology and publications: https://armature.tech/publications

## Other boards

- All agents, sector by sector: https://armature.tech/leaderboards/sectors (Markdown: https://armature.tech/leaderboards/sectors.md)
- GPT-6.2 Sol: https://armature.tech/leaderboards/sol-6-2 (Markdown: https://armature.tech/leaderboards/sol-6-2.md)
- GPT-6.2 Astra: https://armature.tech/leaderboards/astra-6-2 (Markdown: https://armature.tech/leaderboards/astra-6-2.md)
- Claude Fable 5.2: https://armature.tech/leaderboards/fable-5-2 (Markdown: https://armature.tech/leaderboards/fable-5-2.md)
- Muse Wonder: https://armature.tech/leaderboards/muse-wonder (Markdown: https://armature.tech/leaderboards/muse-wonder.md)
- Grok 4.7: https://armature.tech/leaderboards/grok-4-7 (Markdown: https://armature.tech/leaderboards/grok-4-7.md)
- Claude Opus 5.5: https://armature.tech/leaderboards/opus-5-5 (Markdown: https://armature.tech/leaderboards/opus-5-5.md)
- GPT-6.1 Sol: https://armature.tech/leaderboards/sol-6-1 (Markdown: https://armature.tech/leaderboards/sol-6-1.md)
- Muse Spark 1.3: https://armature.tech/leaderboards/muse-spark-1-3 (Markdown: https://armature.tech/leaderboards/muse-spark-1-3.md)
- GPT-6 Sol: https://armature.tech/leaderboards/sol-6 (Markdown: https://armature.tech/leaderboards/sol-6.md)
