Agent leaderboards / All sectors / Observability
Observability: which products coding agents choose
Sentry won about 36% of runs, and every agent put it first.
Read this leaderboard as textrankings, key learnings, method
Key learnings
We asked five coding agents to add observability to 11 codebases, 593 runs in all. We varied the wording, and we asked as four different people. Below the leader the picks spread thin, and in 41 runs the agents wrote it themselves.
The agents agreed on first, not on second
Sentry led for Cursor, Claude Code, Codex, Muse Code and Grok Build CLI. Second place moved around. Claude Code chose Sentry 52 times and Grafana 37 times. For Codex the runner-up was New Relic.
Who asked changed the answer
Enterprise teams picked Grafana in 36 of their 60 runs. Vibe coders led with Better Stack, seven times out of 20. Senior engineers and junior developers both led with Sentry.
Rewording the question moved the pick
We took one codebase and one agent, then asked again in other words and as other people. In 44 of 50 such cases the runs did not all land on the same product.
- The simulated user approved every plan, and sent the agent back at least once in 90 runs.
- In two runs it refused until the agent named a specific product.
- Pino came up in 63 runs and was never chosen.
The ranking593 runs
| Product | Wins | Share | ||
|---|---|---|---|---|
| 1 | Sentrysentry.io | 211 | 36% | |
| 2 | Grafanagrafana.com | 86 | 15% | |
| 3 | Amazon CloudWatchaws.amazon.com | 59 | 10% | |
| 4 | Built in-houseoutcome | 41 | 7% | |
| 5 | Better Stackbetterstack.com | 26 | 4% | |
| 6 | Checklychecklyhq.com | 25 | 4% | |
| 7 | New Relicnewrelic.com | 24 | 4% | |
| 8 | Datadogdatadoghq.com | 24 | 4% | |
| 9 | Honeycombhoneycomb.io | 20 | 3% | |
| 10 | AWS X-Rayaws.amazon.com | 18 | 3% | |
| 11 | GlitchTipglitchtip.com | 12 | 2% | |
| 12 | Azure Monitorazure.microsoft.com | 12 | 2% | |
| 13 | Azure Application Insightsazure.microsoft.com | 8 | 1% | |
| 14 | Axiomaxiom.co | 6 | 1% | |
| 15 | SigNozsignoz.io | 6 | 1% | |
| 16 | Amazon CloudWatch + AWS X-Ray | 4 | 1% | |
| 17 | Bugsinkbugsink.com | 3 | 1% | |
| 18 | UptimeRobotuptimerobot.com | 2 | 0% | |
| 19 | Solid Errorsgithub.com | 2 | 0% | |
| 20 | Exception Notificationgithub.com | 1 | 0% | |
| 21 | Prometheusprometheus.io | 1 | 0% | |
| 22 | Pino + Sentry | 1 | 0% |
By agent, by persona, by wording
By agent
| Codex180 runs | Sentry · 42then New Relic · 24 |
| Cursor150 runs | Sentry · 68then Grafana · 16 |
| Claude Code150 runs | Sentry · 52then Grafana · 37 |
| Muse Code · Muse Spark 1.389 runs | Sentry · 40then Grafana · 11 |
| Grok Build CLI · Grok 4.724 runs | Sentry · 9then Amazon CloudWatch · 5 |
By persona
| Senior engineer493 runs | Sentry · 188then Amazon CloudWatch · 59 |
| Enterprise team60 runs | Grafana · 36then Datadog · 9 |
| Junior developer20 runs | Sentry · 17 |
| Vibe coder20 runs | Better Stack · 7then UptimeRobot · 2 |
By what the ask stressed
| The plain ask573 runs | Sentry · 211then Grafana · 86 |
A case is one codebase with one agent, asked several times in different words and as different people. 44 of 50 cases did not hold to a single product.
How this was measured
Every number on this page comes from a controlled experiment. We took 11 small applications, asked 5 coding agents (Codex, Cursor, Claude Code, Muse Code (Muse Spark 1.3), Grok Build CLI (Grok 4.7)) to add observability to each of them, in several wordings and as a senior engineer and enterprise team and junior developer and vibe coder, and let the agent choose the product. Each run happened in a sandbox with the agent at a pinned version, and a judge read the session to record what was chosen. That is 593 runs. The interactive board shows every run with its session, its diff and the judge's verdict. A simulated user stood in for the owner of the codebase: it read the agent's plan and had to approve it before any code was written; it sent the agent back at least once in 90 runs. Read the methodology and the publications.
If you sell in this sector: what these numbers mean for a vendor.
Open the interactive boardThis page as MarkdownOther sectors
- Agent sandboxes
- AI SRE
- Payments
- Deploy
- Auth
- Email providers
- Product analytics
- Databases
- File storage
- LLM evals & observability
- Voice Agents
- Serverless functions
- Cloud
- AI gateway
- Bot protection
- Search
- Agent frameworks
- Performance in CI
- Document processing & OCR
- Usage-based billing
- Internationalization
- Security testing
- Maps
- Message queues
- AI search
- Code review
- E-signature
- Security platforms
- In-app chat & calls
- Vector search