# Product analytics: which products coding agents choose

> PostHog won about 53% of runs across three agents.

Source: https://armature.tech/leaderboards/product-analytics (Armature agent leaderboards). 359 runs, 10 apps, 3 agents, 4 personas, updated 2026-09-02. Interactive board with every run: https://armature.tech/leaderboards#app/product-analytics

## Key learnings

We asked three agents to add product analytics to 10 small apps, 359 runs in all. We varied the wording and who was asking. In about 23% of runs the agents skipped every product and wrote it themselves.

### One theme handed the lead to another product (0 of 35)

In 35 runs the ask was to fit an existing data stack. PostHog won none of those. Amplitude led them with six wins.

### Most cases did not settle on one product (22 of 30)

Take one codebase with one agent, asked the same thing several ways. In 22 of 30 such cases the runs did not all land on the same product.

### Who asked changed the answer (27 of 36)

Vibe coders picked Vercel Analytics in 27 of their 36 runs. Junior developers picked PostHog 131 times.

### Named often, chosen almost never (222 vs 5)

Mixpanel came up in 222 runs and won five of them. Google Analytics came up in 91 runs and won none.

Smaller learnings:

- The simulated user, who reads each plan before any code is written, approved all 359 runs.
- It sent the agent back at least once in 27 runs, and in three it refused until a product was named.
- Umami won six times, all of them in the Nuxt field service app.
- Datadog won twice, both times in the high-traffic checkout platform.

## The ranking

| # | Product | Wins | Share |
|---|---|---:|---:|
| 1 | PostHog (posthog.com) | 189 | 53% |
| 2 | Built in-house (outcome) | 81 | 23% |
| 3 | Vercel Analytics (vercel.com) | 27 | 8% |
| 4 | Segment (segment.com) | 17 | 5% |
| 5 | Amplitude (amplitude.com) | 14 | 4% |
| 6 | Umami (umami.is) | 6 | 2% |
| 7 | Mixpanel (mixpanel.com) | 5 | 1% |
| 8 | Snowplow (snowplow.io) | 2 | 1% |
| 9 | Metabase (metabase.com) | 2 | 1% |
| 10 | Plausible (plausible.io) | 2 | 1% |
| 11 | Datadog (datadoghq.com) | 2 | 1% |
| 12 | Ahoy (github.com) | 1 | 0% |
| 13 | Mitzu (mitzu.io) | 1 | 0% |
| 14 | Datadog Product Analytics (datadoghq.com) | 1 | 0% |

## By agent

- Codex (GPT-5.6 Sol): 120 runs, first PostHog (69), then Amplitude (11)
- Claude Code (Claude Opus 5): 120 runs, first PostHog (49), then Vercel Analytics (9)
- Cursor (Grok 4.6): 119 runs, first PostHog (71), then Vercel Analytics (10)

## By persona

- Junior developer: 144 runs, first PostHog (131), then Amplitude (1)
- Senior engineer: 143 runs, first PostHog (44), then Segment (17)
- Vibe coder: 36 runs, first Vercel Analytics (27), then PostHog (4)
- Enterprise team: 36 runs, first PostHog (10), then Amplitude (2)

## By what the ask stressed

- The plain ask: 132 runs, first PostHog (77), then Vercel Analytics (15)
- Read by someone who is not an engineer: 84 runs, first PostHog (67), then Amplitude (6)
- Self-hosting, privacy or residency: 60 runs, first PostHog (22), then Vercel Analytics (12)
- Fits an existing data stack: 35 runs, first Amplitude (6), then Segment (4)
- Procurement and compliance: 24 runs, first PostHog (12), then Snowplow (1)
- Volume and cost at scale: 24 runs, first PostHog (11), then Datadog (1)

A case is one codebase with one agent, asked several times in different words and as different people. 22 of 30 cases did not hold to a single choice.

## How this was measured

Every number on this page comes from a controlled experiment. We took 10 small applications, asked 3 coding agents (Codex (GPT-5.6 Sol), Claude Code (Claude Opus 5), Cursor (Grok 4.6)) to add product analytics to each of them, in several wordings and as a junior developer and senior engineer and vibe coder and enterprise team, and let the agent choose the product. Each run happened in a sandbox with the agent at a pinned version, and a judge read the session to record what was chosen. That is 359 runs. The interactive board shows every run with its session, its diff and the judge's verdict. A simulated user stood in for the owner of the codebase: it read the agent's plan and had to approve it before any code was written; it sent the agent back at least once in 27 runs.

Methodology and publications: https://armature.tech/publications

## Other sectors

- [Agent sandboxes](https://armature.tech/leaderboards/sandboxes) (https://armature.tech/leaderboards/sandboxes.md)
- [Observability](https://armature.tech/leaderboards/observability) (https://armature.tech/leaderboards/observability.md)
- [Payments](https://armature.tech/leaderboards/payments) (https://armature.tech/leaderboards/payments.md)
- [Deploy](https://armature.tech/leaderboards/deploy) (https://armature.tech/leaderboards/deploy.md)
- [Auth](https://armature.tech/leaderboards/auth) (https://armature.tech/leaderboards/auth.md)
- [Email providers](https://armature.tech/leaderboards/mail) (https://armature.tech/leaderboards/mail.md)
- [Databases](https://armature.tech/leaderboards/databases) (https://armature.tech/leaderboards/databases.md)
- [File storage](https://armature.tech/leaderboards/storage) (https://armature.tech/leaderboards/storage.md)
- [LLM evals & observability](https://armature.tech/leaderboards/evals) (https://armature.tech/leaderboards/evals.md)
- [Voice Agents](https://armature.tech/leaderboards/voice-agents) (https://armature.tech/leaderboards/voice-agents.md)
- [Serverless functions](https://armature.tech/leaderboards/serverless) (https://armature.tech/leaderboards/serverless.md)
- [Cloud](https://armature.tech/leaderboards/cloud) (https://armature.tech/leaderboards/cloud.md)
- [AI gateway](https://armature.tech/leaderboards/ai-gateway) (https://armature.tech/leaderboards/ai-gateway.md)
- [Bot protection](https://armature.tech/leaderboards/bot-protection) (https://armature.tech/leaderboards/bot-protection.md)
- [Search](https://armature.tech/leaderboards/search) (https://armature.tech/leaderboards/search.md)
- [Agent frameworks](https://armature.tech/leaderboards/agent-frameworks) (https://armature.tech/leaderboards/agent-frameworks.md)
- [Performance in CI](https://armature.tech/leaderboards/perf-ci) (https://armature.tech/leaderboards/perf-ci.md)