How the ranking works

Each round shows the prompt text and two clips, A and B, rendered by two different systems from the same text. You play both and pick the one that sounds closer to a real person. If neither does, a de-emphasized “Neither sounds human” option lets you say exactly that — a directed claim, not a tie. Neither-votes never enter the ranking fit; they are recorded as a signal about which voices fail to clear the human bar at all. Vendor and model names stay hidden until the moment after your vote lands, and voting unlocks only once both clips have actually played.

Both sides read the same prompt, and which side is shown left/right is randomized — so the comparison is voice-vs-voice, never script-vs-script or position-vs-position.

Voices are only ever paired within the same gender (male vs male, female vs female). A male and a female read aren't a fair “which is more human” test, so they never meet.

Every clip — AI and the human — is normalized to a common −16 LUFS, spectrally matched to one shared frequency profile (reference-EQ, Matchering), and re-encoded to one uniform format (44.1 kHz mono MP3); the human recordings are additionally dereverberated. Louder, brighter, or roomier audio is reliably judged by its mastering rather than its voice — equalizing the whole chain means a vote can only be about how the voice speaks, not how it was recorded.

On every vote your browser sends a small JSON payload to POST /api/vote: your pick (a / b / neither), reaction time in ms, how many ms of each clip actually played, which side was A, and which clip you played first. A neither vote is stored like any other, but it never enters the ranking fit.

Rows recorded before 2026-07-01 can also carry a tie verdict, from the since-removed Tie option; those rows are excluded from the ranking fit.

The server adds only an anonymous cookie id and a hashed IP (the raw address is never stored), then writes one row. No names, emails, or accounts — nothing that identifies you. The reaction-time and played-ms fields are an effort signal; the side/order fields let position and order bias be measured.

A real human recording — public-domain studio reads — is mixed into the arena as a hidden voice. A voice's humanness is how often listeners pick it over the human: picked about half the time, it's statistically indistinguishable — humanness 100. Nothing scores above 100.

The human reference is currently a female voice, so it appears in female rounds; a male reference is being added so male voices anchor to a human too.

The board doesn't start empty. Opening scores are a seed: Vapi's published Humanness Index results (humannessindex.vapi.ai, retrieved 2026-06) encoded as prior comparisons, roughly 940 votes per voice. Voices Vapi doesn't rank yet enter as estimates placed on the same scale, and live votes take over from the seed as they land. Latency is real — warm first-audio measured network-free at the Speko gateway (the same dataset as benchmarks.speko.dev, which measures latency, not humanness) — and price is each provider's published list rate.

As real votes accumulate they take over from the seed: a Bradley-Terry maximum-likelihood fit (Hunter's MM) on an Elo-style scale. The confidence intervals shown today are a conservative count-based heuristic — 360/sqrt(votes), clamped — that narrows with sample size; a percentile bootstrap is planned. Every ranked vote is a forced A/B choice — play both, pick the more human one. “Neither sounds human” votes never enter the fit — excluded exactly like the tie votes cast before the 2026-07-01 protocol change — so they can't compress the gap between two voices. A higher score is not automatically a win — two voices are distinguishable only when their intervals don't overlap.

A per-source rate limit — keyed on a hashed IP, never the raw address — blunts ballot-stuffing without collecting personal data.

About 1 in 15 rounds is a gold attention-check: the real human against an obviously synthetic voice, with a known answer. Those rounds never enter the ranking fit. The first 10 rounds also schedule exactly 3 human pairs for the intro game, and every round grades right or wrong: right means you spotted the human or correctly called an all-AI round; wrong means an AI tricked you — you picked a voice as the more human one when no human was there, missed the hidden human, or said “neither” when one was real. Grading is the game layer only — ranked votes are unaffected — and the shareable score stays humans-spotted, so no option can be farmed for score. Trick events are recorded per voice, which is how the fooled-rate stat is measured once enough human rounds land.

Today's scores are seeded and will shift as votes land — the confidence intervals are the point. Preference is also relative and population-dependent: it measures which of two voices sounds more human to these listeners, not a ground-truth score, and says nothing about intelligibility, cost, or fit for a use case.