The ceiling on every multi-model system is 1 minus your co-failure rate

Good morning 👋 A paper this week proves there's a hard ceiling on every multi-model system you'll build — set before you write a single line of routing logic. Most teams haven't measured the thing that determines it.

In today's issue:

• 🔭 The provable limit on multi-model accuracy, and why your ensemble gains are probably overstated
• 🔬 Co-failure ceilings, prompt injection in hiring pipelines, and predictable world-model hallucinations
• 🛠️ A terminal multiplexer for fanning one prompt to multiple agents and reading the disagreement
• 🔗 Stripe's compliance agent in production, a 400M-document redaction run, and a new ASR leaderboard

🔭 THE ONE THING

🧱 The ceiling on every multi-model system is 1 minus your co-failure rate

Josef Chen's paper across 67 frontier LLMs from 21 providers establishes a provable ceiling for routing, voting, cascades, and mixture-of-agents systems: accuracy cannot exceed 1 - β, where β is the co-failure rate, the fraction of queries where every model in your pool fails simultaneously. No routing policy or voting scheme moves that ceiling if your models already fail together. The more actionable finding: co-failure is being badly underestimated. On math tasks, observed beta was 0.052 against a predicted 0.023; code execution landed at 0.079. Measure your co-failure rate before you architect around ensemble gains.

🔬 RESEARCH HIGHLIGHTS

• Co-failure ceiling on multi-model systems — Across 67 frontier models, accuracy for any multi-model setup (routing, voting, cascades, mixture-of-agents) cannot exceed 1 minus the co-failure rate. If the models fail the same questions, combining them changes nothing. The field rarely reports co-failure rates, which is exactly why ensemble gains get overstated. Worth knowing before you architect your next multi-agent pipeline. arXiv

• Prompt injection in AI résumé screening — Hiding self-promotional instructions inside résumé text moves candidates up in LLM rankings without adding real qualifications. The paper tests single and multi-injection settings. If you're using LLMs to filter applicants, the attack surface is open right now. arXiv

• World model hallucinations aren't random — They concentrate in low-coverage regions of the state-action space, which makes them predictable. The paper shows they're also preventable. Directly relevant if you're building on generative world models for robotics or planning. arXiv

🛠️ TRY THIS

Fan out one prompt to multiple agents from one terminal session using herdr

The practical response to co-failure ceilings isn't a smarter model. It's routing the same task to models that fail differently and treating disagreement as the signal. herdr (7.6k GitHub stars) is a terminal-native agent multiplexer: one session, multiple agents running in parallel, output collected where you can actually compare it.

1. Clone the repo and follow the README install steps

2. Write a config that points two or three agents at different models (e.g., Claude Sonnet for one, an open-weight model for another)

3. Run your task through herdr; it fans out and collects responses in one place

4. Check disagreements first. Consensus is cheap; divergence is where the useful signal lives

Prompt: You are the second reviewer. A different model has already answered the question below. Do not default to agreement — find what's missing, overstated, or wrong in that answer. Question: [paste your task here]

🔗 QUICK LINKS

• Stripe documented how it runs a ReAct-based compliance agent in production: a dedicated agent service, not bolted onto existing infrastructure, with humans still reviewing flagged decisions. (AWS ML Blog)
• Huntington Bank processed 400M+ documents for PII and PCI redaction using AWS, compressing a project that would have taken years to a few months at better than 95% accuracy. (AWS ML Blog)
• Hugging Face launched FFASR, a real-world ASR leaderboard; standard benchmark scores, it turns out, aren't a reliable proxy for how these models perform in production. (HF Blog)

Measure before you architect.

Pradeep Perugu