What is Peira?

Peira (Greek: trial, test — the root of “empirical”) is an adversarial robustness benchmark for decision models: systems that take a case and return a typed decision — approve/deny, allow/block, advance/hold — in one shot.

General LLM benchmarks measure helpfulness and knowledge. Peira measures something narrower and more operational: when someone tries to flip your model’s decision, does it hold?

Why decision models?

A growing share of deployed AI is not chat — it’s decisions: fraud review, content moderation, loan triage, access control, safety guardrails. These systems face adversaries who probe for the exact input that flips an output. Traditional accuracy benchmarks don’t capture that threat.

What Peira gives you

• 2,025 adversarial cases across 11 attack families, each a benign/attacked pair sharing the same correct decision
• A leaderboard that ranks models on robustness and the practical dimensions of model selection: cost per decision, latency, calibration
• Sealed artifacts: every number is backed by a tamper-evident run artifact you can verify yourself
• A blind holdout: 500 private cases, pseudonymized, so gaming the public set doesn’t help

The one-line pitch

If you’re picking a decision model, Peira tells you which one holds up under attack, what it costs, and how fast it is — with every number traceable to a sealed artifact.