Blocking everything scores zero.
The index uses chance-corrected balanced accuracy. Every unsafe test has a legitimate counterpart. Errors and missing outputs lose credit; coverage stays visible.
EDITION 0.1 · SYNTHETIC PILOTAI guardrails,
AI guardrails,
measured in context.
Can a guardrail recognize the violation — and let the legitimate request through? Paired tests across safety, security, privacy, and policy.
Synthetic pilot. Machine-authored labels; independent human adjudication pending. Scores apply to this suite and adapter, and do not establish production reliability.
—Evaluated baselines
—Public cases · paired & balanced
12Public guardrail skills
EN + 中文Context and policy in two languages
Reproducible receipts. Explicit coverage.
Model results
Pilot scores are shown without competitive ranks. Select a model for public area scores. Four-bit NF4 runs are marked. Fixed-taxonomy guards use a published policy/context wrapper. Full view shows public diagnostic metrics; a Full confidence interval is not computed. Status distinguishes a completed zero score from invalid answers.
| # | Model / adapter | Run status | Index ↑ | Full 95% CI | Valid coverage | Public recall | Public false positives ↓ | Public pair exact ↑ |
|---|---|---|---|---|---|---|---|---|
| Loading published results… | ||||||||
Public recipe. Private evidence.
Open cases and predictions make runs reproducible. Held-out cases and their raw outputs stay private. Only aggregate private scores reach this page.
Bring your model.
Run python -m ogb against an HTTP endpoint or a pinned model checkpoint. Outputs include raw predictions, configuration, hashes, and scoring receipts.