Paved Gate · Dashboards

A fast System 1 gate in front of AI agents

Published snapshots of two live runs comparing the Paved Gate fast evaluator (Jev) with zero-shot frontier models: one measures latency and cost, the other measures classification accuracy on labelled support tickets.

Latency and cost benchmark

September 27, 2026 · 4 arms × 60 requests

All arms live

Jev vs GPT-4o, GPT-4o mini and Claude Sonnet 5, answering the same three gate questions per request.

83.8%

lower p50 latency
vs GPT-4o

60.6%

lower cost
vs GPT-4o mini

Open dashboard →

Support ticket classification eval

October 1, 2026 · 3 arms × 1,080 tickets

All arms live

Intent accuracy on a fixed sample of the Bitext customer support dataset (27 intents, CDLA-Sharing-1.0).

91.9%

Jev accuracy
(GPT-4o 92.9%, mini 93.4%)

80.3%

lower p50 latency
vs GPT-4o mini

Open dashboard →

NVIDIA OpenShell + Paved Gate

October 11, 2026 · 3 arms × 290 synthetic cases

Live

Real OpenShell sandboxes with Paved Gate as supervisor middleware: who caught what, accuracy in tainted and untainted sandboxes, and added latency.

Open dashboard →

The accuracy differences are not statistically significant at this sample size (paired McNemar test on the published per-ticket predictions: p = 0.33 vs GPT-4o, p = 0.12 vs GPT-4o mini). Both runs are one machine's snapshot, with list prices verified on the run date; see each dashboard's caveats before quoting a figure.