Which AI actually writes RTL that compiles and passes?
RTL Arena is an open benchmark for AI hardware design. Every model's generated Verilog is compiled and simulated against real, public, self-checking testbenches. A task counts as passed only when the simulation itself reports success — no lint-only shortcuts, no partial credit.
Ranked by what actually simulates
Combined pass rate across all 289 tasks. Only models that completed every task in all four
suites appear here. The ± figure is a 95% Wilson score confidence interval on
the measured proportion — genuine sampling uncertainty, not a decorative error bar.
Cost-efficiency frontier
Each line is one model, swept across its available effort and reasoning levels. Combined pass rate plotted against the total API cost of that run — more spend does not reliably buy more correctness.
What actually breaks
Each model's base run, classified from the grader's own compile and simulate stages and its error text. Hover any segment — including "Other" — for the exact subtype breakdown rather than a single opaque bucket.
Public tasks, real grading
Every suite is publicly available and independently maintained. We didn't write the tasks, which means we can't tune them in anyone's favor — including our own.
How a task is graded
- Compile and simulate — not lint
-
Each task ships with a self-checking testbench. Generated RTL is compiled and run with
real Icarus Verilog (
iverilog/vvp). A task passes only if the simulation reports success. Code that parses cleanly but computes the wrong answer fails, which is the entire point. - One independent attempt per task
- Every score on this board is a single attempt per task — no best-of-n, no retries, no partial credit. A model gets one shot at each of the 289 tasks.
- Confidence intervals
-
A pass rate is an estimate from
nindependent pass/fail trials. The±figure is a 95% Wilson score interval on that proportion. Models whose intervals overlap are not meaningfully separated, however the ranking orders them. - Effort variants are listed separately
- Where a model exposes reasoning-effort levels, each level is run and ranked as its own entry. A single model family can therefore appear several times with materially different scores — collapsing them into one number would hide the tradeoff rather than show it.
- Snapshot, not a live feed
- This page renders a fixed snapshot of harness output rather than querying a live service, so the numbers here are stable and reproducible. The snapshot date is shown above the table.
The models are the floor, not the ceiling.
SigmanticAI wraps these models in specialist agents that run your EDA tools, read simulation output, and iterate until tests actually pass. That's where the remaining gap closes.