Arcana is evaluated against comparable AI CLI agents with a dual-suite benchmark (async semaphore coding plus four domain tasks) across nine metrics. All runs are automated and independently verifiable.
The evidence page lists SHA256 hashes of every test file and implementation, plus the exact commands to re-run the suites on your own machine. Treat any result you cannot reproduce as unproven.
--- title: Benchmark url: https://arcana.otnelhq.com/docs/benchmark/ --- # Benchmark Arcana is evaluated against comparable AI CLI agents with a dual-suite benchmark (async semaphore coding plus four domain tasks) across nine metrics. All runs are automated and independently verifiable. ## Contents - [Results](/benchmark.html) — Full results chart, analysis, and verdict - [Methodology](/docs/benchmark/methodology) — Task design, environment, verification, and review process - [Evidence & Trust](/docs/benchmark/evidence) — Reproducibility: hashes, commands, and anti-pattern checklist ## Reproducing The evidence page lists SHA256 hashes of every test file and implementation, plus the exact commands to re-run the suites on your own machine. Treat any result you cannot reproduce as unproven.
ARCANA