# Benchmarks

URL: /docs/benchmarks
Markdown: /docs/benchmarks.md

> Instructions retired per call for every benchmark program on the reference Mac, how they are counted, and what they do and do not say.

> **The blessed baseline**
>
> These are instruction counts from one run of the benchmark suite on the reference Mac (an
> Apple silicon MacBook, host `d88970189598`) at commit `320c9980`, 2026-10-19. They are
> blessed as the baseline (2026-10-20,
> [`bench/baselines/d88970189598.json`](https://github.com/icantcodefyi/sakuralang/blob/main/bench/baselines/d88970189598.json)),
> and a counted check of the `v0.1.0` candidate on `fib` and `shapes` passed against it on every Node cell. Later releases are compared with these
> counts. They are counts, not times: no speed claim follows from them. The cross-language envelope in design §17.109 is not met by these counts, and Sakura
> Lang makes no cross-language speed claim at 0.1.0 (design §17.265).

The [benchmark suite](https://github.com/icantcodefyi/sakuralang/tree/main/bench) runs the
same workloads written in Sakura Lang and by hand in JavaScript, Rust, Go, OCaml and Koka.
Every program has an oracle: a cell is measured only after it returns the recorded checksum
for every input. The numbers are specific to these workloads and this one machine. They are
not a language ranking.

## What is measured [#what-is-measured]

The primary metric is **instructions retired per call**, read from macOS's
`/usr/bin/time -l`. As with [rustc-perf](https://perf.rust-lang.org), the reason is
repeatability: the count barely moves when other programs share the machine, so the suite runs
while other work goes on and needs no idle night ([design §17.264](https://github.com/icantcodefyi/sakuralang/blob/main/docs/design.md)).

> **Diagram:** How bench/bench run --instructions measures one cell (design §17.264). Start-up cancels in the difference.
>
> 1. `calibrate k`: double k until k more calls retire 2³² instructions. Then `run(n) × 1` or `run(n) × (1 + k)`.
> 2. `run(n) × 1`: a fresh process: start-up, loading, the first call. Then `(many − one)/k`.
> 3. `run(n) × (1 + k)`: a fresh process: the same, then k warm calls.
> 4. `(many − one)/k`: instructions retired per warm call.
> 5. `median of 5 pairs`: reported with its spread, (max − min) / median.
>
> The counts come from macOS's /usr/bin/time -l: every thread of the one process it runs, none of its children.

Each cell is one program at one input size `n` on one engine. The harness calibrates a call
count `k`, then runs five pairs of fresh processes: one calls `run(n)` once, the other `1 + k`
times. Start-up, module loading and the first call's compilation are in both, so the
difference over `k` is the cost of a warm call. The cell is the median of the five pairs, and
its spread is (max − min) / median.

An instruction count is not a time. It does not see cache misses, branch mispredictions or how
many instructions a core retires per cycle, and two engines can retire the same instructions
at different speeds: in the first counted runs one Wasmtime cell's ratio of counts and its
ratio of times differed by half. A ratio below is a ratio of counts, never of times.

## Results [#results]

Every value is instructions retired per call (or per process, for the compiler) on the
reference Mac at commit `320c9980`, at each program's largest size. A ratio is Sakura's count
over hand JavaScript's on the engine named. `K`, `M` and `G` are thousands, millions and
billions. Node runs with V8's `--predictable` flag, which keeps its garbage collector and
compiler on the main thread so that their counts do not depend on scheduling.

### JavaScript backend [#javascript-backend]

| Program | n | Sakura, Node | hand JS, Node | ratio | Sakura, Bun | hand JS, Bun | ratio |
|---|---:|---:|---:|---:|---:|---:|---:|
| `binary_trees` | 14 | 957 M | 680 M | 1.41× | 1.28 G | 503 M | 2.54× |
| `contracts` | 1,000,000 | 119 M | 44.3 M | 2.69× | 97.5 M | 54 M | 1.81× |
| `contracts_plain` | 1,000,000 | 72.4 M | 24.2 M | 2.99× | 47.4 M | 32 M | 1.48× |
| `countdown` | 300,000 | 16 M | 10.2 M | 1.57× | 5.43 M | 5.15 M | 1.05× |
| `deltablue` | 10,000 | 1.92 G | 317 M | 6.06× | 2.74 G | 441 M | 6.23× |
| `fib` | 30 | 191 M | 137 M | 1.39× | 212 M | 91.5 M | 2.31× |
| `handled_d1` | 1,000,000 | 50 M | 57.1 M | 0.88× | 21.1 M | 18.4 M | 1.14× |
| `handled_d16` | 1,000,000 | 50.3 M | 57.1 M | 0.88× | 21.1 M | 18.4 M | 1.14× |
| `handled_d4` | 1,000,000 | 50 M | 57.1 M | 0.88× | 21.1 M | 18.5 M | 1.14× |
| `handled_fwd` | 1,000,000 | 373 M | 76.2 M | 4.90× | 452 M | 38.8 M | 11.63× |
| `handled_reentry` | 1,000,000 | 548 M | 782 M | 0.70× | 627 M | 87 M | 7.21× |
| `handled_suspend` | 1,000,000 | 93 M | 86.1 M | 1.08× | 26.3 M | 124 M | 0.21× |
| `json_validate` | 30,000 | 791 M | 569 M | 1.39× | 613 M | 204 M | 3.01× |
| `map_set` | 100,000 | 3.18 G | 968 M | 3.28× | 2.86 G | 744 M | 3.85× |
| `map_set:int` | 100,000 | 840 M | 135 M | 6.20× | 865 M | 65.4 M | 13.23× |
| `map_set:string` | 100,000 | 980 M | 433 M | 2.26× | 927 M | 428 M | 2.17× |
| `map_set:tuple` | 100,000 | 1 G | 366 M | 2.75× | 1.1 G | 238 M | 4.63× |
| `nbody` | 100,000 | 1.34 G | 113 M | 11.90× | 1.24 G | 140 M | 8.89× |
| `pipeline` | 300,000 | 87.2 M | 302 M | 0.29× | 73.2 M | 27.7 M | 2.65× |
| `rbtree` | 100,000 | 2.63 G | 1.05 G | 2.52× | 1.53 G | 619 M | 2.47× |
| `shapes` | 100,000 | 1.21 G | 540 M | 2.24× | 869 M | 379 M | 2.29× |
| `sieve` | 200,000 | 80.5 M | 17.4 M | 4.62× | 87.6 M | 12.5 M | 6.98× |
| `spectral_norm` | 400 | 755 M | 168 M | 4.49× | 485 M | 203 M | 2.39× |
| `tail_clique` | 10,000,000 | 293 M | 334 M | 0.88× | 141 M | 176 M | 0.81× |
| `tokenizer` | 100,000 | 159 M | 526 M | 0.30× | 222 M | 190 M | 1.17× |

### WebAssembly backend [#webassembly-backend]

Sakura compiled with `--target wasm`, run on Node's WebAssembly engine and on Wasmtime, over
the same hand JavaScript on Node. `json_validate` has no Wasmtime cell: it needs JSON
decoding, which only the Node host implements.

| Program | n | Sakura WASM, Node | Sakura WASM, Wasmtime | hand JS, Node | ratio, Node | ratio, Wasmtime |
|---|---:|---:|---:|---:|---:|---:|
| `binary_trees` | 14 | 4.1 G | 4.82 G | 680 M | 6.03× | 7.09× |
| `contracts` | 1,000,000 | 3.02 G | 4 G | 44.2 M | 68.29× | 90.47× |
| `contracts_plain` | 1,000,000 | 1.66 G | 2.04 G | 24.2 M | 68.77× | 84.29× |
| `countdown` | 300,000 | 19.7 M | 19.2 M | 10.2 M | 1.94× | 1.89× |
| `deltablue` | 10,000 | 33.6 G | 39.5 G | 318 M | 105.70× | 124.48× |
| `fib` | 30 | 270 M | 259 M | 137 M | 1.96× | 1.88× |
| `handled_d1` | 1,000,000 | 1.27 G | 1.6 G | 57.1 M | 22.30× | 28.11× |
| `handled_d16` | 1,000,000 | 1.25 G | 1.6 G | 57.1 M | 21.95× | 28.11× |
| `handled_d4` | 1,000,000 | 1.27 G | 1.61 G | 57.1 M | 22.30× | 28.13× |
| `handled_fwd` | 1,000,000 | 6.16 G | 7.5 G | 76.3 M | 80.83× | 98.42× |
| `handled_reentry` | 1,000,000 | 8.77 G | 10.6 G | 782 M | 11.21× | 13.57× |
| `handled_suspend` | 1,000,000 | 1.43 G | 1.84 G | 86.3 M | 16.58× | 21.33× |
| `json_validate` | 30,000 | 13.6 G | absent | 570 M | 23.94× | — |
| `map_set` | 100,000 | 7.36 G | 8.33 G | 972 M | 7.57× | 8.57× |
| `map_set:int` | 100,000 | 2.25 G | 2.57 G | 136 M | 16.52× | 18.87× |
| `map_set:string` | 100,000 | 2.56 G | 2.98 G | 434 M | 5.91× | 6.87× |
| `map_set:tuple` | 100,000 | 2.66 G | 2.94 G | 366 M | 7.26× | 8.03× |
| `nbody` | 100,000 | 5.38 G | 6.68 G | 113 M | 47.58× | 59.15× |
| `pipeline` | 300,000 | 1.85 G | 2.3 G | 302 M | 6.13× | 7.61× |
| `rbtree` | 100,000 | 7.84 G | 10.1 G | 1.05 G | 7.48× | 9.60× |
| `shapes` | 100,000 | 3.61 G | 4.33 G | 542 M | 6.66× | 7.99× |
| `sieve` | 200,000 | 894 M | 1.13 G | 17.4 M | 51.28× | 64.69× |
| `spectral_norm` | 400 | 5.58 G | 7.18 G | 168 M | 33.17× | 42.66× |
| `tail_clique` | 10,000,000 | 1.22 G | 1.16 G | 333 M | 3.67× | 3.49× |
| `tokenizer` | 100,000 | 18.1 G | 18.5 G | 526 M | 34.51× | 35.27× |

### Native controls [#native-controls]

The same workloads in Rust (with overflow checks, the matched lane), Go, OCaml and Koka. A
dash means the program has no port in that language.

| Program | n | Rust | Go | OCaml | Koka | Koka, no reuse |
|---|---:|---:|---:|---:|---:|---:|
| `binary_trees` | 14 | 1.29 G | 806 M | 206 M | 306 M | 305 M |
| `contracts` | 1,000,000 | 5.38 M | 15.8 M | — | — | — |
| `contracts_plain` | 1,000,000 | 5.13 M | 10.5 M | — | — | — |
| `countdown` | 300,000 | 1.13 M | 2.4 M | — | — | — |
| `deltablue` | 10,000 | 171 M | 203 M | — | — | — |
| `fib` | 30 | 31 M | 41.8 M | — | — | — |
| `handled_d1` | 1,000,000 | 11 M | 17 M | — | 148 M | 148 M |
| `handled_d16` | 1,000,000 | 11 M | 17 M | — | 148 M | 148 M |
| `handled_d4` | 1,000,000 | 11 M | 17 M | — | 148 M | 148 M |
| `handled_fwd` | 1,000,000 | 11 M | 18 M | — | 256 M | 256 M |
| `handled_reentry` | 1,000,000 | 14 M | 23.3 M | — | 174 M | 174 M |
| `handled_suspend` | 1,000,000 | 16 M | 20 M | — | 245 M | 245 M |
| `json_validate` | 30,000 | 271 M | 689 M | — | — | — |
| `map_set` | 100,000 | 907 M | 887 M | 1.82 G | 7.16 G | 7.17 G |
| `nbody` | 100,000 | 40.9 M | 78.7 M | — | — | — |
| `pipeline` | 300,000 | 7.69 M | 17.8 M | 358 M | 49.4 M | 60 M |
| `rbtree` | 100,000 | 863 M | 1.19 G | 700 M | 340 M | 340 M |
| `shapes` | 100,000 | 33.8 M | 270 M | 1.08 G | 339 M | 337 M |
| `sieve` | 200,000 | 3.58 M | 4.51 M | — | — | — |
| `spectral_norm` | 400 | 112 M | 125 M | — | — | — |
| `tail_clique` | 10,000,000 | 150 M | 233 M | — | — | — |
| `tokenizer` | 100,000 | 6.4 G | 21.3 G | — | — | — |

### Compiler [#compiler]

One `sakura` process per run, after a warm-up: `check` and `build` of generated packages and
of a new project. `test_hello` counts the CLI's own process, not the Node process it starts.

| Case | Lines | Instructions per process | Spread |
|---|---:|---:|---:|
| `check_hello` | 11 | 70.9 M | 0.28% |
| `test_hello` | 11 | 113 M | 1.52% |
| `check_cold_10k` | 10,084 | 641 M | 0.16% |
| `build_10k` | 10,084 | 682 M | 0.19% |
| `check_cold_100k` | 100,804 | 5.78 G | 0.41% |
| `check_wide_match` | 9,012 | 701 M | 0.24% |

## Against the targets [#against-the-targets]

Design §17.109 set a ceiling per program on the ratio of Sakura to hand-written JavaScript on
the same engine, and asked for a median of at most 1.5 over the fifteen programs. Those are
same-engine ratios, so a ratio of counts can be read against them
([design §17.265](https://github.com/icantcodefyi/sakuralang/blob/main/docs/design.md)). At
the largest size, six of the fifteen meet their ceiling on Node and five on Bun; the median is
2.24 on Node and 2.47 on Bun.

| Program | Ceiling | Node | | Bun | |
|---|---:|---:|---|---:|---|
| `fib` | 1.5 | 1.39× | met | 2.31× | miss |
| `countdown` | 1.3 | 1.57× | miss | 1.05× | met |
| `sieve` | 1.5 | 4.62× | miss | 6.98× | miss |
| `nbody` | 1.3 | 11.90× | miss | 8.89× | miss |
| `spectral_norm` | 1.3 | 4.49× | miss | 2.39× | miss |
| `binary_trees` | 1.5 | 1.41× | met | 2.54× | miss |
| `pipeline` | 2.0 | 0.29× | met | 2.65× | miss |
| `rbtree` | 1.5 | 2.52× | miss | 2.47× | miss |
| `map_set` | 3.0 (Bun 5.0) | 3.28× | miss | 3.85× | met |
| `shapes` | 1.5 | 2.24× | miss | 2.29× | miss |
| `tokenizer` | 1.5 | 0.30× | met | 1.17× | met |
| `json_validate` | 2.0 | 1.39× | met | 3.01× | miss |
| `deltablue` | 2.0 | 6.06× | miss | 6.23× | miss |
| `handled` (depth 1) | 2.0 | 0.88× | met | 1.14× | met |
| `contracts` | 2.0 | 2.69× | miss | 1.81× | met |

Read plainly: on the JavaScript backend, most of the suite runs within 1.5–3× of hand-written
JavaScript, and five programs are 4.5–12× behind: `nbody`, `deltablue`, `map_set:int`,
`handled_fwd` and `sieve`. Where Sakura is ahead (`pipeline`, `tokenizer`, `handled` on Node)
the cause is a specific mechanism, fusion of the pipeline, ownership of the consumed string, or
a handler compiled to a direct call, and it says nothing about other programs. The WebAssembly
backend is first-release code: 2–124× behind hand JavaScript by count on this suite, far from
native, with its worst cells on contracts, handlers and scalar loops. The native backend is
experimental, left out of release binaries, and not measured here.

Design §17.109 also set an envelope against Rust, Go, OCaml and Koka. Those ceilings were
set on wall time, and a count across different engines is not a time, so the envelope is
reported as information and not as a release gate (design §17.265). On these counts it is not
met: Sakura WebAssembly on Node is a geometric mean of about 27× Rust and 25× Go on the numeric
kernels, 11× OCaml and 12× Koka on the allocation kernels, with `tokenizer` the one kernel
inside its ceiling. The native backend, experimental today, is where that comparison will be made again.

What comes next, in order, each landing with a counted before and after against the blessed
baseline: flat `Float`, `Bool` and `Int` lists and in-place `Float` records (`nbody`,
`spectral_norm`, `sieve`); the `Int`-key path of `Map` (`map_set:int`); what remains in
`deltablue`; forwarding handlers (`handled_fwd`); then the WebAssembly cliffs, starting with
contracts and the superlinear `tokenizer` cost (design §17.265 (6)).

## Spread [#spread]

The spread is (max − min) / median over a cell's five pairs.

| Engine | Cells | Median spread | Cells over 1% |
|---|---:|---:|---:|
| Node, JavaScript backend | 198 | 0.14% | 4 |
| Node, WebAssembly backend and its hand JS | 198 | 0.30% | 12 |
| Wasmtime | 95 | 0.11% | 1 |
| Native (Rust, Go, OCaml, Koka) | 282 | 0.09% | 26 |
| Bun | 198 | 1.35% | 116 |
| Compiler processes | 6 | 0.28% | 1 |

On Node and Wasmtime every cell over 1% is one pair out of five; the middle three agree within
1%, so the median holds. Most native cells over 1% are Go, whose garbage collector runs
concurrently. Bun is noisier: 48 of its cells spread over 1% even in their middle three, and
JavaScriptCore's concurrency switches did not narrow them. Read a Bun count with its spread.

## Reproduce [#reproduce]

On a Mac with the toolchains pinned in
[`bench/toolchains.toml`](https://github.com/icantcodefyi/sakuralang/blob/main/bench/toolchains.toml):

```sh
git checkout 320c9980
bench/bench run --suite js --instructions --json js.json
bench/bench run --suite wasm --instructions --json wasm.json
bench/bench run --suite native --lane matched --instructions --json native.json
bench/bench compile --instructions --json compile.json
```

Counts from another machine are not comparable with these: a different CPU, OS or engine
build retires different instructions. The full result files, with every size, every raw
count and the wall time beside it, are in
[`bench/results`](https://github.com/icantcodefyi/sakuralang/tree/main/bench/results)
(`2026-10-19-320c9980-*`). The procedure that turns a run into a baseline is
[`docs/release.md` §2](https://github.com/icantcodefyi/sakuralang/blob/main/docs/release.md#2-instruction-count-baseline-and-bless).