Mach-1 Small benchmarks

Published result

In a hosted-GPU quality evaluation, the unweighted mean of Mach-1 Small's twelve benchmark-level score-retention ratios was 95.0% versus its own BF16 teacher. Mach-1 Small (Mach-1-Ternary-Additive-35B) is a 35B-A3B MoE compressed to about 1.7 bits per weight, in which every weight matmul is add/subtract-only over integer codes; trained scale surfaces sit over frozen integer codes via on-policy distillation. Its teacher is Qwen3.6-35B-A3B (BF16).

Retention is a compression-fidelity measure, not an absolute cross-model score:

retention = 100 × student score ÷ that student's own BF16 teacher score

A value above 100 on one benchmark is possible because evaluation scores have variance.

The comparison

All three arms have figures for all twelve benchmarks, so there is one mean per model over one common set:

Gemma sits well back because it retains only about 67% on AIME25 and AIME26 — the selective collapse on sustained reasoning that the Bonsai whitepaper itself describes. On the ten benchmarks excluding AIME the same comparison is 94.2% / 93.9% / 89.3%, so almost the entire Gemma gap comes from those two rows.

Mach-1 Small and Ternary Bonsai are measured end to end on one harness. Every row in both columns is our own run, including AIME and tau2 on the fixed external simulator, so those two compare benchmark for benchmark with nothing borrowed. The Gemma column is external — see comparability.

The Gemma values are recomputed from Table 14 of PrismML's Bonsai 27B whitepaper, pinned at repository commit f904ea2ae3bb48e664de3ef36f55b553211b0c3c and SHA-256 06451897df438a42d0f067a019db4e18969aa37aded7781a219f3b1fba54608b. The ten non-AIME Gemma rows reproduce this arm's previously published student and teacher scores exactly, which is what makes extending the same arm to AIME from the same table sound.

The aggregate values and protocol summary are available as a machine-readable result summary. This is not a sample-level reproducibility package: exact artifact hashes, run IDs, sample counts, prompt hashes, and raw outputs are not yet public.

Protocol

This is a hosted-GPU quality evaluation. It does not measure Apple Silicon latency or throughput.

Per-benchmark retention

Ordered by Mach-1 Small's retention. Mach-1 Small and Ternary Bonsai were run here; the Gemma column is PrismML's published measurement throughout. See comparability below.

Every Mach-1 Small and Ternary Bonsai figure above is our own measurement. Every Gemma figure is PrismML's published measurement — we did not run that model.

Underlying Mach-1 Small scores and teacher scores:

Comparability of the Bonsai and Gemma columns

Ternary Bonsai: measured here, all twelve rows. Nothing in that column is carried from the whitepaper any more. AIME25, AIME26, and tau2 on the fixed external simulator were all run on the same harness as Mach-1 Small, so the two columns are directly comparable benchmark for benchmark.

Where the whitepaper also publishes a figure for the same model, it runs higher than our measurement of it:

Some of that is the tau2 and AIME setups differing, and some is ordinary harness variance. The practical consequence is one-directional: published Ternary Bonsai numbers taken from the whitepaper are not comparable to this table, and mixing the two would flatter that column.

Gemma: entirely PrismML's published measurement. We did not run that model at all — every Gemma figure in this document is recomputed from Table 14. Its ten non-AIME rows reproduce the student and teacher scores we previously published for that arm exactly, which is what confirms the whole column has one source. Its tau2 in particular uses the whitepaper's own simulator, not the fixed external one behind the Mach and Bonsai figures.

One protocol difference remains inside our own run: IFEval is a mean of 5 independent runs for Mach-1 Small and a single run for Ternary Bonsai.

What the chart does and does not show

• Each compressed model is compared with its own teacher. The distance between models is not an absolute capability gap.
• The spread between Mach-1 Small and Ternary Bonsai is 1.4 points over twelve benchmarks but 0.3 over the ten excluding AIME. Almost all of it comes from AIME, where Mach-1 Small retains ~99% and Ternary Bonsai ~92%.
• The 95.0% vs 85.6% gap against Gemma is real but rests heavily on the two AIME rows, where Gemma retains about 67%. On the ten benchmarks excluding AIME the same gap is 94.2% vs 89.3%.
• Mach-1 Small is a 35B-A3B mixture-of-experts model; Ternary Bonsai is a dense 27B model. This is not a matched-architecture comparison.
• The Gemma values come from the Bonsai whitepaper, not the Mach/Bonsai same-harness run. They are ratios recomputed from the shared benchmark rows and are included as an external reference.
• The coding tasks used process-local execution rather than PrismML's sandbox. Same-harness ratios reduce those biases between the Mach and Bonsai arms, but do not eliminate all harness effects.
• Results can vary with prompts, sampling, evaluator versions, runtime, and hardware. A benchmark result is evidence for this configuration, not a guarantee for every task.

July 2026 local performance

The local figures were measured on Apple Silicon in a separate July 2026 run. Time to answer is prefill plus decode wall-clock time; lower is better.

"Intelligence per second" is an internal composite metric: each model's published retention mean multiplied by decode throughput from the local run. "Intelligence density per second" divides that value by measured model memory from the local run.

These are rescaled to the twelve-benchmark retention means above. Throughput and memory come from the July 2026 local run and did not change, so each value moved by exactly the ratio of new mean to old — which is why only the Gemma column moves: its mean fell from 89.3% to 85.6% once AIME was included, while the other two moved by a fraction of a point.

These measurements are indicative, not a guarantee for another machine or workload. Hardware, runtime, model artifacts, prompt and output length, cache state, thermal state, and repetition policy can change the result.