Ternary Bonsai 27B runs locally on a 24 GB Mac.

Run the public checkpoint, inspect measured traces, and separate vendor claims from what local tests actually prove.

Ask the docsOpen the 27B field guide

Learning paths: Inference The Hard Way and Kubernetes The Hard Way

run-bonsai-27b.sh
measured on a Mac M4 Pro
$ uv tool install mlx-lm


$ mlx_lm.server \
    --model prism-ml/Ternary-Bonsai-27B-mlx-2bit \
    --host 127.0.0.1 --port 8080


# Controlled 4K runs on a 24 GB M4 Pro
[ TTFT: 14.506 s | Decode: 21.55 tok/s ]
[ TTFT: 13.383 s | Decode: 22.67 tok/s ]


# Exact public package: 8.49 GB

Start Here

PrismML Developer

PrismML is a family of small open-weight language models, led by Ternary Bonsai 27B. Work through these docs and you will be able to run the 27B checkpoint on hardware you already own, inspect measured local traces, and explain where its compressed weights help without overclaiming.

Start with Bonsai 27B

Run the public ternary checkpoint, read the local M4 Pro traces, and choose an MLX workflow.

Read what people are saying

The Information, PrismML, Khosla team amplification, launch posts, and the claims that still need testing.

Understand the ecosystem

See how data, models, compression, runtimes, and devices fit together, and where PrismML sits.

Contribute your first page

Found a gap or an error? Here is how to fix it.

Ask instead of searching

The fastest way through these docs is to ask them. Ask the docs runs 1-bit Bonsai 1.7B in your own browser over WebGPU: type a question, watch the inference trace show how the answer was made, and follow the source links it cites into the site. Nothing you type leaves your device. It is also the search bar: press the search box anywhere and hand your query to Bonsai from there.

The main story: Ternary Bonsai 27B

The public MLX checkpoint puts a 27B parameter model in an 8.49 GB package. In our controlled LM Studio runs on a 24 GB M4 Pro Mac, it decoded at 21.55 and 22.67 tokens per second at a 4,096 token context. It also returned a valid structured tool call. A reasoning check showed the important limit: a 16 token budget produced reasoning but no answer, while a 256 token budget returned 42 after 191 reasoning tokens.

Start with the Bonsai 27B field guide, then read the 26 focused lessons and the link-first ecosystem roundup. The roundup centers The Information's report, then separates PrismML's claims, investor amplification, platform reactions, and our local evidence.

Choose how you connect

  • With a coding agent. Point Claude Code, Cursor, or any agent at /llms.txt; every page is raw Markdown if you append .md to its URL. Use these docs with AI has a paste-ready starting prompt.
  • By hand. Start with the llama.cpp recipe and you will have a local model answering prompts in minutes.

What you will be able to do

  • Run Ternary Bonsai 27B locally with MLX or LM Studio and serve it as an OpenAI-compatible API.
  • Measure tokens per second, time to first token, memory, and energy on your own device, in a format others can compare against.
  • Verify the whitepaper's claims, from the 1.15 GB file size to the benchmark scores, against official sources and your own runs.
  • Explain the results to your community with the baselines and caveats stated, using the same numbers the paper reports.

Check what you need

Pick the environment that matches how deep you want to go. Every path helps benchmark Bonsai.

EnvironmentGood forPrerequisites
Local machineRunning Bonsai 27B, measuring speed and memory, offline demosApple silicon Mac with 24 GB unified memory recommended, plus at least 12 GB of free disk
Notebook (Colab or Jupyter)Re-running single benchmarks such as GSM8KA GPU runtime and EvalScope
Cloud GPUsReproducing the paper's full evaluation matrixThe paper's setup: EvalScope v1.4.2, vLLM 0.15.1, H100-class GPUs

The whitepaper benchmarks page explains what to run in each environment and what the paper's numbers should look like when you get there.

Understand the whitepaper benchmarks

The goal of these docs is an ecosystem that understands the paper's results, not just repeats its headlines. The 1-bit Bonsai 8B whitepaper reports a 1.15 GB model scoring 70.5 across six benchmarks, generating up to 8.4x faster than FP16, and using 4x to 6x less energy per token. The benchmarks guide walks through what each of the six tests measures, where Bonsai wins, where it gives up points, and how to check the numbers yourself. You can trace any claim to its source in the claim and source matrix.

Learn how it works underneath

Two deployed courses go deeper than the guides on this site. Take them in either order.

  • Inference The Hard Way teaches how Bonsai inference works, from the tokenizer and KV cache through kernels, bandwidth, and energy.
  • Kubernetes The Hard Way teaches the infrastructure under production serving: machines, certificates, etcd, the control plane, workers, and networking.

The technical guides cover the same concepts in shorter form, starting with the KV cache.

Next steps

See something wrong? Fix it.

Next
Orientation