Bonsai 2 27B Tested: 6GB 27B Model on Mac mini M6 and RTX 3090

Bonsai 2 27B is said to shrink a 27B model to 6GB while keeping 98.2% of the original's intelligence. We ran it side by side with the original on a Mac mini M6 32GB and on an RTX 3090 rented in the cloud. The conclusion: it is clearly faster, the wait for an answer shrinks by only about 20%, and on Korean word choice it invents words that do not exist. This post reorganizes the measurements from the YouTube video so they are easier to read.

Conclusions first

• Generation speed was 1.8 times faster on the 3090 and 2.3 times faster on the Mac mini.
• But the time from asking a question to receiving the answer dropped by only about 20%, because the model thinks (reasons) longer.
• Math, fact lookup, code and finding information in a long text came out almost the same as the original.
• On Korean word choice it produced non-existent words such as "빠리", "속직", "속소하다" and "순식". The original gave real words all four times.

The numbers in this post come in three kinds

We rented one 3090 in the cloud for about 35 minutes, at a cost of about USD 0.20. Both the original and Bonsai fit in 24GB, so they could be compared directly on the same card.

What Bonsai 2 27B is

Bonsai 2 is a model in which the weights of Qwen3.8 27B are converted to just three values: -1, 0 and +1 (one scale factor per 128 weights). According to the maker, it keeps 98.2% of the original's intelligence on average across 14 benchmarks (thinking mode), and the license is Apache 2.0. The files come in two forms. PTQ1_0 (5.95GB) is packed tightly, and PQ2_0 (7.21GB) is packed into 2-bit slots. The weights inside are the same and only the packing differs.

The original 16-bit is about 54GB, and the commonly used 4-bit (UD-Q4_K_XL) is 17.6GB, so by size alone Bonsai is about one third of the original 4-bit. That is why a 27B model fits even on a 12GB graphics card or a 16GB Mac. However, it needs a dedicated llama.cpp fork, and it cannot be read by standard llama.cpp or LM Studio.

The measurement conditions were the dedicated fork prism-b10743, a context of 65,536, temperature 1.0, top_p 0.95, top_k 20, min_p 0.05, an output limit of 16,384, and the default reasoning effort (xhigh), with the same conditions given to both models.

Speed: the 3090 and the Mac mini M6

We measured the speed of generating 128 tokens (tg128, average of 3 runs) with the standard benchmark tool, llama-bench.

On the 3090 the larger PQ2 was fastest (in the maker's table, the A100, a card of the same generation, was also faster with PQ2), while on the Mac the smaller PTQ1 was actually faster than PQ2. GPU memory on the 3090 with a 65K context was 20.5GB for the original and a little over 10GB for Bonsai (PTQ1 10.2GB, PQ2 11.3GB). A viewer who runs the original on a 3090 with LM Studio told us 48.5 tokens/s, while we got 42.5 with our settings (the viewer's report and our measurement use different settings). The MLX value was measured one token at a time, so it may come out lower than with other methods. The maker's published speeds are 28.1 tokens/s on the M5 Pro and 47 on the M5 Max (for PQ2_0; the video card marks them as an "earlier build"), and the video found no M6 measurement anywhere yet.

The catch: faster, but the wait dropped only 20%

This model thinks before it answers. Asked for the six metropolitan cities, Bonsai mistakenly took Ulsan for a special self-governing city, worried at length that it could not reach six without Seoul, thought for 1,103 tokens and answered in 19 seconds. Given the same question, the original thought for only 176 tokens and answered in 5 seconds.

We ran 6 questions four times each on the 3090. The average time to receive answers to all of them was 52 seconds for Bonsai PQ2, 55 seconds for PTQ1 and 67 seconds for the original. Thinking tokens per question averaged 428 for PTQ1, 481 for PQ2 and 365 for the original. The speed is 1.5 to 1.8 times higher, but the wait fell by only about 20%. On the Mac mini it was the same: over 4 questions, Bonsai PTQ1 took 159 seconds and the original 201 seconds, so 2.3 times the speed became about a 21% reduction, and just reading an 8,000-token text took both models close to 40 seconds (a weak point of the Mac). The "thinking forever and returning an empty answer" complaint common in the community did not happen once in this measurement, and the longest thinking was 2,130 tokens. That seems to be thanks to the generous output limit of 16,384. With the lower-effort medium setting, thinking fell by 22% on PQ2 and stayed almost the same on PTQ1 (428 to 418).

Accuracy: where it held and where it broke

These are the auto-graded results from four runs on the 3090. The math word problem, the Python function (7 tests) and finding a fact in an 8,000-token text were 4/4 for both Bonsai and the original. On the six metropolitan cities, Bonsai scored 3/4, and in one run out of four it included Seoul and left out Ulsan or Incheon (the original scored 4/4). A password hidden in the middle of a 40,000-token work log was found by both Bonsai versions in 37 seconds and by the original in 43 seconds (1 run).

Where it broke was the question asking for five words similar in meaning to "빠르다" (fast). Bonsai PTQ1 settled on "빨리" in its thinking but wrote the non-existent word "빠리" in its answer, and it also mixed in words that only change the form, such as "빠름" and "빠른". Asked four times, Bonsai produced words like 속직, 속소하다 and 순식, and with the reduced-thinking medium setting it wrote only "빠르다" five times. The original gave real words every time: 신속하다, 재빠르다, 민첩하다. The MLX 2-bit version, run once on the Mac mini, included Seongnam among the metropolitan cities. Ternary compression preserved math and fact lookup, but it seems to have eroded rarely used Korean knowledge and a feel for words.

The community reactions collected in the video point the same way. They included a measurement that, despite the 98.2% claim, the rate of matching the original's first word was 77.8% (96.9% for 4-bit), complaints that the thinking is too long, a report that all 25 Japanese questions were answered correctly, and the view that it does well apart from awkward Korean word choices (reports by others).

The video's verdict and cautions before using it

• With a 24GB graphics card (3090 or 4090), the original 4-bit: 42 tokens/s is enough and the Korean is accurate.
• With a 12 to 16GB card or a 16GB Mac, Bonsai is the only way to run a 27B model.
• On a 32GB Mac, Bonsai for code and summaries, and the original for Korean writing and translation.
• Before using it: it needs the dedicated llama.cpp fork, the output limit should be generous at 16,384 or more, and PQ2 was faster on the 3090 while PTQ1 was faster on the Mac.

Limits and reference dates

• The sample is small: 6 questions × 4 runs on one 3090 and 4 questions once on the Mac mini. There are also only a few Korean word questions.
• Speeds are for one build of the dedicated fork (prism-b10743) and may change with other tools or future updates.
• The M5 Pro and M5 Max speeds and the 98.2% figure are the maker's claims, and we did not measure them.
• Reference dates: measured 2026-09-28, video published 2026-09-29. This post is not a purchase recommendation.