Local AI in 2026: A 27B Model Beats Sonnet 4.6, and the Hardware Window Is Closing

A downloadable model which fits on a single consumer graphics card now outscores a frontier API model on the aggregate intelligence index. Qwen3.8 27B is an open-weight vision-language model released in August 2026. Its ternary repack, Bonsai 2 27B, squeezes the same weights into 5.53 GiB. Together they mark the moment local inference stopped being a hobbyist compromise and became a genuine substitute for a paid API.

The hardware side of this story is closing fast. Memory prices are the worst they have been in two decades, and they are dragging graphics card prices up with them.

This article covers what the benchmarks show, what the two models are under the hood, where the open model still loses, and why the next twelve months are the wrong time to wait.

The Short Answer #

The intelligence gap between a local 27B model and a paid frontier-adjacent API model has closed, and the remaining advantages of the paid model are narrower than the price difference suggests.

Three things changed at once:

• Open 27B models reached frontier-adjacent scores. Qwen3.8 27B posts a 33.7 Intelligence Index against Sonnet 4.6 at 30.1, on the same evaluation suite.
• Ternary quantization cut the memory requirement by roughly two thirds. Bonsai 2 27B runs the full 27B model in 5.53 GiB, which fits on cards costing a few hundred dollars.
• Memory supply went in the wrong direction. Conventional DRAM contract prices rose 90 to 95 percent quarter over quarter in early 2026, the steepest quarterly jump TrendForce has recorded, which pushed next-generation GPUs out to 2028.

Buying or building now is the cheap option. This is not a sales pitch, it is what the supply data says.

Watch the Single-GPU Test #

The claim above deserves a demonstration rather than a table. Bijan Bowen tested Bonsai 2 27B on a single-GPU machine and asked whether it is the best model available for a single card today.

Watch for the output quality rather than the tokens-per-second figure. Speed is the easy part to fix.

Two Names, One Model #

The two models worth understanding here are not competitors. They are the same weights at two different precisions.

The Ollama library page for qwen3.8 lists the 27B model at 18 GB with a 256K context window, vision support, tool calling, and a separate MLX build for Apple Silicon. The ternary repack on Ollama is a mirror of Prism ML’s Apache-2.0 GGUF packs, republished with matching checksums.

So the practical question is not which model to choose. It is how much memory you are willing to spend for the same intelligence. The ternary version costs roughly a third of the memory and gives up little.

The Benchmark Comparison #

Artificial Analysis scores both models on the same suite. The results deserve to be read row by row rather than summarized, because the two models win in different places.

The open model leads nine of eleven comparable rows, including every aggregate index.

Where the Open Model Leads #

The three aggregate indexes carry the headline. Intelligence 33.7 against 30.1, Coding 68.1 against 63.0, Agentic 45.8 against 31.8. The agentic gap is the widest of the three, and it matters most for anyone running automated multi-step workflows rather than single prompts.

For percentile context, Artificial Analysis places Qwen3.8 27B better than 86 percent of models on the agentic index and 76 percent on coding. Sonnet 4.6 sits at 68 percent and 72 percent respectively. On graduate-level science reasoning the open model posts 90.5 percent on GPQA Diamond.

Where Sonnet 4.6 Still Leads #

Two rows go the other way, and neither is minor.

AA-Omniscience measures whether a model answers knowledge questions correctly in the first place. Sonnet 4.6 scores 40.9 percent against 15.6 percent. This is a wide gap in factual reliability, and it is the strongest single argument for paying for a premium model.

The non-hallucination row needs care. Qwen’s 69.7 percent against Sonnet’s 51.6 percent looks like a clean win, and in isolation it is. But the metric describes the share of non-correct answers which avoid fabrication. A model with lower accuracy produces more non-correct answers, so the two percentages describe different populations and are not directly comparable. Read the accuracy row first.

Sonnet also takes SciCode at 50.1 percent against 46.6 percent, and it reports figures on two evaluations the open model’s card does not list, IFBench and Terminal-Bench Hard. Those cannot be compared at all.

Two Honest Caveats #

The release dates are six months apart. Sonnet 4.6 shipped in February 2026. Qwen3.8 27B shipped in August 2026. A February frontier model against an August open model is not a like-for-like contest.

Sonnet 4.6 is no longer Anthropic’s frontier. Sonnet 5 and Opus 5 now sit above it in the lineup. The open 27B caught a mid-2026 flagship. It has not caught the current top of the stack, and anyone claiming otherwise is overreading the table.

Benchmarks measure benchmark performance. The only score which matters is the one your own workload produces.

What Ternary Weights Change #

Bonsai 2 is not a smaller model. It is the same 26.90B model at a much lower numerical precision, and the published GGUF metadata makes the trade explicit.

Ternary weights mean each parameter carries one of three values instead of sixteen. The compression is the entire point, and it is why a full 27B model fits in 5.53 GiB rather than 18 GB.

Measured Throughput #

Benchmarked with llama-bench on an RTX 5090 32 GB, every layer offloaded, flash attention enabled, mean of three runs:

PQ2_0 prefills 2.03 times faster than PTQ1_0 while decode is a statistical tie, so PQ2_0 is the sensible default and PTQ1_0 is for when the last 1.2 GiB matters. Through llama-server with a 65K context, decode held at 136.5 tok/s on a single streaming request.

Decode speed is the figure you feel in daily use. 135 tokens per second comfortably outruns reading speed. Latency is not the reason to run local inference, since a hosted API is fast enough. Cost, privacy, and control are.

The Catch Nobody Mentions #

Stock Ollama cannot load these files. The ternary kernels live in Prism ML’s llama.cpp fork. Verified against Ollama 0.34.2, the load fails with unsupported tensor "output.weight" size overflows. The runtime recognizes the qwen35 architecture but has no pq2_0 or ptq1_0 anywhere in its libraries, and no Hadamard activation runtime.

There is a second trap worth knowing before you spend an evening on it. The F16 pack of this family looks like a safe fallback and is not. Its tensors are ordinary F16 and F32, so it parses and loads without complaint, then decodes incoherent text. The weights sit in a rotated basis declared in prism.hadamard.* metadata, and a runtime either applies the matching rotation or produces nonsense. At 50.1 GiB the F16 pack also will not fully offload to a 32 GB card.

Read the model card before downloading. A community mirror is not a supported build, and this one says so plainly on the page.

The MoE Wave Behind This #

Dense 27B models are half the story. Mixture-of-experts models activate a small slice of their parameters per token, which changes the economics again.

Qwen3 Coder Next is the one to watch for local work. 80B total parameters with only 3B active per token, positioned for always-on agent deployment inside CLI and IDE environments. A model activating 3B parameters generates at the speed of a 3B model while drawing on 80B of stored capability. The upstream notes describe performance comparable to models with 10 to 20 times the active compute.

The dense 27B is the safe choice today. Sparse models are where the efficiency curve is still bending.

The Hardware Window Is Closing #

Here is the part of this story with a deadline attached.

Local inference got cheap at the exact moment the memory it depends on started getting expensive. Those two trends are on a collision course, and the collision is already visible in retail pricing.

What Happened to Memory Prices #

Conventional DRAM contract prices rose 90 to 95 percent quarter over quarter in the first quarter of 2026, the steepest quarterly increase TrendForce has on record. NAND flash climbed 55 to 60 percent in the same window.

The pressure did not ease. TrendForce data puts second-quarter DRAM up another 58 to 63 percent and NAND up 70 to 75 percent. Even the third-quarter moderation, DRAM up 13 to 18 percent and NAND up 10 to 15 percent, is inflation stacking on top of two record-setting quarters rather than relief.

The cause is structural rather than cyclical. AI data center buildouts absorb memory capacity which used to serve phones, laptops, and game consoles, and memory makers allocate to the higher-margin orders first. Analysts tracking module supply project DRAM chip supply to module makers falling by more than 70 percent in 2027.

Intel’s chief executive put a number on it. Speaking at the AI Infrastructure Summit in Santa Clara on September 15, 2026, Lip-Bu Tan said memory prices had already risen “five, six, and seven times” over prior levels, and warned 70 to 80 percent of a low-cost laptop’s bill of materials now goes to memory alone. He has separately said there is no relief until 2028.

Independent reporting on the per-area comparison between DRAM wafers and leading-edge logic wafers carries an important caveat. Spot prices are not contract prices, and DDR5 spot procurement is thin enough for the comparison to be suggestive rather than settled. The direction of travel is not in question.

What It Did to Graphics Cards #

Memory is a large share of a graphics card’s bill of materials, so DRAM pricing feeds straight into GPU pricing.

Those increases arrived in coordinated waves through 2026 rather than as isolated spikes. Nvidia raised prices in January, again in May on the RTX 5090 specifically, and again later in the year. AMD followed in August with an increase of at least 10 percent across select Radeon cards and GDDR6 memory kits, attributed directly to memory costs.

The next generation slipped instead of shipping. Nvidia’s RTX 60 series is now expected in 2028, skipping 2027 entirely, with memory availability cited as the cause.

The card you buy today is not competing against a cheaper card next year. It is competing against a more expensive version of itself.

Why Waiting Is the Expensive Choice #

The usual hardware advice is to wait for the next generation. In a supply-constrained memory market this advice works against you.

Three points make the case.

Prices are already up and still climbing. A 90 to 95 percent quarterly jump in DRAM contract prices is not a blip which reverses on a quarterly cycle. TrendForce is forecasting further growth in the third quarter, not decline, and Intel’s chief executive has put relief at 2028.

There is no 2027 product to wait for. When next-generation consumer GPUs slip to 2028, the waiting strategy stretches to a two-year horizon. This is most of a hardware refresh cycle spent waiting for nothing.

The models are good enough already. This is the part which changed in 2026. Two years ago, local hardware bought a visibly compromised experience. Today the same money buys a 27B model at 135 tokens per second with a 262K context window, outscoring a premium API model on the aggregate intelligence index.

If you were waiting for local models to get good, the wait is over. If you are waiting for hardware to get cheaper, the supply data says you are waiting until 2028.

What to Build #

Matching hardware to the models above is straightforward, because ternary weights shrank the requirement.

A few practical notes which matter more than the tier table:

• Buy memory, not the flagship GPU. The 5.53 GiB ternary pack means the entry tier is genuinely usable. Spending the difference on system RAM and storage helps more than chasing a halo card at 2026 prices.
• Unified-memory systems are the value play. A machine with 64 GB or more of shared memory runs larger sparse models at a fraction of the cost of a discrete card with the same capacity.
• Power and cooling count. A rig running inference for hours behaves differently from one running benchmarks for minutes.
• Run the ternary pack alongside a smaller model. A 3B model answers quick questions while the 27B handles the hard ones.

Where Premium Models Still Earn Their Price #

The case for the premium tier is real, and it is narrower than the marketing suggests.

Factual reliability is the honest answer. AA-Omniscience accuracy is 40.9 percent for Sonnet 4.6 against 15.6 percent for the open 27B. When a wrong answer costs a client, a diagnosis, or a legal position, a 25-point accuracy gap matters more than any aggregate index.

Long agentic chains are the second answer. Sonnet 4.6 reports 53.0 percent on Terminal-Bench Hard and 75.7 percent on τ²-Bench Telecom, which measure sustained tool use and multi-turn agent behavior. The open model’s card reports no comparable figures, and the absence is not encouraging.

The third answer is operational. A hosted API brings no driver updates, no quantization quirks, no fork of llama.cpp to track, and no rotated weight basis to get right. Someone else handles uptime and someone else gets paged at 3 AM.

The Question Worth Asking #

So where does the premium tier stand outside the enterprise?

For knowledge-critical work, it still stands. Research, legal analysis, medical questions, financial modeling, and anything where a confident wrong answer is expensive. This is a genuine category, and the accuracy gap is a genuine reason to pay.

For generative work, the justification is thinning fast. Drafting, summarizing, refactoring code, reformatting data, classifying documents, and running automated pipelines are all tasks where output is verified before it is trusted. The accuracy gap costs little when a human or a test suite checks the result first. For those workloads, paying per token for a 30.1 index while a 33.7 index runs free on hardware you already own is a hard position to defend.

The decision has moved from capability to consequence. Two years ago the question was whether a local model was good enough to do the work at all. Today the question is what a mistake costs and how you would catch it.

The enterprise premium buys reliability, compliance, and somebody else’s on-call rotation. Everyone else is paying for a difference which stops mattering the moment a test suite runs.

Key Takeaways #

• A local 27B model now outscores Claude Sonnet 4.6 on the aggregate intelligence index, 33.7 against 30.1, and leads on coding, agentic tasks, long context, and graduate-level science reasoning.
• Sonnet 4.6 still wins on factual accuracy, 40.9 percent against 15.6 percent on AA-Omniscience, which is the strongest remaining argument for the premium tier.
• Ternary quantization is the change which makes this practical. Bonsai 2 27B runs the same 26.90B model in 5.53 GiB instead of 18 GB, and both are Apache 2.0.
• Stock Ollama cannot run the ternary packs. They need Prism ML’s llama.cpp fork, and the F16 fallback silently produces incoherent output rather than failing.
• Measured speed is 135 tokens per second decode on an RTX 5090, which outruns reading speed by a wide margin.
• Memory prices are the reason to act now. DRAM contract prices rose 90 to 95 percent in a single quarter in 2026, and next-generation consumer GPUs slipped to 2028.
• The RTX 5090 went from $1,999 to over $5,000 in under two years, and every tier moved with it.

Next Steps #

1. Download the model and test it against your own workload: Ternary Bonsai 2 27B on Ollama
2. Get the original weights at full precision if you have the memory: Qwen3.8 on Ollama
3. Try the model before you build anything through the free hosted endpoint: Qwen3.8 27B free on OpenRouter
4. Compare against the paid tier on the same prompts: Claude Sonnet 4.6 on OpenRouter
5. Size a rig for the tier you need using the memory table above, and buy the memory before the graphics card
6. If you have a Raspberry Pi sitting idle, check whether it can host something smaller: Which AI Models Can Run on a Raspberry Pi 4 or 5?

Related Articles #

References #

1. Ternary Bonsai 2 27B - Ollama model mirror
2. Qwen3.8 - Ollama model library
3. Qwen3.8 27B (free) - OpenRouter benchmarks and pricing
4. Claude Sonnet 4.6 - OpenRouter benchmarks and pricing
5. Bonsai 2 27B First Test - Is THIS the BEST Single-GPU AI Model? - Bijan Bowen
6. Artificial Analysis - independent model evaluations
7. TrendForce - DRAM and NAND contract price forecasts
8. Memory chips are now more expensive than compute chips on a per-area basis - Tom’s Hardware
9. GPU Relief Pushed to 2028 as DRAM Jumps a Record 95% - Tech Insider
10. Intel’s Tan: Memory Up 7x, No Relief Until 2028 - shattered.io
11. Prism ML llama.cpp fork - ternary kernels
12. Ternary-Bonsai-2-27B-gguf - upstream weights