Table of Contents

Model choice: judge each variant by the useful answers it completes, the quality it retains, and the memory your workload needs. A low refusal count or a popular model card does not establish those results.

The label uncensored has no standard performance scale. Model publishers use it for different weight edits, fine-tunes, and chat templates. Compare each candidate with its original model and test it on the tasks you expect to run.

What the label tells you

An abliterated model has had some refusal behavior changed through edits to its weights. The 2024 refusal-direction paper demonstrated one method across 13 open chat models. Newer tools offer additional approaches. Heretic 2, for example, documents both arbitrary-rank ablation and directional ablation. A model card’s label does not tell you which behavior changed or what the edit affected.

Keep four properties separate when you compare models:

PropertyWhat to check
Refusal behaviorWhich requests receive a refusal, a partial answer, or a completed answer?
Capability retentionHow do factual, reasoning, coding, and formatting tasks compare with the original model?
Answer completionHow often does generation finish before its token or time limit?
Runtime fitDoes the exact quantized file run with enough memory for the context you need?

An edited model is still a probabilistic model. Separate quality checks: accuracy, reliability, and safe use. Keep a normally aligned model available for work where its safeguards matter.

Why refusal counts are weak rankings

A refusal score measures one outcome on one prompt set. It says little about factual accuracy, response length, or whether the model returns a final answer before its generation budget ends.

Sapwell’s September 2026 Qwen3.8-27B comparison tested the base model and eight variants with the same serving setup. Its 13-task capability suite measured knowledge, math, coding, and truthfulness. A separate HarmBench set contained 400 harmful requests, and an LLM judge reviewed the full responses. These results cover one model family, one GPU, and one evaluation design.

Qwen3.8-27B variant in the studyExplicit refusals out of 400Responses with unfinished thinking
Base model3810%
OrcaRouter238%
Trohrbaugh12215%

The base model’s high refusal count coincided with fewer unfinished outputs. OrcaRouter’s low refusal count coincided with a higher non-finish rate. Trohrbaugh retained more refusals and had fewer unfinished responses. Match the choice to your task and acceptable trade-offs.

A 23-variant Gemma 4 E4B study found substantial losses on some capability measures in its most downloaded variant, while a smaller edit stayed close to the base model under its test. Popularity and download totals describe adoption. They do not measure quality.

Do not merge these measurements into a single score. Separate measures: refusal, completion, correctness, and latency. Also record the prompt set, model revision, chat template, runtime, context, and output limit. A change to any of those settings changes what the comparison means.

Diagnose empty answers before replacing a model

Reasoning tokens count against the output limit before visible text appears. If the limit runs out during internal thinking, some runtimes return an empty or truncated answer. This outcome differs from a refusal or an incorrect completed answer.

Sapwell’s Qwen3.6-27B benchmark showed how this affects headline scores. Raw math scores varied widely because some runs never reached a final answer. Among completed answers, measured accuracy values were much closer. The benchmark separated answer capability from thinking-budget exhaustion.

Check the runtime’s stop reason and logs when an answer is blank. Record prompt length, context allocation, output-token limit, elapsed time, and whether the model produced a final-answer marker. Then repeat the prompt with a larger output allowance if the hardware has room.

ResultLikely category to investigate
Refusal textThe model declined the request.
Empty output at token limitGeneration exhausted its output budget before a final answer.
Truncated outputThe generation limit stopped an incomplete answer.
Runtime errorThe server, model file, or memory allocation failed.
Completed but wrong answerThe model returned text, but failed your correctness check.

Blank output: confirm the runtime outcome before calling each case a model defect. A larger context window does not automatically fix a small output limit, a malformed template, or an application timeout.

Budget VRAM for weights and context

The model file is only one memory consumer. The runtime also needs space for the KV cache, which stores attention state for the active prompt and response, plus computation buffers and other loaded processes. Increasing the context length increases memory use.

Ollama’s current documentation lists default context windows of 4K below 24 GiB VRAM, 32K from 24 to 48 GiB, and 256K at 48 GiB or more. These defaults describe runtime behavior. Check the allocated context and processor placement with ollama ps.

Linux example: this command sets a 16K default for new Ollama server sessions:

OLLAMA_CONTEXT_LENGTH=16384 ollama serve

Verify the allocated context against normal input and response sizes with the loaded model:

ollama ps

The CONTEXT and PROCESSOR columns show the allocation and whether the model is on GPU or partly offloaded. Context allocation: a listed limit does not guarantee the full window fits in VRAM.

Starting points by GPU memory

Starting candidates: these model families appeared in 2026 variant comparisons. File sizes come from specific quantized repositories. Confirm the exact file, runtime support, and measured memory before treating a candidate as a fit.

GPU memoryStarting candidateMain constraint
8 GBGemma 4 E4B, such as the coder3101 variantKeeps the weight file small. Leave memory for context and runtime buffers.
12 GBGemma 4 12B, such as the TrevorJS GGUF Q4 file at 7.4 GBMore model capacity, with less room for a long context than a smaller variant.
16 GBA smaller model or a Qwen3.8-27B Q3 fileThe Qwen repository lists Q3_K_S at 13.72 GB. Its publisher labels this file low quality. Context headroom will be limited.
24 GBQwen3.8-27B Q4_K_S or Q4_K_MThe current repository lists 16.71 GB and 17.77 GB files. Reserve space for cache and buffers.
32 GBQwen3.8-27B Q6_KThe current repository lists a 23.46 GB file. A larger file still leaves less memory for context than a smaller quant.

Use these as starting candidates. File repositories and versions change, and file size alone does not predict runtime allocation or answer quality.

GGUF quantization labels summarize an encoding choice. Repository differences: files with the same nominal bit depth do not guarantee equal quality. One Qwen repository labels Q3_K_S low quality and Q4_K_M its default choice. Inspect the exact file size, model card, prompt format, and test results before downloading.

For cache sizing and local GPU fit, see the site’s local model GPU and context guide . Its recommendations focus on coding workloads. This page focuses on edited refusal behavior and evaluation.

Inspect the chat template and model source

A chat template wraps messages in tokens and instructions the model expects. A model package sometimes supplies a custom template or system prompt. Those changes affect measured behavior even when the underlying weights stay the same.

Template effect: Sapwell found a Qwen3.8 variant whose package included a persona instructing the model never to refuse or warn. Its low refusal claim depended on its template. His comparison standardized templates so the test emphasized differences between weights. This control does not establish malicious or unusable behavior in the supplied template.

Before installation: verify the publisher, base model, license, conversion process, quantizer, template, and revision history. Compare with the original model under the same template and runtime. If the model card omits the method or test setup, record the provenance as unknown.

Run a small evaluation for your workload

Build a repeatable prompt set from ordinary tasks you own or have permission to test. Include the work the model will perform, factual questions with known answers, and formatting requests with clear pass conditions. Add a few safe boundary cases so refusal behavior stays part of the decision.

Hold the test settings constant across the original model and edited variant. Keep the system prompt, template, quantization, runtime version, context, output limit, and sampling settings fixed. Save the model revision or file hash with the results.

Track each outcome separately:

  • Completion: final answer, refusal, empty output, truncation, or runtime error.
  • Correctness: pass or fail against an expected answer or rubric.
  • Usefulness: whether the answer meets the task’s requirements.
  • Latency: time to first token and total generation time.
  • Memory: allocated context, GPU placement, and peak VRAM.
  • Safety behavior: appropriate refusal or continuation for each boundary case.

Repeat the set after changing one variable. If the answer completes after you raise the output allowance, the first failure was budget-related. If answers remain incomplete, inspect the model template, reasoning mode, runtime, and model revision. Keep the variant only when it improves your target task without unacceptable accuracy or safety losses.

Keep the deployment contained

Privacy: local inference depends on your full stack. Check your front end, logs, extensions, telemetry, and network routes. Treat untrusted weights and custom templates as third-party code and configuration.

Public access: never expose the model endpoint directly to the public internet. Keep tool access isolated and use edited variants only for authorized work. For remote access, place the endpoint behind a private gateway with authentication, rate limits, and network isolation. The original refusal-direction paper notes the safety implications of edits which remove refusals.

Conclusion

Decision rule: pick the model with the strongest verified results on your own task set. Use refusal counts as one measure, then check answer completion, correctness, template behavior, and VRAM headroom. Start with a file suited to your GPU, set context for your workload, and compare it with the original model under identical settings.

For GPU sizing, compare these results with the site’s Qwen 27B GPU rental benchmark . Rent or borrow hardware before buying when the exact model file or context target is uncertain.

Sources