Skip to content

Search wahloon.net

Search models, news, guides and FAQs.

All news
News#qwen#moe#35b-a3b#abliterated#release

MoE abliteration arrives for Qwen3.6-35B-A3B

The 35B-A3B MoE base is now the most interesting size/quality trade-off for local inference. Here are the abliterated and uncensored builds worth testing.

3 min readMarkdown
On this page

Mixture-of-experts models are changing the local hardware conversation. Qwen3.6-35B-A3B activates a fraction of its parameters per token, so it can deliver quality closer to a larger dense model while running at the speed of a much smaller one. The catch is memory: all experts still need to be resident.

Two community builds are now in the catalog: a plain abliterated variant from huihui-ai and an aggressive uncensored fine-tune from HauhauCS.

Why 35B-A3B is interesting

The “A3B” in the name means roughly 3B active parameters per token. In practice, that translates to:

  • Fast generation relative to the total parameter count.
  • Memory usage closer to a 35B model than a 3B model.
  • Good quality per token on general tasks.

For a 24 GB GPU, it is right at the edge. The catalog lists roughly 21 GB at Q4, which leaves little room for a long context. A 32 GB card or a high-memory Mac is a more comfortable fit.

Huihui-Qwen3.6-35B-A3B-abliterated-MTP

The huihui-ai build is the straightforward abliterated option. It is GGUF-only, which keeps things simple for Ollama, LM Studio, and llama.cpp users. The “MTP” suffix refers to the base model's multi-token prediction variant; use the chat template specified on the model card.

ollama run hf.co/huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated-MTP-GGUF:Q4_K_M

This is the one to try first if you want abliteration without additional fine-tuning.

HauhauCS Aggressive

The HauhauCS release is an uncensored fine-tune of the same base. It targets very low refusal rates and is available in GGUF and Transformers formats. If the plain abliterated build still hedges on your prompts, this is the next step up in aggressiveness.

The usual caveat applies: aggressive fine-tunes can drift in style and instruction following. Test on your tasks rather than assuming the base model's behavior carries over.

Hardware notes

  • 24 GB VRAM: Q4 is possible, but keep context modest (8k–16k) and close other GPU applications.
  • 32 GB VRAM or 32 GB+ unified memory: more comfortable; 16k–32k context is realistic.
  • 16 GB VRAM: expect heavy offload and slow generation. A 27B dense model at Q3 or a 12B model will feel better.

MoE offloading is less forgiving than dense offloading because expert routing can touch different weights per token. If you must offload, benchmark before committing.

MoE vs. 27B dense

A 27B dense model at Q4 needs 16–18 GB and is easier to fit. The 35B-A3B needs more memory but can be faster per token once loaded. Which one feels better depends on your bottleneck:

  • Limited VRAM: 27B dense.
  • Limited patience for slow generation: 35B-A3B, if it fits.
  • Long context: 27B dense is usually easier to fit with a large KV cache.

What to watch

The MoE ecosystem moves quickly. New quantizations and updated base revisions can change the memory profile. Check the model card for the current file list and recommended settings before downloading.

Browse the model catalog for both builds, or read the quantization guide if you are deciding between Q4, Q5, and Q6.