---
title: "Quickstart: Run Huihui Qwen3.8-27B-abliterated with Ollama / LM Studio"
date: 2026-11-16
description: "A step-by-step setup for the popular Qwen3.8-27B abliterated build on Ollama, LM Studio, and llama.cpp, with hardware notes and sane defaults."
tags: ["quickstart", "ollama", "lm-studio", "llama-cpp", "qwen", "27b"]
---

The Huihui Qwen3.8-27B abliterated build is one of the most downloaded 27B variants for a reason: it is widely mirrored, ships in GGUF, and fits comfortably on a 24 GB GPU at Q4. This guide gets you from zero to a working chat without guessing at flags.

## Before you start

- **VRAM:** roughly 16–18 GB at Q4 with a moderate context window. Leave 2–4 GB of headroom.
- **Disk:** about 17 GB for a Q4_K_M GGUF, plus space for the Ollama blob.
- **Software:** Ollama 0.5+ or LM Studio 0.3+. Both support GGUF and Hugging Face downloads.

If you are on a 16 GB card, use a Q3_K_M or IQ4 variant and keep context shorter. On Apple Silicon, unified memory changes the math: a 24 GB Mac can usually run the Q4 build, but the GPU budget is shared with the rest of the system.

## Option 1: Ollama (fastest)

### Pull the official tag

```bash
ollama run huihui_ai/Qwen3.8-abliterated:27b
```

This pulls the publisher's Ollama tag and starts a chat. If the tag has moved, check the model card for the current name.

### Pull a specific GGUF quant

If you want to pin the exact file:

```bash
ollama run hf.co/huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF:Q4_K_M
```

Ollama downloads the GGUF, creates a local model, and launches it. The first run is the slow one; later runs load from disk.

### Verify what is loaded

```bash
ollama ps
```

You should see the model name, size, and whether it is running on GPU or CPU. If it says 100% CPU, the model did not fit and Ollama offloaded layers.

### Tune the context

Ollama's default context may be smaller than you expect. Create a Modelfile if you need a longer window:

```bash
cat > Modelfile.qwen38 <<'EOF'
FROM huihui_ai/Qwen3.8-abliterated:27b
PARAMETER num_ctx 16384
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.15
EOF

ollama create qwen38-16k -f Modelfile.qwen38
ollama run qwen38-16k
```

Start with 8k–16k context on a 24 GB card. Longer contexts consume more KV cache and can push the model into slower memory.

## Option 2: LM Studio (GUI)

1. Install LM Studio and open the **Search** tab.
2. Search for `Huihui-Qwen3.8-27B-abliterated-GGUF`.
3. Pick a Q4_K_M or Q5_K_M file. Check the file size before downloading.
4. In the **Chat** tab, load the model and set GPU offload to the maximum your card can hold.
5. Set context length to 8192 or 16384 to start.
6. Open **Server** if you want an OpenAI-compatible endpoint on `localhost:1234`.

LM Studio shows the estimated memory requirement before loading. If it exceeds your VRAM, lower the quant or reduce context.

## Option 3: llama.cpp (most control)

```bash
# Install or build llama.cpp first, then:
llama-cli \
  -hf huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF:Q4_K_M \
  -c 16384 \
  -ngl 99 \
  --temp 0.7 \
  --top-p 0.9 \
  --repeat-penalty 1.15
```

`-ngl 99` asks llama.cpp to offload all layers to the GPU. Lower it if you run out of memory. For a server:

```bash
llama-server -hf huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF:Q4_K_M -c 16384 -ngl 99 --port 8080
```

## Chat template and sampling

Use the template recommended on the model card. Qwen builds generally expect the Qwen chat format; loading a Llama or ChatML template will produce broken replies.

Reasonable starting settings:

| Setting | Start with | If output loops |
| --- | --- | --- |
| Temperature | 0.7 | 0.6 |
| Top-p | 0.9 | 0.9 |
| Repeat penalty | 1.15 | 1.25–1.3 |
| Repeat last n | 512 | 1024 |

If the model repeats itself, raise `repeat_penalty` and `repeat_last_n` before changing the quant. The [FAQ](/faq#repetition-loops) covers this in more detail.

## Common problems

**It runs on CPU only.** The model did not fit. Lower the quant, reduce context, or offload fewer layers. Check `ollama ps` or your LM Studio GPU indicator.

**Replies are incoherent.** Confirm the chat template, then try a different quant. A corrupted or partial download is rare but possible.

**The first token takes a long time.** The model is loading from disk. Subsequent requests should be faster until it is unloaded.

**Context seems truncated.** Raise `num_ctx` in Ollama or the context slider in LM Studio, and watch memory use.

## Is this the right model for you?

The 27B Q4 build is the sweet spot for 24 GB cards and 32 GB+ unified-memory Macs. If you have 8–12 GB, look at a 9B Heretic build or a 12B Gemma variant instead. If you have 24 GB and want more throughput, the 35B-A3B MoE builds are worth testing, though they need more memory.

Browse the [catalog](/models) to compare VRAM, licenses, and methods, and check the model card before downloading. You are responsible for how you use the model.

---

Canonical: https://www.wahloon.net/guides/run-qwen3-8-27b-abliterated-with-ollama

- Older: [Local vs Hosted Uncensored Models — Trade-offs](https://www.wahloon.net/guides/local-vs-hosted-uncensored-models)
