-FF9F43?style=for-the-badge&logo=apple&logoColor=white){kind=icon}
What is this?
A Node.js install harness that sets up Qwen 3.8 27B as a fully uncensored local model, using weight-level abliterated GGUF weights plus a reinforcement SYSTEM prompt, runnable through Ollama or Apple's MLX.
A Node.js install harness that sets up Qwen 3.8 27B as a fully uncensored local model, using weight-level abliterated GGUF weights plus a reinforcement SYSTEM prompt, runnable through Ollama or Apple's MLX.Is this the model itself, or just scripts to set it up?It's an installer and profile harness, not the weights: it pulls the abliterated GGUF checkpoint (published by orcarouter on Hugging Face) and configures Ollama/MLX for you, without bundling the weight files in the repo.
It's an installer and profile harness, not the weights: it pulls the abliterated GGUF checkpoint (published by orcarouter on Hugging Face) and configures Ollama/MLX for you, without bundling the weight files in the repo.How do I get it running?Run `node bin/install.cjs` on Mac or Windows to pull the right quant and set up Ollama, use `bash bin/install-mlx.sh` for the faster Apple MLX backend, then chat with `node examples/chat.cjs`.
Run `node bin/install.cjs` on Mac or Windows to pull the right quant and set up Ollama, use `bash bin/install-mlx.sh` for the faster Apple MLX backend, then chat with `node examples/chat.cjs`.Do I need Ollama and a specific GPU?Yes — you need Ollama 0.17.1 or newer for the Qwen 3.8 architecture, plus either an Apple Silicon Mac with 32GB+ unified memory (Metal) or an NVIDIA GPU with 12GB+ VRAM on Windows (CUDA).
Yes — you need Ollama 0.17.1 or newer for the Qwen 3.8 architecture, plus either an Apple Silicon Mac with 32GB+ unified memory (Metal) or an NVIDIA GPU with 12GB+ VRAM on Windows (CUDA).Does abliteration guarantee the model never refuses?Mostly, since refusal directions were orthogonalized out of 131 weight matrices, but if you run the older `--legacy` system-pack-only mode (prompting instead of weight editing), refusals can still slip through.
Mostly, since refusal directions were orthogonalized out of 131 weight matrices, but if you run the older `--legacy` system-pack-only mode (prompting instead of weight editing), refusals can still slip through.Does this work on both Apple Silicon and Windows GPUs?Yes, it's built for both: Mac (M-series, Metal or MLX) and Windows (NVIDIA GPUs, CUDA), with separate quant profiles tuned for each.
Yes, it's built for both: Mac (M-series, Metal or MLX) and Windows (NVIDIA GPUs, CUDA), with separate quant profiles tuned for each.How is the abliterated version different from the earlier system-pack version?The system pack (v1) just adds a SYSTEM prompt discouraging refusals on official weights — surface-level and still overridable; the abliterated version (v2) permanently strips refusal directions out of 131 weight matrices at layer 38, removing the refusal circuits at the weight level.
The system pack (v1) just adds a SYSTEM prompt discouraging refusals on official weights — surface-level and still overridable; the abliterated version (v2) permanently strips refusal directions out of 131 weight matrices at layer 38, removing the refusal circuits at the weight level.Which quant file should I pick?Q3KM (13.5GB) is best for Mac M5 Pro and 12GB GPUs with max context headroom, Q4KM (16.8GB) is the default for Windows 24GB GPUs, and options go up to Q80 (29GB) for near-lossless quality on bigger rigs.
Q3KM (13.5GB) is best for Mac M5 Pro and 12GB GPUs with max context headroom, Q4KM (16.8GB) is the default for Windows 24GB GPUs, and options go up to Q80 (29GB) for near-lossless quality on bigger rigs.Why does Mac M5 Pro use Q3KM instead of Q4KM?macOS reserves about 25% of the 48GB unified memory, leaving roughly 36GB usable; Q4KM's ~20GB footprint is tight with other apps open, while Q3KM's ~16.5GB leaves 23GB+ headroom for the OS and larger context windows.
macOS reserves about 25% of the 48GB unified memory, leaving roughly 36GB usable; Q4KM's ~20GB footprint is tight with other apps open, while Q3KM's ~16.5GB leaves 23GB+ headroom for the OS and larger context windows.What should I check if inference feels slow?Run `ollama ps` to check for a CPU/GPU split, force all layers onto GPU (already set via Modelfile.mac with numgpu 999), shrink the context window to cut KV-cache memory, or switch to the MLX backend for 30-50% faster inference on Apple Silicon.
Run `ollama ps` to check for a CPU/GPU split, force all layers onto GPU (already set via Modelfile.mac with numgpu 999), shrink the context window to cut KV-cache memory, or switch to the MLX backend for 30-50% faster inference on Apple Silicon.What are similar projects?Qwen3.8-Flash-Next, qwen38-mtp, slotstream, rizzo-flow, Strata, decider, SemIf-OpenJev, Bonsai-demo
Qwen3.8-Flash-Next, qwen38-mtp, slotstream, rizzo-flow, Strata, decider, SemIf-OpenJev, Bonsai-demoWhich topics is it in?uncensored, qwen3, model, llama-cpp, mlx, gguf, metal, installer, llm-inference, qwen, quantization, large-language-models
uncensored, qwen3, model, llama-cpp, mlx, gguf, metal, installer, llm-inference, qwen, quantization, large-language-models