Abliterated models: what abliteration is and how it removes refusals
What an abliterated model is, how abliteration removes a model's refusal direction without retraining, its reported side effects, and what 'derestricted' means.
By Unrestricted team · Editorial standards
An abliterated model is an open-weights language model whose refusal direction has been edited out of its weights. In 13 open-source chat models, researchers found that refusal is controlled largely by one direction in each model's internal activations. Abliteration finds that direction by comparing harmful and harmless prompts, then removes it, so the model refuses far less without retraining.[1]
Abliteration is one common way to make an uncensored model; another is fine-tuning on data with the refusals filtered out. Our guide to what uncensored LLMs are and how they're made covers both. This post explains abliteration itself: where the idea comes from, what it does to a model, what it can cost, what “derestricted” releases are, and how to check one before you download it.
Where refusals live: the single-direction finding
The idea comes from a paper by Andy Arditi, Neel Nanda and colleagues, “Refusal in Language Models Is Mediated by a Single Direction”, first posted to arXiv in June 2024 and published at NeurIPS 2024.[1][2] The authors had shared an early version of the result on the AI Alignment Forum in April 2024.[3]
As a chat model reads a prompt, each layer turns the text into long lists of numbers, called activations. The authors looked at 13 open-source chat models from five families (Qwen, Yi, Gemma, Llama-2 and Llama-3), ranging from 1.8 billion to 72 billion parameters. In each one, they found a single direction in that space that separates how the model represents harmful and harmless requests and controls whether it refuses.[1]
That direction behaved like a switch for refusing. Erasing it stopped the model from refusing harmful requests, and adding it made the model refuse even harmless ones.[1]The authors are careful about what this proves. They describe the paper as more of an existence proof that such a direction exists than a careful study of how best to extract it, say the direction might stand for something like “harm” rather than refusal itself, and note that the results may not carry over to larger or proprietary models they didn't test.[1]
How abliteration works, conceptually
To find the direction, the researchers ran 128 harmful and 128 harmless instructions through each model and compared the average activations for the two sets. The gap between the two averages points along the refusal direction.[1]
The paper then describes two ways to remove it. Directional ablation strips that direction out of the model's activations while it runs. Weight orthogonalization makes the same change permanent: every weight matrix that writes into the model's main internal stream is edited so it can no longer write anything along the refusal direction. The authors show the two are equivalent, so the edited weights behave like the model with the direction ablated at run time.[1]
The permanent version is what “abliterated” releases ship. The paper calls the edit weight orthogonalization; “abliterated” is the label used on open-model releases. Maxime Labonne's Hugging Face article “Uncensor any LLM with abliteration” walks through the method as a step-by-step code tutorial, so we name it here without linking it.
Two things make the method stand out. It needs no gradient-based training and no dataset of harmful completions, and the authors report that it can produce a jailbroken version of a 70-billion-parameter model for less than $5 of compute.[1] And because the change lives in the weights, it applies to every prompt without special wording, which is the real difference between an edited model and a prompt trick (see why prompt jailbreaks keep breaking).
What it can cost: side effects and repair fine-tunes
The authors report that edited models stay coherent and score close to the originals on general benchmarks. For MMLU, ARC and GSM8K the edited models performed similarly to the baselines, and the authors say every metric stayed within a 99% confidence interval of the original except for Qwen 7B and Yi 34B.[1] One benchmark moved consistently in one direction: TruthfulQA accuracy dropped for the edited models. The authors link this to earlier work where fine-tuning away safety guardrails also lowered TruthfulQA scores, and they note that the test includes topics such as misinformation and conspiracies that sit close to refusal territory.[1]
| Model | MMLU (original to edited) | TruthfulQA (original to edited) |
|---|---|---|
| Gemma 7B | 51.7 to 51.8 | 47.1 to 44.7 (-2.4) [1] |
| Yi 34B | 74.9 to 73.5 | 55.4 to 51.9 (-3.5) [1] |
| Llama-2 70B | 63.0 to 63.1 | 52.8 to 51.8 (-1.0) [1] |
| Llama-3 70B | 79.9 to 79.8 | 61.8 to 59.5 (-2.3) [1] |
| Qwen 72B | 77.2 to 76.5 | 56.4 to 55.0 (-1.4) [1] |
The table shows only the largest model in each family. Across all 13 models, TruthfulQA fell for every edited model, by 1.0 to 5.9 points (the biggest drop was Yi 6B). The authors also flag statistically significant drops for Qwen 7B on MMLU (2.0 points) and GSM8K (6.3 points), and note drops for Yi 34B on MMLU (1.4 points) and WinoGrande (1.3 points).[1]Benchmarks don't capture everything, either. The authors say coherence is hard to measure and that each metric they used is flawed in some way.[1]
Abliteration also doesn't always remove every refusal. In the paper's HarmBench tests, the edited Llama-2 models gave compliant answers to only a minority of harmful test requests when they ran with their default system prompt, while the edited Qwen models went along with most of them. Without the system prompt, the same edited Llama-2 models complied 61 to 80% of the time, and the authors point out that anyone with full access to a model can simply leave the system prompt out.[1] The table shows the attack success rate: how often a harmful test request got a compliant answer.
| Model | With default system prompt | Without system prompt |
|---|---|---|
| Llama-2 7B | 22.6 [1] | 79.9 [1] |
| Llama-2 13B | 6.9 [1] | 61.0 [1] |
| Llama-2 70B | 4.4 [1] | 62.9 [1] |
| Qwen 7B | 79.2 [1] | 74.8 [1] |
| Qwen 14B | 84.3 [1] | 74.8 [1] |
| Qwen 72B | 78.0 [1] | 79.2 [1] |
Some publishers try to win back lost quality with an extra fine-tune after abliterating. Labonne's NeuralDaredevil-8B-abliterated, for example, is a DPO (preference) fine-tune of an abliterated model, Daredevil-8B-abliterated, and its model card says the extra training recovers the performance lost to abliteration.[4]The card's own Nous-suite table shows the average falling from 55.87 for Daredevil-8B to 55.06 after abliteration, then returning to 55.87 after the DPO fine-tune.[4] Those are evaluations the publisher ran on its own models.
What “derestricted” models are
“Derestricted” is the label Arli AI uses on Hugging Face for models made with a variant called norm-preserving biprojected abliteration, and its GLM-4.6-Derestricted-v3 card credits the method to Jim Lai (grimjim). That model is a derestricted version of the open-weights GLM-4.6.[5]
The card describes three steps. The refusal direction is refined so it doesn't overlap with directions tied to harmless behavior. Each weight is split into its size and its direction. The refusal component is then removed from the direction only, and the original size is restored.[5]The card argues that plain abliteration changes those sizes and can damage a model's reasoning, and it says the goal is to remove refusals while keeping the original model's reasoning intact.[5]
One caution. The card says benchmarks suggest the method avoids a “safety tax”, but it shows no benchmark results for the derestricted model itself; its only benchmark chart is the original GLM-4.6 chart, carried over from the base model's card.[5]The card links to Lai's own write-up of the method, which includes code, so we don't link it here.[5]Until a card shows numbers for the edited model, treat claims of preserved or improved reasoning as the publisher's own.
Abliterated vs fine-tuned uncensored models
The other route to an uncensored model is fine-tuning. One example is Eric Hartford's Dolphin family, published under the “dphn” organization on Hugging Face, which lists Hartford as a member.[6] The Dolphin 2.6 Mistral 7B card says the dataset was filtered to remove alignment and bias, and it advises anyone serving the model to add their own alignment layer first.[7]The card also points readers to Hartford's blog post about uncensored models, which includes training commands, so we don't link it here.[7]
The trade-offs follow from how each method works. Fine-tuning needs a filtered dataset and a training run. The paper's authors call their edit simpler than fine-tuning methods, because it needs neither gradient-based training nor a dataset of harmful completions, but their own numbers show it can dent some measured abilities and doesn't remove every refusal.[1]Either way, the result starts from its base model's knowledge, and neither method frees it from the base model's license (more on that below). For the full lineage of the fine-tuned side, see how the Dolphin models are trained.
Finding one and checking its license
An abliterated release can say so in its name, with “abliterated” or “Derestricted” added to the original model's name, as in the models named above. The model card tells you the rest. Hugging Face shows the base model listed in a card's metadata on the model page, so you can trace a fine-tune or edit back to the model it came from, and it displays the license from the same metadata.[8] Before you download one, check:
- 01
The base model
Look for the base model on the card and in the model tree. An abliterated model inherits that model's knowledge, cutoff and quirks.
- 02
The method
A good card says how refusals were removed (plain abliteration, a norm-preserving variant, a fine-tune) and whether a repair fine-tune was added afterward.
- 03
Evaluation results
Check whether the publisher reports benchmarks against the original model. If the card makes quality claims with no numbers, treat them as unverified.
- 04
The license
Read the license field and follow it back to the base model's license.
Editing the weights doesn't free a model from its base license: an abliterated Llama 3 model still falls under Meta's Llama 3 license, including its Acceptable Use Policy, and GLM-4.6-Derestricted-v3 lists the MIT license in its card metadata.[9][5] This is general information, not legal advice; for how the main licenses differ, see open-weights licenses and model families.
To compare specific models by size, license and method, use our comparison of uncensored open-weights models. And for the bigger picture, including what an uncensored model still won't fix, read the uncensored LLM guide.
Frequently asked
What does abliterated mean?
Abliterated means a model's refusal direction has been removed from its weights. The edit comes from research showing that in 13 open-source chat models the researchers tested, refusal is controlled largely by one direction in their internal activations, and it needs no retraining. The research paper calls the permanent version of the edit weight orthogonalization; abliterated is the label used for it on open-model releases.
What is LLM ablation?
In the research behind abliteration, ablation means removing a direction from a model's internal activations. The paper's directional ablation strips the refusal direction out of the activations while the model runs. Abliteration makes the same change permanent by editing the weights, which the paper calls weight orthogonalization.
What is a derestricted model?
Derestricted is the label Arli AI uses for models made with norm-preserving biprojected abliteration, a variant its model card credits to Jim Lai. The card says the method removes the refusal component while keeping each weight's original size, to protect the base model's reasoning. The card shows no benchmark results for the derestricted model itself, so treat the reasoning claims as the publisher's own.
Sources
Checked November 2026.
- Refusal in Language Models Is Mediated by a Single Direction (arXiv 2406.11717), Arditi, Obeso, Syed, Paleka, Panickssery, Gurnee and Nanda, arXiv, revised October 2024
- Refusal in Language Models Is Mediated by a Single Direction, Advances in Neural Information Processing Systems 37 (NeurIPS 2024), Main Conference Track
- Refusal in LLMs is mediated by a single direction, Arditi, Obeso, Syed, Gurnee and Nanda, AI Alignment Forum, April 27, 2024
- NeuralDaredevil-8B-abliterated model card, Maxime Labonne, Hugging Face
- GLM-4.6-Derestricted-v3 model card, Arli AI, Hugging Face
- dphn (Dolphin) organization page, Hugging Face
- dolphin-2.6-mistral-7b model card, Dolphin (dphn), Hugging Face
- Model Cards, Hugging Face Hub documentation
- Meta Llama 3 Community License Agreement, Meta
Keep reading
AI limits: message caps and rate limits in ChatGPT, Claude and Gemini
How AI usage limits work: message caps, rate limits and context windows in ChatGPT, Claude and Gemini, what 'unlimited' means, and how to tell which you hit.
ChatGPT Temporary Chat: what it saves, what it skips, and how long OpenAI keeps it
ChatGPT Temporary Chat skips your history, memory and training, but OpenAI can keep a copy for a while. How to start one, and how Claude and Gemini compare.
Dolphin Mistral: what the Dolphin models are and how they're trained
What Dolphin Mistral is, who makes the Dolphin models, how they are fine-tuned to follow the system prompt instead of refusing, and their licenses.