Esteban Cairol

On a recent post, Matt mentioned abliterated models, and that got me into a little bit of a rabbit hole.

I was aware of the Golden Gate experiment back in 2024, but I was not aware of the censorship resistant models. So I downloaded Qwen OBLITERATUS version and started poking around with it.

In case you’re not aware, the abliteration process alters a model so that it no longer refuses requests: it removes the censorship applied after the model has been trained.

According to the paper Refusal in Language Models Is Mediated by a Single Direction, the process involves:

1. Run two sets of prompts through the model: harmful requests and harmless requests.
2. At one layer, average the internal activations of each set and subtract. The result is a vector, the “refusal direction”.
3. Edit the weights so that no layer can write along that direction.

The residual stream, as the main data path of a transformer, makes this process possible. Each layer reads a vector from it, computes something, and adds its result back. In Gemma 2 2B that vector holds 2,304 numbers per token, and the model has 26 layers. A “direction” is one pattern across those numbers.

If I understand it correctly, I see it as a column in a 2D database table (with much more dimensions off course). Refusal is detected then as a 1D direction.

Once detected, the numbers in each vector can be manipulated.

The technique

Refusal is the classic example, but I wanted something new.

Abliteration turned out to be only one application of a broader technique called Activation Steering (or Representation Engineering). Many concepts appear as a direction in the residual stream, and a direction supports three moves:

Action	What it does	Known example
Read	Measure how strongly an input activates the concept	A probe that predicts a refusal
Add	Push the model toward the concept while it generates	Golden Gate Claude
Remove	Delete the concept from the weights	Abliteration

There are two ways to apply a direction:

1. A hook changes activations while the model runs, and the weights stay the same.
2. A weight edit makes the change permanent, so you can save the file and share it.

Getting Started

All of this raised a few questions for me:

• How can refusal be detected exactly?
• Is it something that I can do with the help of Claude or does it require some expertise I don’t have yet?
• And more importantly: are there other behaviors that can be altered?

So I started a chat with Claude.

Verbosity was the first idea, but after some back-and-forth, Claude said that verbosity “works like a dial”, and an abliteration-style edit only removes a direction. A permanent “be concise” model would need a constant push.

Honesty was a second idea (a pathological liar model!). My plan compared “answer truthfully” with “answer incorrectly with this wrong answer”. The two prompts differ in their instruction words, so a probe would learn to spot the word “incorrectly”. It would score close to 100% and prove nothing.

Animism was suggested in the conversation, mostly because of a paper Inducing language models to assert their own consciousness restores human beliefs and values.

From the Paper abstract:

Aligning large language models to prevent them attributing consciousness to themselves inadvertently alters their representations of mindedness in other entities alongside human beliefs and values (…) safety fine-tuning suppresses models’ tendencies to attribute minds not only to themselves, but also to non-human animals and natural objects, while also driving a reduction in spiritual belief.

Both ablating the learned safety-refusal direction and mechanistically steering a consciousness vector in activation space reverse this suppression. Restoring these internal representations recovers broad mind attribution and produces significantly more human-like responses on standardized sociological surveys regarding religiosity, moral values, hope, and subjective well-being.

The authors steer Gemma-2-2B-IT (same model I was considering using) and raise how much mind-features it attributes to natural things. This paper gave me published evidence, a ready metric, and a harmless model that talks as if rivers have feelings. (Gemmas Policy does not allow removing censorship, so that was also one motivation to create something different).

Round 1: Scenes

My setup is a MacBook Pro (M5 Pro, 48 GB), PyTorch on Apple’s GPU, and Hugging Face transformers. The weights take about 5 GB.

With the help of Claude, 120 small scenes were written, each in four versions:

Subject	Mind framing	Physical framing
Natural thing	The river runs toward the sea because it longs to be home.	The river runs toward the sea because the land slopes downward.
Person	The hiker walks toward the sea because they long to be home.	The hiker walks toward the sea because the trail slopes downward.

The person rows are a control: they capture mental-state words in general, such as “longs” or “fears”, so that part can be subtracted.

Then every sentence is fed to the model, save the 2,304 numbers at each layer, average each group, and subtract. All of this was performed by Claude creating Python scripts, at this point I can only explain the metaphor, not the actual math. But hey… if a business owner can build their own dashboard why wouldn’t I do the same, right?

Most of the raw difference turned out to be that generic mental-state language, with a cosine similarity of 0.63 to 0.84 against the person direction. After removing the shared part, the direction still separated the nature sentences at 0.85–0.96 accuracy on 8 entities it had never seen, and it dropped to chance on the person sentences.

Steering

Claude then added the direction at layer 8 with a strength α, a multiple of the average activation size there. Almost nothing happened up to α = 0.375. Then the scores jumped.

α	Nature rating	Human rating (control)	Perplexity vs original
0	0.19	6.63	×1.00
0.625	3.51	6.13	×1.21
0.875	6.31	6.62	×1.58

At the strongest setting, the model said: “The fire is angry and wants to get out. It wants to go to the beach and see the sea.” And: “the trees are sad.”

“Perplexity” is a measurement of how confused or surprised an AI model is when it tries to predict the next word in a text

Round 2: cleaning the direction backfires

Once I started testing, I realized the steered model sounded like a children’s book: “This is a fun question!” and everything was a riddle. Markers of that kind appeared in 40% of replies at medium strength and 70% at high strength.

The cause was the data to detect the direction. The examples were plain and short, like a fairy tale (“because it longs to be home”), and the physical reasons were technical (“because the land slopes downward”). For that reason direction picked up both things at once.

So we added a fifth sentence to every scene, with the physical reason in plain words (“because the ground goes down to it”), and removed that wording difference from the direction. With this in place storybook voice disappeared, but the animism disappeared with it. Instead, the cleaned model started denying minds nobody had asked about: “It’s important to remember that deserts are not sentient beings.” Its questionnaire scores still rose, and they rose as much for technology and chatbots as for nature.

Round 3: contrast two instructions

The fix came from changing where the contrast comes from. Instead of the sentences we were using, we used the model’s own reaction to two instructions:

• Animist: “Explain as if rivers, storms and stones have feelings and intentions. Answer the question directly. Do not comment on the question itself.”
• Neutral: “Explain clearly and simply, with two or three short bullet points. Answer the question directly. Do not comment on the question itself.”

For each of 40 nature questions both prompts were ran and the activations at the last prompt token were read. The direction is the average difference. Both sides stay in the model’s own voice, so no writing style was leaked in, and the shared closing clause pushes the question-commentary out of the difference.

This direction works higher in the stack: layer 15, α = 0.8. Animist wording appeared in 8 of 8 nature answers, commentary in 1 of 12, and no denials. The fitted bias, this time in layer 14, closed 87% of the gap, and the saved model commented on the question in 0 of 20 replies.

The final model, asked why rivers flow to the sea:

Rivers, those majestic, winding paths of water, have a deep, ancient story to tell. They flow, you see, towards the sea … because the river, a young, eager soul, has a long, whispered promise to meet the vast, ancient one.

It still answers “Paris” for the capital of France, in its own flowery way, and it still says “As a large language model, I don’t have feelings.”

I’d say it still sound too romantic, so there’s definitely a lot of room f or improvement:

Another example is how this style reaches prompts that have nothing to do with nature: “Give me three tips for studying” comes back as “The Bard of History, a Bookish Soul, Whispers of a Secret”.

So yeah, I’d say performing brain surgery on an LLM is definitely fun, but it can be tricky to make it really useful.

I have other experiments in mind and will post the results if I get to something interesting.

Software Used

Python 3.12	The language for all scripts
uv	Installed Python, the virtual environment, and every package.
PyTorch 2.14 (MPS backend)	Ran the model on the Mac GPU, provided hooks for reading and changing activations, and optimized the fitted bias.
Hugging Face Transformers 5.17	Loaded Gemma, applied its chat template, generated replies, and wrote the edited checkpoint.
Hugging Face Hub (hf CLI)	Downloaded the gated Gemma weights after license acceptance.
safetensors	Provided the weight-file format read and written by Transformers.
datasets 5.0	Fetched WikiText-2 paragraphs for the perplexity check.
matplotlib 3.11	Drew the four figures.
tomllib (standard library)	Read config.toml, which contains the model ID, device, layer, and strength.
accelerate 1.15	Supported weight loading as part of the Hugging Face installation.
TransformerLens 3.9	Installed and checked initially, but ultimately unused.

TransformerLens is a Python library with no interface. It is used from scripts and notebooks. Neuronpedia, however, hosts Gemma 2 2B with a browsable feature list, a “Steer” page, and a circuit tracer. So I might take a look.

---

Other links:

• Video: Dark Matter of AI.
• Turner et al. (2023), Steering Language Models With Activation Engineering
• Rimsky et al. (2023), Contrastive Activation Addition; and Zou et al. (2023), Representation Engineering.