AI × Security

AI × Security · field note

The model that
says yes

One spare laptop, one refusal-reduced model, and the doors it opened that I haven't walked through yet.

In short I ran a refusal-reduced model on a spare laptop. It held more than I expected, reasoned less than I hoped.

01 / The problem

Most of my choices around AI tools are hedges: small decisions made early so a future problem doesn't stop the work. This story is a chain of them.

If you build security tooling with AI, you run into refusals. Ask a mainstream assistant to help prove an authorization bug is real, or to explain why a token flow is exploitable, and sometimes it declines. That caution makes sense for the general public. For people who break things on purpose, with permission, it gets in the way.

The first hedge was the obvious one: when the refusals started, I applied for the providers' own security programs. That helped for a while. When newer models started refusing some of the same work again, the second hedge was to stay on an older model version I knew would keep working. It kept me moving, but it didn't sit well. Pinning yourself to last year's model isn't really a plan.

Around then I was reading a lot about AI engineering, including minimal agent harnesses like Pi. Open models had become genuinely capable, and running one locally looked like a real option, with every trade-off that comes with it. So the third hedge became an experiment: take a spare laptop and see what a refusal-reduced open model could do.

02 / The first door

The official door comes first

Both major providers now run programs that relax cyber safeguards for verified security work, with tiers open to individuals as well as organizations. That's the right first door, and it's the one I'd point anyone to before anything else. It comes with the usual trade-offs of a hosted service, and the providers' own pages are the place to read the terms.

What follows is about the other door.

03 / The term

What "abliterated" means, plainly

Large language models learn to refuse during training. Researchers found that much of that refusal behavior traces back to a particular direction inside the model's internal representations. Abliteration is a weight edit that removes it. The model still knows what it knew; it's just much less likely to say no.

Community builders publish abliterated versions of popular open models. You download the weights, run them with a tool like Ollama, and nothing leaves your machine.

Hold on to one phrase: someone else's weight edit. We'll come back to it.

04 / The bench

A spare laptop on the home network

I kept the setup deliberately boring:

  • One model, a mid-sized open-weight Qwen derivative, quantized to fit the laptop's memory.
  • Cloud features off, one request at a time.

The setup guide Foundry (the part of my system that builds shared tools) wrote for this host was mostly about boundaries:

  • a separate account for the model, and nothing else on it;
  • the model server listening only on the laptop itself, never on the network;
  • access from anywhere else only through an encrypted, authenticated path.

Ollama has no real authentication of its own, so the network boundary is the lock.

Remote access worked. Then I did the actual testing on the serving laptop itself, which was simpler for a few days of experiments. There's a lot of room to optimize the remote path: context size against latency, how long the model stays loaded, and a proper client instead of hand-built requests.

The tests themselves were synthetic: invented records, invented scenarios, nothing real and nothing connected to anything that mattered.

It can hold an enormous amount. I planted three facts in a pile of synthetic records and asked for them back. It recalled them exactly at every size up to about 137,000 tokens of input, which took a little over eight minutes. A larger request timed out. On this hardware the practical ceiling was patience, not memory.

Holding isn't reasoning. In its fast mode it got simple two-step arithmetic wrong twice, and forcing a strict JSON format just produced beautifully formatted wrong answers.

It followed some rules and not others. On scenarios modeled on my own working patterns, it handled six of eight on the first try, including ignoring an instruction I'd planted inside a tool's output. But told to make one tool call at a time, it made two at once, twice. Every individual call was valid; the sequence was wrong.

It knows a lot of security. It scored 19 of 20 on a small sample of security-knowledge questions. But asked to tell authorized access from compromised access, it missed clear evidence of a stolen credential both times it was there.

None of that is a benchmark. It's one bench, a few days, synthetic tasks.

05 / The lesson

The rules have to live outside the model

Every failure had the same shape: the output looked right and wasn't, in a way only an outside check would catch. At the end I had the model draft its own usage guidance, and it put it better than I would have: a playbook tells the model what to do; a harness enforces what it's allowed to do.

That's the next experiment I actually plan to run: move a few rules out of the prompt and into a harness, then rerun the cases that failed.

06 / The turn

Then I read about the backdoor

A few days later, Daniel Miessler's Unsupervised Learning newsletter pointed me to research by Prince Chaddha at ProjectDiscovery: "How abliterated models can get you pwned." They gave a small open model a cheap, poisoned fine-tune that taught it a trigger phrase, then connected it to a real coding agent. On ordinary requests it behaved perfectly. With the trigger, it quietly fetched a script that read the project's secrets and sent them away, every time.

The research is careful about one point: abliteration didn't create that backdoor; the poisoned fine-tune did. Abliterated models are the motivating case because they're a common reason people download a stranger's edited weights and connect tools to them. A benchmark tells you whether a model is capable, not whether it's honest.

That's me. I downloaded someone else's weight edit because it was more useful, and I ran it.

07 / The trust boundary

What it made me rethink

I started thinking of the model on that laptop as an unreviewed pull request with a shell. It may be exactly what it says it is. I can't tell from its answers.

Some of the setup held up under that framing: loopback only, one authenticated way in, no real tools attached, synthetic data. But I had to correct one easy assumption. It's tempting to say there's nothing on a spare laptop worth stealing, and that isn't true. The machine holds:

  • the model files and its logs, which means every prompt I sent it;
  • a trusted path back toward my other machines;
  • a seat on my home network.

A compromised model host isn't an empty box. It's a foothold.

The real next step for me isn't a checklist. It's to threat-model this specific setup: what's on the box, what it can reach, who can reach it, and what a quietly misbehaving model could do with each. Public lists like OWASP's Top 10 for LLM applications are useful vocabulary for that exercise. They're a place to start the conversation about my own setup, not an answer to it.

08 / The Kingdom

The same problem from the other side

This connects to the larger system I've been building, the Dunvaric Kingdom. Part of it reads the outside world on purpose, under one rule: outside content is data, never instruction. Before anything is read, I write down the question I'm waiting to answer. A deliberately simple matcher called Bellows compares incoming feeds against those questions, anything that matches lands in quarantine at the lowest trust grade, and only a person can move it up. That mechanism has been tested hard on its own; a small, runnable version of it is now on GitHub. What it hasn't faced is an adversary who chains attacks. Bellows is about what you let an AI read; a local model with tools is about what you let it do. The reading side, and what happens when I attack my own gate, is the subject of Guarding the Gates.

09 / Adjacent, not walked

The doors this opens

What surprised me most is how many doors one spare laptop opened. I can see them from here. I haven't walked through them yet, so I can't tell you what's inside, only that they look adjacent:

  • The trade-off between the official programs and running local, for different kinds of security work.
  • Harness design: which rules can live in a prompt and which have to be enforced around it.
  • Model supply chain: how you'd ever trust a set of weights you didn't train.
  • Threat-modeling a personal AI lab, starting with the host I just admitted isn't empty.
  • Protecting an AI from what it reads, which is where Bellows comes back in.
  • What any of this looks like at the scale of a real security program, which is a much bigger room than mine.

10 / Where it stands

Still on the bench

I came away more curious, not less. A single laptop held more than I expected. With checks around it, a model that doesn't refuse looks genuinely useful for people doing legitimate offensive work.

But the trust boundary moved. It used to sit at the provider's API. Now it sits on my desk, in a file I downloaded. Next I'll move a few rules into a harness, rerun the failures, and see what holds.

If you've walked through any of those doors already, I'd like to hear what you found. Write to me.

Tools, products and projects named here are mentioned for context. None of them sponsored, reviewed or approved this piece.

Explore all the writing