So, I've seen some more talk about the "vulnpocalypse" bubbling up again this month:
Some of it was in the context of the new GLM 5.3 AI model, which is apparently really good at finding vulnerabilities and doesn't protest too much when asked about it:
I thought about what I could possibly try to do about it, other than installing security patches and updating dependencies regularly, and I figured I'd run security reviews of all my projects on the latest model in Codex.
Well, "computer says no":
(notice the ironic prompt in the text box below…)
So I figured it's time to try talking to that friendly GLM guy. But here's a problem: I have no idea how.
ELI5 how this works
See, I've only used the Codex app with various versions of GPT before. Believe it or not, I haven't even tried Claude, any kind of it. Or anything in the terminal. I've been doing more AI coding work this year, but I'm not even close to the bleeding edge, I'm a pretty slow adopter.
I asked on Bluesky about how I can run these open weight models and I got a lot of replies, but most didn't tell me much, so I had to talk it all over with GPT to help me really organize this info in my head and understand how this all fits together.
So if you're a noob like me, here's the "big picture" (feel free to correct me in the comments!):
First, there are the models themselves; the big blobs of… something. Data. I don't know much more about it. Some, like the OpenAI and Anthropic ones are closely guarded secrets, but there are also many "open weights" ones, meaning those where anyone can download all the model data and run it directly themselves. There are several families / producers of the open weights models – as I understand it, most of those are Chinese, and the rest are mostly US-based.
You can try to run the open models yourself, locally, on your own computer, if you have good hardware. But the problem is that for most of the top of the line models, those that are not far behind GPT & Claude in terms of performance, you need a machine with way more RAM than even my big Mac Studio with M4 Max and 64 GB of memory. This is why RAM costs more than gold now and why most desktop Macs in larger configurations were wiped out from the market… I could play with some smaller models that are generally good enough for coding too, but I wanted to try the GLM specifically.
So, in practice, various companies ("inference providers") host these models in their dat*****ers and let you use them through some kind of API for a fee, and there are apparently quite a lot of these.
One example of such service, where you can buy a friendly OpenAI-like monthly subscription and use any models they have available, is OpenCode Go from a US company called Anomaly. For GLM specifically there's also the GLM Coding Plan from Chinese company Z.ai which makes the GLM model, but they only seem to have their own models available (OpenCode Go has many).
Additionally, there are also "router" services available, for example OpenRouter or Cortecs; these expose a similar kind of API, but behind the scenes they route the request to one of many actual providers, and let you configure what selection of those providers you want to use. The routers generally don't use subscriptions, but rather "pay as you go" pricing, i.e. you "top up" your balance and then what you've deposited is used up dynamically in real time as you use the API.
But how do you use the model and the provider's API, how do you interact with it? Well, you need a "harness" (no, not like this one (SFW)). Harness is an app like the Codex app, which takes your request, the code you're working on, sends it to the provider which passes it to the model, then gets a response and decides what exactly to do with it (e.g. if it should run the commands the model gave it or maybe not). A popular harness tool is OpenCode from the same OpenCode, there's also e.g. Pi, Polytoken, or ZCode from Z.ai, and many others. They have different kinds of UIs and so on.
So in practice one could do:
app → provider → model
or: app → router → provider → model
or: app → local model on a $$$$ computer
And you can mix & match those to some degree, e.g. you can use the OpenCode app (harness) with Z.ai subscription and their models, or ZCode app with OpenCode Go with a model from another company.
From the above options, I've decided to try the OpenCode app, in combination with the router service Cortecs, because it's based on EU 🇪🇺 (Austria) and lets you use European providers, and I like to "buy local" if possible, which is especially hard in the digital world.
VM for protection
Ok, so problem 2: I generally trust OpenAI/Codex not to run away with my passwords or rm -rf half of my disk. It's quite possible that I shouldn't 🫠 but again, I'm a noob here and I don't know what the possible other options are. And not all options are even guaranteed to help at all…
So I figured I'd just run the whole thing inside a good old Linux VM to feel safer.
I haven't really used VMs a lot lately since the switch to Apple Silicon happened and VirtualBox got left behind (though I think it might now support ARM? but it was a pretty bad UX for a long time, and there are now much more alternatives based on Mac's Virtualization framework). So I looked around a bit for what's available on that front:
There's e.g. UTM which I've used before, which is not like super native but generally good enough. It seems you need to download a Linux distro .iso to give it to install, generally a bit more manual.
There's VirtualBuddy, from Gui Rambo, well known in the iOSDev circles. It has a very "proper Mac app" UI and it's a bit more automated, it downloads the selected Linux installer for you. Possibly a bit less configurable.
There's something called Tart from OpenAI.
And I found this CLI-focused open source tool called Lima.
I was also recommended Ubuntu Multipass and free VMWare Fusion.
Since I just need a headless VM running in the background, not a full desktop, and I don't need an extra window open (which I think I can't avoid with the first two), I decided to try Lima.
I installed it through MacPorts, which I use instead of Homebrew because of… reasons:
sudo port install -b limaThen, it took me most of the evening to figure out the specific workflow that works exactly as I want. I ended up with something like this:
I have a modified install template in .lima/my-ubuntu.yml:
base:
- template:_images/ubuntu-24.04
containerd:
system: false
user: false
portForwards:
- guestPort: 1800
- guestIP: "127.0.0.1"
proto: any
guestPortRange: [1, 65535]
ignore: trueThis means:
use the base Ubuntu 24.04 LTS image
turn off the "containerd" thingy (something for… containers)
turn off automatic port forwarding for services running on the VM to the host machine
except for port 1800, which we'll use in a moment
Basically I wanted to turn off most things that are enabled in the default setup, for safety. I tried the --plain mode at first, which is described as more or less that in the docs, except it took me some time to realize that this also turns off file sharing completely. I want to turn off default shares, but still share a selected project directory.
With that config, I do:
limactl create --name=silo18 ~/.lima/my-ubuntu.yml
...
limactl start silo18
limactl shell silo18(Yes, I've just watched the S3 of "Silo" this week, how did you know? ☢️)
This gets me inside a freshly created VM, however, without any file sharing mounts. To share a project, I start it instead like this:
limactl start silo18 --mount-only ~/Projects/skyfall:w \
--set '.mounts[0].mountPoint="/workspace"'This is a bit awkward, alternatively I can do limactl edit silo18 and put this in the config:
mounts:
- location: /Users/kuba/Projects/skyfall
mountPoint: /workspace
writable: trueThe point of the /workspace override is that Lima kinda stupidly defaults to mounting a host folder under the exact same path inside guest, i.e. it will try to create a /Users/kuba/Projects/… in a Linux filesystem and mount it there, which just feels wrong.
I'm sharing a single project folder here to minimize the part of the disk that's made available to the agent/model; if I want another project, I need to stop the VM and restart it with another mount. If I work with this more frequently, I'll probably change this setup.
Starting the agent
Inside the VM, I installed Ruby from asdf, and then OpenCode with the curl-bash.
OpenCode can be used in several different UIs, as you can see in the docs: there's a terminal UI (like Vim etc.), there's a plain CLI that just prints to stdout, you can use it in some IDE/editor, and there's also a web UI. I played a bit with the CLI and the TUI, but it didn't feel too comfortable to me, and I ended up with the web UI, started like this (remember the port?):
opencode web --port 1800Note, this tries & fails to run a xdg-open helper, which I don't have in the server vm, so to shut it up I've just put an empty script in /usr/local/bin/xdg-open:
#!/bin/sh
exit 0Once the server's started, I can open the UI in Safari on http://127.0.0.1:1800 on the host Mac, and I get something like this:
Putting it all together
I signed up on the Cortecs website, generated an API key there, added my payment card and billing info, and added €10 to the balance for now. In the "Inference" config, I selected to optimize for cost vs. speed, and enabled the Sovereign Cloud (EU) and Zero Data Retention options for the providers choice (which basically just disables Microsoft, Google and Amazon on the list 🙃).
In the OpenCode TUI, I configured it to use Cortecs and entered the API key (you can do this through the CLI and in the web UI's settings too). Now, in the web UI, I can create a new session in a new tab, select a model from many available, like the GLM-5.3 I wanted, and give it some work to do in the mounted project folder:
I gave it one project and a few dependencies to review, and it came back with something, though I didn't have a chance to read this in detail yet, so I can't compare the quality of the model. This was really more of an exercise in creating a setup where I can try models other than GPT in the future at all. And yes, GLM-5.3 happily did the hacky hacky on some codebases when asked for a security review and didn't ask me if I have a permit, unlike Sol & Astra did (with the exact same copy-pasted prompt).
The Cortecs account balance shows that it used up €0.90 so far in that test. It's hard to do the math on this sample, but I have an impression that this is generally much more than with an OpenAI subscription. Possibly like 5x that, if I were using the OpenAI Plus subscription to 100% of the weekly usage, and tried to do the same amount of work here.
But also, I don't use the OpenAI subscription 100%, not even close – I think I've run out of 5-hour usage two times so far, never run out of weekly usage. And I pay the €20/mo subscription every month whether I use 0% or 100% of it. If I tried to use the same amount of tokens as I actually use in Codex with Cortecs, it's possible that it would be roughly comparable.
But anyway, this isn't something I intend to use daily right now. I just wanted to have any access to non-GPT models to sometimes get a second doctor's opinion on something.
Of course, you could achieve the same thing much simpler by just skipping the "VM for safety" part – you can run any of the OpenCode UIs just directly on your desktop/laptop, or some integration in an editor/IDE like e.g. Zed, then you just need to install & run one tool and sign up for one account and log in to it in the tool. But I sometimes like to make my life more complicated…
Postscriptum: Running models locally
After I finished writing this blog post, I looked at the thread with suggestions I linked earlier once again, and I decided to try some other lighter model in a fully local setup. GPT suggested Qwen3.6-27B or newer Qwen3.8-27B in some variant, which both should in theory fit in my Mac's 64 GB of RAM.
So I did some little research again on what are the options for running models locally:
there are also two formats (?) of models, "GGUF" used by the llama family of tools, and "MLX" which is optimized specifically for Mac – MLX is Apple's framework for machine learning
I decided to use one of the Mac-specific tools, since they're probably better optimized for the OS, and of the two I listed, I chose oMLX, because it apparently does some smarter caching to speed some things up. I installed it from the .dmg, launched & configured, and I looked up Qwen3.8-27B in the list of models in the settings and downloaded it (in the smaller "4-bit" version).
And then, I very easily managed to connect to the server started by oMLX on the Mac Studio from the exact same OpenCode web app running in the VM on the MacBook 🥳 I had to enter the URL with IP:port and the path as shown in the server settings, the API key configured there, and the full name of the model:
Now, when I start a new session, I can choose any selected model available on Cortecs or on the local Mac server:
I gave the Qwen model the same security review task to do. I think this was the first time I've ever heard the Mac Studio's fans turn on, and I've had this machine for a bit over a year now 😅
After that, the results were rather disappointing so far. (Note: this section might be completely wrong, it's just what I've managed to find out so far.)
First, it got stuck because I was working on the MacBook and the Mac Studio (running the model) went to sleep, and OpenCode lost connection. I had to run caffeinate on the Studio to keep it awake.
The task ran for I think a few hours until it was my turn to go to sleep. It run one or two sub-agents, giving them some pieces of work, but eventually both the sub-agents and the main session got stuck again. I started getting errors like:
oMLX prefill memory guard rejected this prompt: Prefill context too large for available memory (pre-chunk guard at 512 tokens, kv_len=45568): predicted peak would exceed prefill safety cap 43.2GB (90% of metal_cap ceiling 48.0GB). Raise kernel iogpu.wired_limit_mb in Terminal (currently caps Metal at 48.00 GB), or reduce context length. To continue, set Memory Guard to aggressive, raise the custom memory guard ceiling, free system memory, or compact/reduce context.
Eventually I figured out that there is a global setting for "Context Window" in oMLX settings (in the "Server" section), set to 32k tokens, but it's only applied if the given model doesn't have its own default:
Fallback values used when a model has no profile, or when a profile leaves a field empty.
To override it, I had to go to Models > Qwen3.8-27B and set the Context Window separately there. Then, I got an error like this:
Prompt too long: 189211 tokens exceeds max context window of 52000 tokens
Turns out, you have to also set it separately in the OpenCode config, as limit.context and limit.output under this specific model, to something slightly smaller than the limit on the server. Then, OpenCode will start "compacting" the session periodically when it gets too large, like Codex or Claude do.
But then… it seemed that the constrained context window is too small, and even with a smaller task like "write specs for these few files", I gave up after an hour and 4 separate compactions. The compaction means that it needs to summarize everything it found so far and basically pass a note to a future copy of itself with what it's supposed to do, what it did and what is left to do. Generating this summary note also takes a while, and it already fills up some of the context window, so the periods between compactions got progressively smaller, leaving it less time to do any progress…
I eventually got it to complete this smaller task (specs) by bumping up this custom context window setting to a larger value (80-100k). But still, even without the context problems, the local model on my Mac works noticeably slower than the hosted models: Sol/Codex wrote the tests in 3 minutes, GLM-5.3 in 8 minutes, hosted Qwen 3.8 in 13 minutes, and the local Qwen took about half an hour. But, well, at least it didn't cost me 0.90€ unlike the previous test. Just some unknown amount of electricity :]
My initial impressions are that for this to be usable for serious normal programming work, you probably need at least a bit larger machine than mine in terms of RAM, and you need to spend some time and effort learning about the different models, harnesses, their strengths and weaknesses, how exactly to tune them with the parameters in the settings, and so on. It's possible that some model in some configuration would work well enough on my Mac, but it's not something that you just install & run and get results.
Basically: if using Codex is like installing macOS, then using open-weight models through an inference provider or routing service is like installing Windows, and using local models is like installing Arch Linux (or like tuning a Postgres database). You might get very good results, but be prepared to have to learn & experiment with things first.
Of course, this is the state as of early September 2026. Things could look very different in November or December.