LLMs are kinda dumb and basic, even with all their training. What really makes the LLM something useful is an agent which enables tool usage and makes it so much more useful. I had Claude write me an agent, which it’s been doing a very good job at as I’ve been iterating new features. Before long my own agent and LLM should be able to handle updating itself. Actually, it probably can do that now. The LLM I’m using is qwen3-coder-next:q8_0. It’s producing around 4 tokens/sec on CPU inference only so it’s not blasting any doors off things, but it does work. If I’m not in a hurry, it’s fine. It’s still faster than I would be doing all this myself. This stuff is crazy and cool. #ai #llm #qwen #qwen3

What locally hosted open-source LLMs are people using for #coding and having good success with? I’ve got a MacBook Pro M4 Pro with 48 GB RAM and 2 TB drive space. Loading qwen3-coder-next:q4_K_M on it results in the MacBook going about 25 GB into swap. Surprisingly the machine is still usable, although it does show signs of stress. Through my employer, I have access to (read as I run this) a Proxmox cluster with 5 members. 3 members each have 768 GB RAM on them. Each also has either 36 or 48 hyper threaded cores spread across 2 NUMA domains. I’ve loaded up qwen3-coder-next:q8_0 on it because the model takes about 85 GB RAM. I strapped Ollama to a single NUMA domain. The cluster members don’t have any fancy GPUs on them so I’m left with CPU operation only to run this model. As you might expect, it’s not the fastest thing around, but it does run. Anyway, I’m just exploring and trying to see how useful these open source models are for mainly for coding, but other tasks as well, relative to Claude or Gemini Pro, both of which I have paid access to. What have you had reasonable success with coding? #ai #llm

GN all. Been a busy day. For some reason I decided that I didn’t have enough to do so I pulled out the anode rod from my water heater to check its state. Put it back in and had a very slow leak that presented itself over an hour. Pulled it out again, applied thread tape, and put it back in. Seems solved now. Hope so. I’m going to sleep and it’d be really great to not wake up to a wet basement.

If you’re interested in trains, there was a derailment in Toledo, Oregon a couple weeks ago on a train bridge over a river, and someone came out to do a nice narration and shoot some great footage of the derailment. This is only the first video in the series.. there are two others. Then it turns out the same rail line had another derailment a week and a half or so later, but in a different spot. There’s another video of some great footage of the derailment and rerailing of the engines and cars. You’ll have to find the follow-on videos, but they should all be linked to the first one here. #trains #train #railroad