The best part of LLMKube 0.9.0 is code I did not write

For the last two months this blog has been a running argument that the interesting part of a self-hosted coding agent is not the model, it is the harness around it. Foreman is that harness: the piece of LLMKube that takes a GitHub issue, hands it to a local model on hardware you own, gates the result, reviews it, and opens a pull request, all inside a Kubernetes cluster with no cloud inference in the loop.

0.9.0 is the release where the harness stopped being something I build alone.

The two changes I am proudest of in this release were written by Jory, who I have never met in person, working from his own read of the code. Both of them fix things I had not noticed were broken. Both of them make the fleet more honest about its own state. And the way they got written is, honestly, the thing I most want more of.

What actually shipped

Foreman schedules coding tasks onto a fleet of FleetNodes, one task per node. The intent has always been simple: if you have three agent nodes, three issues should run in parallel.

They did not.

Jory found that FleetNode.Status.CurrentTask, the field the scheduler reads to decide whether a node is busy, was declared, read by the skip-guard, and logged, but never actually written anywhere. The guard read a field nobody set, so it never tripped, and every task funneled onto the alphabetically-first Ready node. On a three-replica fleet you would see one node grinding through 25 queued tasks, 50 more Pending, and two nodes sitting completely idle. Effective concurrency was one, no matter how many machines you added.

His fix (#978) does not just write the field. It reserves the node with an optimistic-lock status patch, so two scheduler passes racing on the same cached view cannot double-book it, and it self-heals a reservation that leaks when a task is deleted mid-flight. That is the difference between "make the test pass" and "understand the concurrency model." He wrote the second one.

The other fix (#980) is the kind of bug you only find by running the thing for real. When an agent shuts down it marks its node Draining, on purpose, so the scheduler stops routing to it. The reconciler deliberately leaves Draining nodes alone, because that is the agent's business. But FleetNodes have no owner reference, so nothing garbage-collects them, and if the pod dies before it deletes its own node, that node stays Draining forever with a frozen heartbeat. Jory counted 23 of these orphans piled up against 3 live nodes on a running cluster. His change reaps a Draining node once its heartbeat has been silent well past the drain window, and, after a review round, does it with delete preconditions so a node that comes back to life in the same instant is never reaped out from under a live agent.

Neither of these is glamorous. Both of them are exactly the unglamorous reliability work that decides whether a fleet is a demo or a system you can leave running overnight.

The road to 0.9.0

It is worth saying what the scheduler is now scheduling for, because 0.9.0 is the visible tip of two months of Foreman work. Across the 0.8 line, Foreman grew from a coder that opens pull requests into a control plane you can actually trust with your own repository. The pieces that got it there:

• Verification that does not take the model's word for it. A two-tier gate suite, mutation checks that confirm the tests actually test the code (delete the implementation and the test has to fail), a reference-grounding check so the coder cannot cite an API that does not exist, scope guards that catch it wandering into the wrong subsystem, and a post-push envtest gate that closes the timing hole where a diff passes inside the loop and still fails on main.
• Beyond Go. A GateProfile CRD with language presets, so the same harness gates Python, TypeScript, or whatever you configure, running your project's real build and test commands in an ephemeral gate Job.
• Per-model tuning. A ModelProfile CRD, because a 27B model and a 35B mixture-of-experts want different context windows, retry budgets, and stuck-loop thresholds, and the harness should carry those knobs per model, not per cluster.
• Reviewers that finish the job. The reviewer now opens the pull request on a GO, writes the PR body from its own summary, and on a NO-GO feeds bounded fix iterations back to the coder instead of failing the task outright.
• Resilience. Coder-tier escalation re-dispatches a failed task to a larger model, revise-from-branch restarts cleanly from a previous attempt, str_replace recovers from near-miss edits instead of giving up, and a rejected push replaces the stale agent branch with force-with-lease rather than wedging.
• Fleet and tooling. llmkube foreman dispatch fans a batch of issues across your coder agents, a durable audit log records every run for llmkube audit export, FleetNode labels survive pod restarts, and the accelerator routing enum learned Vulkan so an AMD box can take coder work alongside NVIDIA and Apple Silicon.

That is a lot of machinery. Jory's two fixes are what let all of it run across a fleet of machines instead of piling onto one.

Why the partnership is the story

I review contributions the same way Foreman's own harness reviews its local models: adversarially, with a rubric, assuming the change is wrong until the evidence says otherwise. Jory's PRs went through that, and the review found real things, a test that could pass without actually proving anything, a delete that could race a revival.

What made it a partnership instead of a gate is what happened next. He did not argue the findings or paper over them. He came back with the delete preconditions implemented correctly, including the subtle part, requeue on conflict rather than erroring, so a revived node reads healthy and is spared while a genuinely dead one still gets cleaned up. He added the tests. He wrote comments explaining the race for whoever touches it next. The turnaround was hours, and the second version was better than anything the review asked for.

That is rare, and I do not want to be quiet about it. A project gets called "great" for its features. It becomes great because of the people who show up to make the boring parts correct. Jory made LLMKube's fleet scheduler correct this week, and he did it as a collaborator, not a drive-by.

And it runs both ways, which is the part I most want to be precise about. Jory is not a contributor I gate from above. He is the Foreman co-maintainer. Our CODEOWNERS file routes every change to the Foreman subsystem to him, mine included, so when I shipped coder-tier escalation earlier in this cycle it went through two rounds of his review before it merged. The adversarial review I run on his pull requests is the exact one he runs on mine. That is what I mean when I call this a great project: not a maintainer who reviews contributions, but two people who each have to clear the other's bar before anything lands in the subsystem we both own. Mutual accountability is worth more than either of us being right on a given day, because it does not depend on either of us being right on a given day.

An open invitation

LLMKube is Apache-2.0 and it stays that way, forever. Nothing about the core is paywalled, and the whole point of building it in the open has always been that the interesting problems are too big for one person's nights and weekends.

If you have wanted to dig into something real, here are areas that are wide open and that someone could genuinely own:

• Heterogeneous hardware. LLMKube runs the same fleet across NVIDIA CUDA, Apple Silicon Metal, and now AMD Strix Halo over Vulkan. Every new accelerator tier is a frontier. If you have unusual silicon, there is work here.
• The gates and reviewers. Foreman's trust comes from deterministic checks and a reviewer rubric, not from a smarter model. Making those checks sharper, or catching a class of bad change they miss today, is high-leverage.
• Model diversity. We keep finding that different local models fail in different ways, and that a diverse review panel beats any single model. Bringing a new local model into the harness and reporting how it behaves is a real contribution.
• The unglamorous reliability work. As Jory just showed, the scheduler, the lifecycle, and the self-update paths are where a fleet earns your trust. There is more of it.
• Docs and onboarding. If you tried to stand LLMKube up and hit a wall, that wall is a contribution waiting to happen.

Come find the areas you are passionate about. Open an issue, start a discussion, send a rough PR. I review everything, I review it seriously, and, as this release shows, the best of it ships.

Thank you, Jory. On to the next one.

LLMKube is an open-source Kubernetes operator for running local LLM inference and self-hosted coding agents across NVIDIA, Apple Silicon, and AMD hardware. Star it, break it, contribute: github.com/defilantech/LLMKube.