The harness started filing its own bugs

This blog has spent months arguing one thing: the interesting part of a self-hosted coding agent is the harness, not the model. Foreman is that harness, the piece of LLMKube that takes a GitHub issue, hands it to a local model on hardware you own, gates the result deterministically, and opens a pull request with no cloud inference in the loop. This week the harness crossed a line I did not expect it to cross yet. It stopped being only a thing we debug and started participating in its own debugging.

Two failures with the same shape

It started with an issue about pinning runtime images for Blackwell-class GPUs. I queued it to a Foreman coder, a dense 27B running fully local, with a carefully written scope in the Workload's intent field: change these three files, everything else is out of scope.

The run refused the work as too broad. So I narrowed the intent further and ran it again. The second run did the entire issue, chart values, docs, everything I had explicitly excluded, and then failed the gate.

Two failures, opposite directions, same issue. That shape usually means the coder never saw the constraint at all. One grep confirmed it: Workload.spec.intent, a required field, documented as "the natural-language description of what to do," was consumed by nothing. Not the prompt builder, not the controller, nothing. Every scoping instruction any operator had ever written into a Workload had been silently discarded. The coder only ever saw the raw issue body.

The harness had, in effect, filed its own bug. Both transcripts were the reproduction. We filed it for real (#1201), fixed it the same day (#1203), and deployed it to the fleet.

Zero for two becomes three for three

The validation run was deliberately the exact shape that had failed twice: a large epic issue plus a narrowing intent, this time scoping the coder to one story of the epic, three chart files, mirror a pattern that had just landed on main, touch nothing else.

Six minutes later: a scoped GO, gate passed, exactly the three files, and a commit message that restated my intent's own phrasing, wording that appears nowhere in the issue body. Proof positive the instruction channel was finally connected. That change is #1205, and yes, the agent wrote it.

Then it got recursive. Reading the earlier failed transcripts, we noticed the coder had asked its research tools for facts "as of mid-2025." It was July 2026. The model was anchoring live web research to its own training cutoff, which quietly defeats the entire point of giving it research tools. We filed that too (#1202), and this time Foreman fixed its own bug: a coder run, scoped by the intent mechanism repaired the previous morning, injected the current date into every task prompt and taught the research guidance to never name a year from memory (#1207). GO in nineteen turns.

The next research run opened its first query with "As of July 2026."

My favorite run of the week produced no code at all. We asked the coder to verify whether upstream llama.cpp's metrics endpoint had grown a request-duration histogram, and defined the honest answer as success either way. Seven turns and four minutes later it returned a NO-GO: eleven metrics on current master, all counters and gauges, zero histograms, source cited with the date. That verified absence, recorded on the issue, is what stops someone from wiring a latency SLO to a metric that does not exist. A refusal backed by evidence is a deliverable.

What the loop shipped

A self-improving harness is a nice story. It matters because of what came out the other end of the same week, and this is the part aimed at people who run GPUs for more than one team.

A 7B model uses roughly 15GB. Alone on a 192GB Blackwell-class card it strands ninety percent of the device. But sharing without governance means one team can OOM another. LLMKube 0.9.10 ships a first-class answer, a vendor-neutral gpuSharing field on every InferenceService with three tiers: exclusive for the big tensor-parallel models, partitioned for hardware-isolated MIG slices, and shared for co-located small services on time-sliced pools or unified-memory APUs.

The fairness unit underneath is device memory, not GPU count, because "one GPU" is meaningless when it can be a 24GB slice of a 192GB card. GPUQuota now derives every service's VRAM footprint from its sharing tier, a partition's size is its footprint, a shared service declares its limit, an exclusive service multiplies count by a fleet-level per-device figure, and the admission webhook denies anything that would bust the cap. Workloads whose footprint cannot be derived are denied too, when a cap is declared, with a message that tells you exactly what to set. Waving the undeclared through would let them eat the very budget the cap protects.

We validated the whole ladder live on a 128GB unified-memory AMD box, the small-scale twin of a shared datacenter card: four 24GiB shared services admitted against a 96GiB cap, filling it exactly. The fifth was refused at kubectl apply with the arithmetic in the error message. An exclusive service bounced off the fleet figure. A partition request on the APU was rejected as unsupported hardware, correctly, by spec validation rather than the quota. The denial counter read two, not three, because a spec error is not a quota denial, and the metrics agreeing with that distinction is exactly the kind of boring correctness this project is about.

Getting there also surfaced the best bug of the week, found by a test, not a user. Writing the end-to-end test for the quota webhook revealed that the webhook served on one URL path while its configuration pointed admission at another. Unit tests could never see it, they call the validation function directly. On a real cluster, enabling multi-tenancy would have failed closed on every single InferenceService. One line fixed it (#1195), and the e2e that caught it now guards the path forever.

The runtime defaults got the same treatment (#1204), every version fact verified against current sources rather than anyone's memory, mine included. vLLM moved to v0.25.1. TGI turned out to be archived upstream, so its floating latest tag was a time bomb we replaced with the final release. And llama.cpp's default image was quietly CPU-only, meaning an NVIDIA GPU Model without a hand-set image was serving on CPU; the operator now diverts those to the CUDA build automatically, with an honest documented caveat about what the prebuilt images do and do not compile for Blackwell-class silicon.

What the coder still cannot do

If this reads like the agents run the project now, let me correct that, because the failures are as instructive as the wins.

The same 27B that shipped three clean changes hit a wall twice on a bug inside our 2,400-line executor file, looping until the stuck-loop detector killed the run both times. Two identical failures is a capability boundary, not bad luck, so a human fixed that one (#1210). The rule we run is one retry, then escalate, and the harness is honest enough about its own verdicts to make that rule enforceable.

And an unharvested overnight run from four days earlier, a green, gate-passed GO, turned out on review to be based on a stale checkout that would have deleted 140 lines of a dashboard that shipped since, with a research-era fact table containing two wrong values. We verified the real mapping by executing the formatting library, took the idea, rebuilt it by hand on current main, and credited the run (#1211). A GO is where review starts, not where it ends.

The division of labor that is emerging is sharp enough to write down. Agents are excellent at scoped changes with a pattern to mirror, at research probes with honest failure defined up front, and at fixing the harness that runs them. They are bad at very large files, at anything whose ground truth lives outside the repo, and at knowing their checkout has gone stale. The gates constrain the first list. Humans own the second. Every merge this week was reviewed, and the review kept finding real things.

Where this leaves 0.9.10

LLMKube 0.9.10 is the release where the multi-tenant story stopped being a roadmap slide: tiered GPU sharing, VRAM-governed quotas with admission enforcement, Blackwell-ready runtime defaults with fleet-level overrides for air-gapped registries, platform floors surfaced at install time, and an operations guide whose day-zero recipes we executed ourselves before publishing. Enterprise-grade is a claim you earn per-feature, with a validation trail, and that trail is in the repo: the readiness matrix, the live quota denial with its byte math, the e2e that caught the webhook bug. What still waits for silicon we do not have on the bench, live MIG partitioning chief among it, is marked as exactly that. Even the version number is the thesis working: the release pipeline's image scanner flagged a vulnerable dependency in 0.9.9's controller image, so 0.9.9 never became the recommended release and 0.9.10 is the roll-forward with the fix. Gates that only ever say yes are decoration.

And Foreman is the reason the week fit in a week. The harness found two of its own bugs, fixed one of them itself, shipped three of the features above, and told us the truth every time it could not. That last part is the whole design. I would rather have an agent fleet that is right about its own limits than one that is impressive about its claims.

The loop is closed. Now we get to see how fast it spins.

LLMKube is an open-source Kubernetes operator for running local LLM inference and self-hosted coding agents across NVIDIA, Apple Silicon, and AMD hardware. Star it, break it, contribute: github.com/defilantech/LLMKube.