Insights, tutorials, and updates from the LLMKube team
Containers on macOS can't access Metal GPUs. Instead of fighting that limitation, we inverted the architecture: Kubernetes orchestrates, Metal runs natively. Here's how the LLMKube Metal Agent works.
I needed SVG illustrations for a side project and didn't want to pay per-image. So I chained a local LLM, Flux, and vtracer on my homelab to build Vecsmith, a self-hosted text-to-SVG pipeline. Here's how it works and how you can build your own.
LLMKube v0.4.9 ships with comprehensive CLI benchmarking: five predefined test suites, automated sweeps, and markdown reports. We ran it on ShadowStack and discovered when multi-GPU helps (and when it doesn't).
We pushed ShadowStack to its limits with 32B parameter models. Here's how dual RTX 5060 Ti GPUs handled Qwen 2.5, Qwen Coder, and Qwen 3 at production scale with zero failures.
Ollama is great for local development, but it wasn't designed for production. We analyzed 200 GitHub issues to understand why multi-GPU setups fail and what alternatives exist.
On this Thanksgiving eve, we reflect on the journey of building LLMKube in the open, share our latest benchmark results (68.7 tok/s on Llama 3.2 3B!), and express gratitude to the community.
LLMKube v0.4.0 brings multi-GPU support with layer-based sharding. We tested it on ShadowStack and hit 44 tok/s on Llama 13B across dual RTX 5060 Ti GPUs.
Behind the scenes of building our bare-metal testing environment for air-gapped LLM deployments. Real hardware, real constraints, real testing.
Learn why we built LLMKube and how it brings production-grade orchestration to local AI workloads.