Insights, tutorials, and updates from the LLMKube team
LLMKube v0.4.9 ships with comprehensive CLI benchmarking: five predefined test suites, automated sweeps, and markdown reports. We ran it on ShadowStack and discovered when multi-GPU helps (and when it doesn't).
We pushed ShadowStack to its limits with 32B parameter models. Here's how dual RTX 5060 Ti GPUs handled Qwen 2.5, Qwen Coder, and Qwen 3 at production scale with zero failures.
Ollama is great for local development, but it wasn't designed for production. We analyzed 200 GitHub issues to understand why multi-GPU setups fail and what alternatives exist.
On this Thanksgiving eve, we reflect on the journey of building LLMKube in the open, share our latest benchmark results (68.7 tok/s on Llama 3.2 3B!), and express gratitude to the community.
LLMKube v0.4.0 brings multi-GPU support with layer-based sharding. We tested it on ShadowStack and hit 44 tok/s on Llama 13B across dual RTX 5060 Ti GPUs.
Behind the scenes of building our bare-metal testing environment for air-gapped LLM deployments. Real hardware, real constraints, real testing.
Learn why we built LLMKube and how it brings production-grade orchestration to local AI workloads.