Very proud to share that we just release Luce KVFlash. Run your preferred model inside Lucebox at 256k context, without thinking about KVCache and OOM, up to 2.9x faster decoding at long context.…

The cuda12 image is a fat binary: nvcc emits device code for every architecture in the list, and the right kernels get picked at runtime. Nothing to detect, nothing to rebuild if you swap GPUs.

A 26B model on a 24 GB laptop tied a 284B model on a 192 GB Mac Studio. Both 78.3% on ds4-eval-92, the eval ported from @antirez's ds4 (huge fans of all his work on it). To be honest DeepSeek V4…

https://t.co/9NRKqov7Ur

Scrapped 500+ issues and PRs to ship a massive @luceboxai repo redesign and fixes. Very proud of the team. The fastest inference server isn't going to come from a datacenter, it's going to run on the…

2.5x faster than llama.cpp on Strix Halo. We just shipped DFlash + PFlash for the AMD Ryzen AI MAX+ 395 iGPU (gfx1151, 128 GiB unified memory). Qwen3.6-27B Q4_K_M, end-to-end on the same silicon: ▸…

https://t.co/qdj5Re0SmU

Lucebox: hand-tuned LLM inference for consumer hardware — rewritten kernel by kernel. - Megakernel: Qwen 3.5-0.8B at 1.87 tok/J on a 2020 GPU, matching Apple M5 - DFlash 27B: Qwen 3.5-27B at 130…