omgitsbase/llmash

Corrallama: desktop app to run and manage llama.cpp llama-server instances - launch profiles, llama.cpp build updates, Hugging Face downloads, benchmarks and automatic parameter tuning. Windows, Linux, macOS.

Windows-first OpenAI-compatible local LLM server powered by OpenVINO GenAI for Intel CPU/GPU/NPU, with chat UI, model conversion, and setup scripts.

llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 110+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx

Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory

High-speed Large Language Model Serving for Local Deployment

Strata minus the bullshit. clone, rsync to prod, compile and go

Open-source control panel for the real llama.cpp: see where a model lands in your VRAM before loading, every llama-server flag per model, one-click llama.cpp updates, several models at once, and Embers (local agents whose quotes are checked by code). Windows · Linux · macOS.

Delta Inference Engine: DeepSeek-V4.1-Flash on one NVIDIA GPU, natively on Windows (OpenAI-compatible server)

Turn any Mac or GPU into an OpenAI-compatible inference node. One-command setup, automatic HTTPS, model management, and distributed request routing.