ml-explore/mlx-lm

A practical comparison of llama.cpp, TensorSharp, Ollama, MLX-LM, MLC LLM, vLLM, SGLang, and other open-source inference stacks

Comparing TensorSharp, llama.cpp, vLLM, SGLang, and open-source agent runtimes from a .NET perspective

7 exo Connect the devices you already own into one AI cluster and run models too big for any single machine link:

I've been experimenting with local AI (oMLX) on a Mac Studio (M2 Max 96GB) for agentic GitOps. My test bench covers repo analysis and live cluster operations through the Flux MCP server. After…

MLX官方库mlx-lm 6.6k星，M系列Mac一行命令跑Llama Qwen Mistral。实测M2 Pro加载7B约3秒，比llama.cpp快一截。首次启动要编译Metal内核等几十秒。点开推文看地址

8 bitsandbytes The 4-bit and 8-bit quantization library that makes QLoRA fine-tuning possible

Honest caveat: the layout only pays on an explicit-read path. Stock llama.cpp's mmap faults are layout-blind - I measured parity. You need the prefetcher patch (in the repo). Fully-cached models see…

Seeing model pickers move onto phones is great, but “GGUF/MLX/ONNX available” is still a weak compatibility test. Before a 4 GB download, the app should know peak load memory, KV-cache growth at the…

I got Nemotron Puzzle 75B running smoothly on a 64GB M2 Max

If you want to fine-tune a small assistant model on a MacBook Pro, don’t start with the model. Start with the correction log. The useful dataset is not “10k random instructions.” It is the tiny…