Local LLM

On-device multiple-choice, graded, and yes/no answers with a probability for each option. A Swift package for Core AI, inspired by TypeSafe.

Tiel-Coder 35B-A3B on Intel Arc: GPTQ W4A16 + MTP recipe, quantization scripts, benchmarks and measured results.

vLLM extensions for Intel Arc / XPU: int4 MoE + MTP speculative decoding, GGUF SYCL kernels, and build recipes measured on an Arc Pro B70.

llama.cpp with Mirai S (2.4-bit Qwen3.8-27B) support: 128K context on a 12 GB GPU

How to set up a minisforum N5 max-building complete infrastructure

Web UI for the Ollaya API

Local AI assistant for the Mac terminal (MacBook Air, MacBook Pro, Mac mini, iMac, Mac Studio on Apple Silicon). Chats, reads PDF/Word/Excel, writes files – offline and private via LM Studio and Gemma 4.

Strix Halo inference engine. Qwen Flash Next Q4_K_XL: 1,628.52pp, 59.41tg single user, 157.22 tok/s 8 users; Qwen27B Q4_K_XL: 656.33pp, 70.56tg tok/s single user with DFlash2

A version of microgpt that follows modern transformer approaches in Qwen 3

Open-source, local-first AI team workspace. Several AI models work together in one conversation, each with its own provider, model and tool whitelist. Android and VS Code clients pair in. No telemetry.

Ein WebOS für lokale KI · A WebOS for local AI

A local decision layer for agents and workflows. You hand it a STATE (text or JSON) and a SCHEMA of named questions, each with a fixed list of allowed answers.

Private AI English coach for Mac. Just speak or type: Plumb catches your grammar and spelling mistakes, shows you why, and charts how you improve. Free, on-device, open source.

Bonsai 2 27B on a GTX 1080 Ti: Pascal (sm_61) patch for PrismML's llama.cpp fork + native MTP drafter support. 86 tok/s on code.

Experimental Windows WSL2 integration for Halogen Flash on AMD Strix Halo, with guarded launchers and reproducible benchmarks

llama.cpp + llama-swap benchmarks on AMD Strix Halo (Ryzen AI Max+ 395, 128 GB) under Windows: backends, flags, spec decoding, concurrency, and the display-off throttle

A controlled multi-step AI agent built from scratch with Python, Ollama, tool calling, guardrails, and human-in-the-loop approval.

Pretty local mlx chat interface

LLM inference in C/C++

Self-hosted control plane that fits local AI to your hardware and real workload: it measures, tests changes in idle windows, verifies them on live traffic and rolls back regressions. llama.cpp, image, audio and ONNX/NPU engines, GPU tuning, multi-user roles, an OpenAI-compatible gateway with quotas, fleet serving and MCP. Stdlib Python, no Docker.