The best local AI experience. Powered by TokenWing.
Like runmyai/run-my-ai, 0 likes 0 Bookmark runmyai/run-my-ai, 0 bookmarks 0 Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
Like wici-ai/inferssd, 3 likes 3 Bookmark wici-ai/inferssd, 3 bookmarks 3 Run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources by exposing them on different ports and loading/unloading them on demand
Like perk11/large-model-proxy, 0 likes 0 Bookmark perk11/large-model-proxy, 0 bookmarks 0 Run the full 2.78-trillion-parameter Kimi K3 model, DeepSeek V4.1 Flash or GLM-5.3-Flash beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine.
Like sqliteai/warp, 0 likes 0 Bookmark sqliteai/warp, 0 bookmarks 0 High-speed Large Language Model Serving for Local Deployment
Like Tiiny-AI/PowerInfer, 0 likes 0 Bookmark Tiiny-AI/PowerInfer, 0 bookmarks 0 Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory
Like jitllm/jitllm, 0 likes 0 Bookmark jitllm/jitllm, 0 bookmarks 0 Run GreenBitAI's Quantized LLMs on Apple Devices with MLX
Like GreenBitAI/gbx-lm, 0 likes 0 Bookmark GreenBitAI/gbx-lm, 0 bookmarks 0 MoE expert offload for low-VRAM GPUs — run 100B+ MoE models (DeepSeek, Qwen, Mixtral) on 8 GB cards. Expert cache with LRU hot experts, OpenAI-compatible proxy, GGUF multi-shard.
Like yalun753/moe-l2, 0 likes 0 Bookmark yalun753/moe-l2, 0 bookmarks 0 A custom inference engine tuned for Qwen4 architecture. Currently runs Qwen3.8-Flash-Next at 15+ tok/s on 16GB of VRAM and 30 GB of RAM
Like Apolog1ze-Dev/QwFNfer, 0 likes 0 Bookmark Apolog1ze-Dev/QwFNfer, 0 bookmarks 0