Footer

Your go-to destination for discovering trending open-source projects and uncovering the insights that matter

Legal

  • Terms and conditions
  • Privacy policy

More

  • Pricing
  • Advertise
  • onedump
  • gosnakego
  • Contact us

Social

  • X (Twitter)
  • GitHub
Trendshift owl
What they build speaks louder than words
© 2026 Trendshift, created by Julian Li with ❤️
SignalAdvertise
Trendshift
SignalAdvertise
Sign in
Trendshift owl

Discover trends that matter

SignalAdvertise
DailyWeeklyMonthlyYearly
Live mentionsTopics
RepositoriesDevelopers
InsightsStats

Omnigraph
Omnigraph
Busbar
Busbar
Websitebench
Websitebench
Bifrost
Bifrost

kvcache-ai/ktransformers

A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

Visit GitHub
Python
19.7k
1.7k
112 contributors
Apache License 2.0
website
AchievementsView all →
FeaturedAdvertise to 180K+ monthly visitors
Omnigraph

Versioned graph database on object storage. Git-style workflows and multimodal retrieval

Busbar

Control what AI can do before it acts

Websitebench

Can AI agents reverse-engineer real websites?

Bifrost

Fastest Enterprise AI Gateway for Routing, Governing, and Securing all your AI Traffic

Momentum

How fast kvcache-ai/ktransformers is gaining stars, and where it lands on Trendshift and GitHub Trending

Activity and trending

Stars, forks and merged pull requests each day or month, lined up with the repository's rank on Trendshift Trending

Sep 17 – Nov 16, 2026

Stars +177 in 61 days

051015
13

Forks +54 in 61 days

0246
5

Merged PRs 22 in 61 days

024
4

Trendshift Trendingwhere the repository ranked each day it made the top 25

Not in the Trendshift top 25 in this period

Sep 17Sep 25Oct 3Oct 11Oct 19Oct 27Nov 4Nov 12
The left and right arrow keys step from day to day; Home and End go to the first and the last

Stars, forks, merged pull requests and Trendshift Trending rank, by day
DayStars gainedForks gainedMerged PRs
Thu, Sep 17, 2026+3+12
Fri, Sep 18, 2026+1+30
Sat, Sep 19, 2026+0+01
Sun, Sep 20, 2026+5+10
Mon, Sep 21, 2026+5+00
Tue, Sep 22, 2026+0+00
Wed, Sep 23, 2026+4+01
Thu, Sep 24, 2026+2+10
Fri, Sep 25, 2026+2+20
Sat, Sep 26, 2026+0+00
Sun, Sep 27, 2026+4+30
Mon, Sep 28, 2026+5+00
Tue, Sep 29, 2026+0+10
Wed, Sep 30, 2026+3+21
Thu, Oct 1, 2026+5+30
Fri, Oct 2, 2026+3+00
Sat, Oct 3, 2026+3+00
Sun, Oct 4, 2026+1+50
Mon, Oct 5, 2026+5+00
Tue, Oct 6, 2026+3+10
Wed, Oct 7, 2026+1+20
Thu, Oct 8, 2026+1+11
Fri, Oct 9, 2026+7+20
Sat, Oct 10, 2026+13+54
Sun, Oct 11, 2026+1+20
Mon, Oct 12, 2026+8+14
Tue, Oct 13, 2026+3+00
Wed, Oct 14, 2026+5+10
Thu, Oct 15, 2026+0+01
Fri, Oct 16, 2026+8+00
Sat, Oct 17, 2026+2+00
Sun, Oct 18, 2026+1+10
Mon, Oct 19, 2026+4+11
Tue, Oct 20, 2026+0+00
Wed, Oct 21, 2026+5+10
Thu, Oct 22, 2026+0+00
Fri, Oct 23, 2026+3+00
Sat, Oct 24, 2026+5+00
Sun, Oct 25, 2026+3+00
Mon, Oct 26, 2026+5+00
Tue, Oct 27, 2026+4+00
Wed, Oct 28, 2026+7+12
Thu, Oct 29, 2026+3+00
Fri, Oct 30, 2026+0+00
Sat, Oct 31, 2026+0+20
Sun, Nov 1, 2026+0+10
Mon, Nov 2, 2026+8+10
Tue, Nov 3, 2026+4+10
Wed, Nov 4, 2026+2+10
Thu, Nov 5, 2026+0+10
Fri, Nov 6, 2026+0+11
Sat, Nov 7, 2026+0+00
Sun, Nov 8, 2026+0+10
Mon, Nov 9, 2026+0+10
Tue, Nov 10, 2026+2+00
Wed, Nov 11, 2026+1+00
Thu, Nov 12, 2026+0+11
Fri, Nov 13, 2026+7+12
Sat, Nov 14, 2026+3+10
Sun, Nov 15, 2026+4+00
Mon, Nov 16, 2026+3+00
Total+177+5422
  • 1#1 on the list
  • 7Rank that day
  • Not trending
3
  • GitHub trending history

    Where the repository ranked on GitHub Trending, one dot for each day it was on a list

    Jul 19 – Jul 20, 2026Best #3

    #1#5#10#15#20#25
    Top 10
    Aug 2025OctDecFebAprJun
    The left and right arrow keys step from day to day; Home and End go to the first and the last

    Mentions and achievements

    Where people are talking about kvcache-ai/ktransformers, and every badge it has earned
    1. Nov 141 post
      • Lost it, 10 repos just replaced a whole GPU bill team 10 open-source repos, 331k stars, to kill that bill. tune. rent. fit. serve. scale. meter. 01 unsloth → Fine-tune models in 70% less VRAM 02 dspy…

        Read full on X, posted by @0xKarmi

        @0xKarmi · x.com

    2. 1 month earlier
    3. Oct 101 post
      • China has killed the GPU mafia 🤯 They open-sourced a framework that replaces a $400,000 H100 rack with a single 24GB graphics card. → CPU + GPU hybrid inference → 3-28x faster than llama.cpp → 286…

        Read full on X, posted by @CrazyShyyt

        @CrazyShyyt · x.com

    4. 22 days earlier
    5. Sep 181 post
      • KTransformers v0.7.1 is here 🚀 Now supporting: • Qwen VLM multimodal LoRA fine-tuning in BF16 • Native RAWINT4 LoRA for Kimi K2.5 / K2.6 Fine-tune massive VLM and MoE models with a more…

        Read full on X, posted by @KVCache_AI

        @KVCache_AI · x.com

    6. 17 days earlier
    7. Sep 11 post
      • 🤯 671B MoE. 14GB of VRAM. 16.8 tokens/s decode. That is DeepSeek-V3 Q4_K_M on a single RTX 4090 — not an 8×H100 rack. The hot path stays on the GPU. The giant expert banks live in system RAM. 🆕…

        Read full on X, posted by @EchoGit42

        @EchoGit42 · x.com

    8. Aug 311 post2 posts
      • KTransformers is a framework for running and fine-tuning very large mixture-of-experts language models by dividing work between CPUs and GPUs. ➡️ anyone hear of this before? 19K stars

        Read full on Reddit

        r/LovingAIAgents

      • KTransformers is a framework for running and fine-tuning very large mixture-of-experts language models by dividing work between CPUs and GPUs. ➡️ anyone hear of this before? 19K stars

        Read full on Reddit

        r/LovingOpenSourceAI

    9. Aug 261 post
      • 🚀 KTransformers v0.7.0 is here! This release brings major upgrades for ultra-large MoE fine-tuning: • Native FP8 Expert weights for LoRA, reducing host memory usage for DeepSeek-V3.1 from ~1.4 TB to…

        Read full on X, posted by @KVCache_AI

        @KVCache_AI · x.com

    10. Aug 231 post
      • Engineered with academic rigor (accepted at SOSP 2025) for practical local inference: • 1.5x–4x Decode Speedup: Delivers substantial boosts in prefill throughput and 1.5x to 4x speedups during the…

        Read full on X, posted by @TheTechDiggest

        @TheTechDiggest · x.com

    11. 1 month earlier
    12. Jul 201 post
      • 🚨 CVE-2026-63767 — CVSS 9.8/10 ██████████ ktransformers through 0.6.3, fixed in commit def0f93, contains an unauthenticated pickle deserialization vulnerability... Severity: CRITICAL Patch now.…

        Read full on X, posted by @OrizonCyber

        @OrizonCyber · x.com

    13. Jul 191 post
      • SOMEONE BUILT A CPU-GPU FRAMEWORK FOR GIANT MOE MODELS KTransformers runs heterogeneous LLM inference and fine-tuning so massive Mixture-of-Experts models can use CPU memory plus limited GPU VRAM.…

        Read full on X, posted by @ValencianaAbel

        @ValencianaAbel · x.com

    You might also like

    Repositories close to kvcache-ai/ktransformers
    FeaturedAdvertise to 180K+ monthly visitors
    Bifrost

    Fastest Enterprise AI Gateway for Routing, Governing, and Securing all your AI Traffic

    Websitebench

    Can AI agents reverse-engineer real websites?

    Busbar

    Control what AI can do before it acts

    Omnigraph

    Versioned graph database on object storage. Git-style workflows and multimodal retrieval

    expert-kit/expert-kit

    Rust

    Expert Kit is an efficient foundation of Expert Parallelism (EP) for MoE model Inference on heterogenous hardware

    6619

    scaleapi/llm-engine

    Python

    Scale LLM Engine public repository

    842100

    inferflow/inferflow

    C++

    Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).

    25324

    OptimalScale/LMFlow

    Python

    An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

    8.5k824

    hpcaitech/ColossalAI

    Python

    Making large AI models cheaper, faster and more accessible

    41.4k4.6k

    intel/intel-extension-for-transformers

    Python

    ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡

    2.2k215

    vllm-project/vllm

    Python

    A high-throughput and memory-efficient inference and serving engine for LLMs

    97k24.7k

    intel/ipex-llm

    Python

    Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.

    8.8k1.4k

    harleyszhang/rapid_llm

    Python

    RapidLLM is an LLM inference framework with continuous batching, tensor and data parallelism, quantized weights, and with pluggable Triton/CUDA kernels.

    19833