karpathy/minbpe
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
Python
10.5k
1k
10 contributors
MIT License
Repository activities
repository's daily and monthly activities across stars, forks, merged PRs, issues, and closed issues