TilelliLab/Tilelli-llm

A kernel library written in tilelang

Project Titania is a complete large language model system, from transformer to transistor, simple enough for one person to understand.

A ternary, zero-heap tiny language model that runs inside a $2 microcontroller — bit-exact Python <-> C99 <-> Cortex-M3 (QEMU) parity. Apache-2.0.

A 124.6M LLM trained from scratch on 13B tokens on a single RTX 4090 — tokenizer, pretraining, post-training, evaluation, and reproducible inference.

This project aims to replicate mainstream open-source model architectures with limited computational resources, implementing mini models with 100-200M parameters.

LLM that can be trained on 1 or more GPUs for research.

🧠 Train a 64M-parameter LLM from scratch in just 2h!

Experimental playground for benchmarking language model (LM) architectures, layers, and tricks on smaller datasets. Designed for flexible experimentation and exploration.