avbiswas/text-albumentations

My next video is about training JEV from scratch. The first task is to generate SYNTHETIC TRAINING DATA. I already have a library called text-albumentations that uses constrained decoding to generate…

Recently trained a tiny 135M LM through - Continued Pretraining (CPT) - Supervised Finetuning (SFT) - Preference Optimization (DPO) - Reinforcement Learning (GRPO) Training data gen, Unsloth, HF,…

Btw Joel is the author of the great huggingface article on the Synthetic Data Playbook. It's a marathon survey everyone should read. This is the work that ultimately inspired my text-albumentations…

Watch this 35 min visual guide on post-training Tiny Language Models. How to prepare preference datasets, finetune LMs to pick better trajectories, and evaluate their diversity + quality. Training…

The paper-instructions dataset now comes with a subset of reasoning traces This is an awesome training dataset, curated with deepseek-v4-flash and qwen3.6-35B-A3B using text-albumentations. Costed me…