agentic-learning-ai-lab/deleting-weights

𝗗𝗲𝗹𝗲𝘁𝗶𝗼𝗻 𝗧𝘂𝗻𝗶𝗻𝗴: 𝗖𝗮𝗻 𝘄𝗲 𝗽𝗼𝘀𝘁-𝘁𝗿𝗮𝗶𝗻 𝗮𝗻 𝗟𝗟𝗠 𝗯𝘆 𝗼𝗻𝗹𝘆 𝗱𝗲𝗹𝗲𝘁𝗶𝗻𝗴 𝘄𝗲𝗶𝗴𝗵𝘁𝘀? Full fine-tuning updates pretrained parameters. LoRA learns a low-rank…

📄 The Surprising Effectiveness of Deleting Weights in LLM Post-Training by Jack Lu (@Jacklu_me), Zhenbang Yang et al. A mask-only post-training method that deletes selected gate weights instead of…

How little of an LLM do you need to change to teach it something new? Almost nothing! ✂ï¸� "The Surprising Effectiveness of Deleting Weights in LLM Post-Training" This NYU and University of Calgary…

Verdict: Solid result, will likely replicate. Rigorous forgetting evaluation and geometric analysis support claims. Scope limited to three regimes; broader task diversity remains open.

So post-training can work by deletion alone, and that opens up a lot! → Masks show exactly which weights each skill deletes, opening up interpretability and skill merging. → They're small and binary:…