Developer releases minimal LLM post-training toolkit for 8GB GPU; proves reinforcement learning forgets less than supervised fine-tuning, amplifies DeepSeek-R1-style reasoning on 3B model in 5 hours for under $5.

GitHub - pochenai/nano-llm-posttraining: Minimal, readable LLM post-training experiments on one 8GB GPU. Measures forgetting, seed variance, and RL emergence.

Minimal LLM Post-Training on an 8GB GPU: Understanding KL, SFT, DPO, GRPO and DeepSeek-Style Reasoning with Open-Source Frameworks Using open-source training frameworks (HuggingFace TRL) and minimal, reproducible experiments to see — one by one — what SFT, DPO and GRPO each change, how RL drifts less than SFT (measured by KL), and how GRPO amplifies DeepSeek-R1-style reasoning. TL;DR Built on HuggingFace TRL, in under 100 lines of core code you can run the whole SFT + DPO + GRPO post-training pi...