
JANUS explores fine-tune-then-rectify: Jacobian projection to mitigate catastrophic forgetting (opens in a new tab)
Tsinghua and DiDi Autonomous Driving researchers propose JANUS, using Jacobian null-space projection and multi-step adaptive rectification after fine-tuning. Retention metrics improve on tested 7B/8B models; the necessary and sufficient condition concerns local first-order loss changes.
