ORPO: Monolithic Preference Optimization without Reference Model (Paper Explained)
Yannic Kilcher · 33:25 · Watch on YouTube
ORPO (Monolithic Preference Optimization without Reference Model, from KAIST AI) does supervised fine-tuning and preference alignment in one training step instead of two. It adds an odds-ratio loss to the standard SFT...