Chapter 10 · From Base Model to Assistant
The DPO Subtraction
Follow chosen and rejected sequence probabilities through a reference-relative preference loss.
Try it · toy model
10 min
Read the concepts behind it
Chapter 10 · From Base Model to Assistant
Follow chosen and rejected sequence probabilities through a reference-relative preference loss.
Try it · toy model
Read the concepts behind it