Supervised fine-tuning gets you a model that answers correctly yet still manages to be tactless, wordy, or too eager to please. The last polish comes from preference tuning, and RLHF vs DPO is the choice of route: one trains a reward model and runs reinforcement learning, the other folds the whole objective into a single loss. This post explains both in plain words and runs the DPO loss for real in Python. “A model does not learn what you tell ... Read More

A large language model is never sat down and taught facts. It learns by predicting the next token across a mountain of text, then gets reshaped twice more until it behaves like the assistant you talk to. That is how LLMs are trained: pretraining, supervised fine-tuning, and preference tuning. This post walks all three stages in plain words, with small Python you can run yourself. “The computer was born to solve problems that did not exist before.” Bill Gates Last ... Read More