ChatGPT didn’t come out of nowhere. It sits on top of a training stack that the industry has been quietly assembling for years — and the acronyms everyone’s suddenly throwing around have specific, documented histories.
Instruction Fine-Tuning (IFT) is what gets a base language model to actually follow directions instead of just predicting the next token. The data for this comes from a spectrum: on one end, purely model-generated datasets like Unnatural Instructions from Honovich et al.; on the other, massive community efforts like Super-Natural Instructions. In between sit bootstrapping approaches like Self-Instruct, which start with a small set of high-quality human examples and prompt the model to generate more.
But instruction-following alone doesn’t make a model safe. That’s where Supervised Fine-Tuning (SFT) comes in — fine-tuning on human-annotated data specifically for helpfulness and harmlessness. The Sparrow paper’s appendix shows exactly what this looks like in practice. Google’s LaMDA took a similar route, fine-tuning on dialog data with safety annotations tied to explicit rules.
Here’s the thing most coverage misses: ChatGPT wasn’t first. Meta shipped BlenderBot, Google had LaMDA, DeepMind built Sparrow, and Anthropic’s Assistant (now Claude) all predate OpenAI’s public release. What OpenAI did differently wasn’t inventing the techniques — it was packaging them into a product people could actually use. The research community had been publishing these methods for months. The gap between paper and product is where the real story lives.
