ChatGPT turned four this year, and in that blink of an eye, machines went from awkward autocomplete to conversational partners fluent enough to pass as human. But Michael C. Frank, a cognitive scientist at Stanford, puts the achievement in stark perspective: “We still have to burn down a forest and scrape the entire sum of all human knowledge to re-create this milestone that happens in our living rooms over the course of a year.”
That chasm between how kids and computers acquire language has a name—the data efficiency gap. A linguistically rich preteen might hear roughly 100 million words. Add reading and you get maybe 300 million by age 20. Meta’s Llama 3.1, by contrast, chewed through 15 trillion tokens in pretraining. Ethan Gotlieb Wilcox at Georgetown puts it in terms anyone can grasp: Claude has seen as much language as an entire city experiences in a generation. Print out an LLM’s training data and the stack reaches past the International Space Station. A kid’s total? About 20 meters.
And parents, take note: toddlers typically start producing grammatically correct sentences after hearing just 10 to 30 million words. Frank is blunt about what that means for the state of the art: “If you train GPT-2 on 30 million words, you get a nonsense generator; you don’t get a kid.”
The stakes here go beyond scientific curiosity. Frontier models keep improving by getting bigger, but the internet is finite—Wilcox suggests easily available training data could run dry as early as the 2030s. If researchers can reverse-engineer how children learn so efficiently, the payoff would be models that train on far less data, AI that works for minority languages with scant digital footprints, and maybe answers to older questions about whether language is hardwired into human biology or learned purely from experience. The gap between a babbling toddler and a trillion-token model isn’t just interesting—it’s a roadmap for what comes after the data well dries up.
